splime 0.4.4__tar.gz → 0.4.5__tar.gz

{splime-0.4.4/src/splime.egg-info → splime-0.4.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: splime
-Version: 0.4.4
+Version: 0.4.5
 Summary: Reuse Python functions across projects without rewriting or redeploying them.
 Author-email: Yastrebov Kirill <yastrebovks@gmail.com>
 License-Expression: Apache-2.0
@@ -76,11 +76,14 @@ redeploying.
 ## Requirements
 
 - Python **3.13+**
+- POSIX for local daemon environment builds and timeout-safe worker execution
+  in 0.4.5 (Windows Job Object support is not yet implemented; Windows
+  client-only use is unaffected)
 
 ## Install
 
 ```bash
-pip install splime
+python3.13 -m pip install "splime==0.4.5"
 ```
 
 The distribution is named `splime`; the Python import package is `spl`.
@@ -119,7 +122,7 @@ it by name and get back the value (plus logs and any artifacts).
 
 The same `call` becomes a remote run when you point it at a library, an owner, or a
 target machine. This requires a connected splime server and a private worker; the local
-daemon builds an isolated environment on the worker before executing.
+daemon builds the declared dependency environment on the worker before executing.
 
 ```python
 client = SPLClient(user_token="…", machine_token="…")   # connect the daemon to your server
@@ -157,10 +160,38 @@ splime runs code that you publish on purpose, on machines you control. It is bui
 
 - **explicit ownership** of every published object,
 - **scoped access** grants per library (read metadata, read objects, execute),
-- **private worker boundaries** — the server coordinates, your own workers execute,
-- **isolated environments** built by the daemon before a run,
+- **private worker topology** — the server coordinates, your own workers execute,
+- **dependency environments** built by the daemon before a run,
+- **metadata-only central telemetry by default** — local inputs, results,
+  error details, streams, and artifact bodies stay on the machine; diagnostic
+  adds only redacted/truncated error and stdout/stderr text, while full also
+  opts in to redacted inputs, results, and supported text artifacts,
 - an **auditable run history**.
 
+Credentialed central-server traffic requires a direct HTTPS endpoint and never
+follows redirects. Loopback HTTP remains available for local development; the
+only other plaintext path is the exact Docker callback carrying only a scoped
+run capability.
+
+Native and `venv-subprocess` runtimes execute trusted code under the
+conductor's OS identity—the daemon user for daemon-managed runs. A virtual
+environment separates dependencies and a subprocess separates execution, but
+neither is an OS sandbox. Docker or a deliberately separate OS identity is
+required when code must not read same-UID daemon files. Docker provides the
+configured process/filesystem boundary, subject to its mounts and network
+options and to trust in the Docker daemon and host.
+
+Scoped callback capabilities limit the authority intentionally passed over the
+worker protocol. They do not protect against arbitrary same-UID file reads and
+do not turn a native or virtual-environment worker into a sandbox.
+
+Docker object runs use one container per run by default and mount only that
+run's writable directory. Warm pooling is explicitly opt-in with
+`spl-daemon serve --docker-pool-enabled --docker-pool-size N`; pooled
+containers share all runs for that daemon and are suitable only for
+single-tenant, mutually trusting workloads. One process also exclusively locks
+each daemon home before opening its database or publishing its endpoint.
+
 ## How it fits together
 
 | Piece | What it does |

{splime-0.4.4 → splime-0.4.5}/README.md

@@ -21,11 +21,14 @@ redeploying.
 ## Requirements
 
 - Python **3.13+**
+- POSIX for local daemon environment builds and timeout-safe worker execution
+  in 0.4.5 (Windows Job Object support is not yet implemented; Windows
+  client-only use is unaffected)
 
 ## Install
 
 ```bash
-pip install splime
+python3.13 -m pip install "splime==0.4.5"
 ```
 
 The distribution is named `splime`; the Python import package is `spl`.
@@ -64,7 +67,7 @@ it by name and get back the value (plus logs and any artifacts).
 
 The same `call` becomes a remote run when you point it at a library, an owner, or a
 target machine. This requires a connected splime server and a private worker; the local
-daemon builds an isolated environment on the worker before executing.
+daemon builds the declared dependency environment on the worker before executing.
 
 ```python
 client = SPLClient(user_token="…", machine_token="…")   # connect the daemon to your server
@@ -102,10 +105,38 @@ splime runs code that you publish on purpose, on machines you control. It is bui
 
 - **explicit ownership** of every published object,
 - **scoped access** grants per library (read metadata, read objects, execute),
-- **private worker boundaries** — the server coordinates, your own workers execute,
-- **isolated environments** built by the daemon before a run,
+- **private worker topology** — the server coordinates, your own workers execute,
+- **dependency environments** built by the daemon before a run,
+- **metadata-only central telemetry by default** — local inputs, results,
+  error details, streams, and artifact bodies stay on the machine; diagnostic
+  adds only redacted/truncated error and stdout/stderr text, while full also
+  opts in to redacted inputs, results, and supported text artifacts,
 - an **auditable run history**.
 
+Credentialed central-server traffic requires a direct HTTPS endpoint and never
+follows redirects. Loopback HTTP remains available for local development; the
+only other plaintext path is the exact Docker callback carrying only a scoped
+run capability.
+
+Native and `venv-subprocess` runtimes execute trusted code under the
+conductor's OS identity—the daemon user for daemon-managed runs. A virtual
+environment separates dependencies and a subprocess separates execution, but
+neither is an OS sandbox. Docker or a deliberately separate OS identity is
+required when code must not read same-UID daemon files. Docker provides the
+configured process/filesystem boundary, subject to its mounts and network
+options and to trust in the Docker daemon and host.
+
+Scoped callback capabilities limit the authority intentionally passed over the
+worker protocol. They do not protect against arbitrary same-UID file reads and
+do not turn a native or virtual-environment worker into a sandbox.
+
+Docker object runs use one container per run by default and mount only that
+run's writable directory. Warm pooling is explicitly opt-in with
+`spl-daemon serve --docker-pool-enabled --docker-pool-size N`; pooled
+containers share all runs for that daemon and are suitable only for
+single-tenant, mutually trusting workloads. One process also exclusively locks
+each daemon home before opening its database or publishing its endpoint.
+
 ## How it fits together
 
 | Piece | What it does |

{splime-0.4.4 → splime-0.4.5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "splime"
-version = "0.4.4"
+version = "0.4.5"
 
 description = "Reuse Python functions across projects without rewriting or redeploying them."
 readme = "README.md"

{splime-0.4.4 → splime-0.4.5}/src/spl/_client.py

@@ -20,6 +20,7 @@ from __future__ import annotations
 
 import builtins
 import re
+import warnings
 from collections.abc import Mapping
 from dataclasses import dataclass, field, replace
 from html import escape
@@ -735,6 +736,30 @@ class RemoteRun:
                 )
         return downloaded
 
+    def _acknowledge_transient_delivery(self) -> None:
+        """Best-effort additive handshake after terminal data is consumed."""
+
+        if self.server_side:
+            return
+        try:
+            self._client._daemon.acknowledge_run_delivery(self.id)
+        except ClientError as exc:
+            if exc.status_code in {404, 405}:
+                # A 0.4.5 client can still collect from a 0.4.4 daemon.  That
+                # daemon does not enforce transient cleanup, so no data is at risk.
+                return
+            warnings.warn(
+                f"run {self.id!r} was delivered, but its retention acknowledgment failed: {exc}",
+                RuntimeWarning,
+                stacklevel=2,
+            )
+        except Exception as exc:
+            warnings.warn(
+                f"run {self.id!r} was delivered, but its retention acknowledgment failed: {exc}",
+                RuntimeWarning,
+                stacklevel=2,
+            )
+
     def collect(
         self,
         *,
@@ -753,18 +778,25 @@ class RemoteRun:
             timeout_seconds=timeout_seconds,
             progress=progress,
         )
-        if final_state["status"] != "succeeded":
+        status = str(final_state["status"])
+        transient = m_manifest.retention_disposition(final_state.get("keep", True), status) == "remove"
+        if status != "succeeded":
             error = final_state.get("error") or "run returned no error message"
+            if transient:
+                self._acknowledge_transient_delivery()
             raise RuntimeError(f"{self.mode} run {self.id!r} ended as {final_state.get('status')!r}: {error}")
 
         payload = self.result()
         downloaded = self.download_artifacts(artifacts_dir) if artifacts_dir is not None else {}
-        return RemoteResult(
+        result = RemoteResult(
             run=final_state,
             payload=payload,
             mode=self.mode,
             downloaded_artifacts=downloaded,
         )
+        if transient:
+            self._acknowledge_transient_delivery()
+        return result
 
 
 class _LibraryAdmin:

splime-0.4.5/src/spl/_http.py

@@ -0,0 +1,289 @@
+"""Safe HTTP helpers shared by local-daemon and central-server clients."""
+
+from __future__ import annotations
+
+import http.client
+import logging
+import ssl
+from functools import lru_cache, partial
+from typing import Any
+from urllib.error import HTTPError
+from urllib.parse import urlsplit
+from urllib.request import (
+    HTTPHandler,
+    HTTPRedirectHandler,
+    HTTPSHandler,
+    ProxyHandler,
+    Request,
+    build_opener,
+)
+
+DEFAULT_CONNECT_TIMEOUT_SECONDS = 10.0
+DEFAULT_HTTP_TIMEOUT_SECONDS = 60.0
+DEFAULT_FILE_TRANSFER_TIMEOUT_SECONDS = 300.0
+
+_LOOPBACK_HTTP_HOSTS = frozenset({"127.0.0.1", "::1", "localhost"})
+_DOCKER_CALLBACK_HTTP_HOST = "host.docker.internal"
+_DOCKER_CALLBACK_PATH = "/remote-nodes/run"
+_CREDENTIAL_HEADER_NAMES = frozenset(
+    {
+        "authorization",
+        "cookie",
+        "proxy-authorization",
+        "x-spl-claim",
+        "x-spl-user-token",
+    }
+)
+_LOGGER = logging.getLogger(__name__)
+
+
+class InsecureCredentialTransportError(ValueError):
+    """Reject a credentialed request that would use unsafe plaintext HTTP."""
+
+
+class _NoRedirectHandler(HTTPRedirectHandler):
+    """Surface redirects without replaying a request or any of its headers."""
+
+    def redirect_request(
+        self,
+        request: Request,
+        file_pointer: Any,
+        code: int,
+        message: str,
+        headers: Any,
+        new_url: str,
+    ) -> Request | None:
+        del message, new_url
+        raise HTTPError(
+            request.full_url,
+            code,
+            (
+                f"HTTP {code} redirect refused: SPL HTTP requests do not follow redirects; "
+                "configure the client with the final HTTPS endpoint instead"
+            ),
+            headers,
+            file_pointer,
+        )
+
+
+@lru_cache(maxsize=1)
+def _warn_loopback_http_once() -> None:
+    _LOGGER.warning(
+        "SPL is sending credentials over plaintext HTTP to an explicit loopback development target; "
+        "use HTTPS outside local development"
+    )
+
+
+@lru_cache(maxsize=1)
+def _warn_docker_callback_http_once() -> None:
+    _LOGGER.warning(
+        "SPL is sending a run-scoped callback capability over plaintext HTTP to the local Docker host bridge"
+    )
+
+
+def _credential_header_names(request: Request) -> set[str]:
+    return {
+        name.casefold()
+        for name, value in request.header_items()
+        if value and name.casefold() in _CREDENTIAL_HEADER_NAMES
+    }
+
+
+def _authorization_bearer_token(request: Request) -> str | None:
+    """Return one syntactically valid bearer token without logging it."""
+
+    authorization = next(
+        (value for name, value in request.header_items() if name.casefold() == "authorization"),
+        "",
+    )
+    scheme, separator, token = authorization.partition(" ")
+    if scheme.casefold() != "bearer" or not separator or not token:
+        return None
+    return token
+
+
+def _validate_transport(
+    request: Request,
+    *,
+    allow_docker_callback_http: bool,
+    contains_credentials: bool,
+) -> bool:
+    """Validate one request and return whether environment proxies must be bypassed."""
+
+    parsed = urlsplit(request.full_url)
+    scheme = parsed.scheme.casefold()
+    host = (parsed.hostname or "").casefold()
+    credential_headers = _credential_header_names(request)
+    url_contains_credentials = parsed.username is not None or parsed.password is not None
+    is_credentialed = contains_credentials or bool(credential_headers) or url_contains_credentials
+
+    if scheme not in {"http", "https"} or not host:
+        raise ValueError("SPL HTTP request URLs must use an absolute http:// or https:// URL with a host")
+
+    if allow_docker_callback_http:
+        # Import lazily to keep the generic transport module out of the
+        # daemon package's compatibility-import cycle while sharing the exact
+        # token namespace with the authority that mints it.
+        from spl.daemon.callback_capability import CALLBACK_CAPABILITY_PREFIX
+
+        bearer_token = _authorization_bearer_token(request)
+        docker_callback_is_valid = (
+            scheme == "http"
+            and host == _DOCKER_CALLBACK_HTTP_HOST
+            and parsed.path == _DOCKER_CALLBACK_PATH
+            and not parsed.query
+            and not parsed.fragment
+            and request.get_method().upper() == "POST"
+            and bearer_token is not None
+            and bearer_token.startswith(CALLBACK_CAPABILITY_PREFIX)
+            and len(bearer_token) > len(CALLBACK_CAPABILITY_PREFIX)
+            and credential_headers == {"authorization"}
+            and not contains_credentials
+        )
+        if not docker_callback_is_valid:
+            raise InsecureCredentialTransportError(
+                "the Docker plaintext callback exception is limited to a POST authenticated with a "
+                "Bearer splcb_... run callback capability to "
+                "http://host.docker.internal/remote-nodes/run without a query or any other credential"
+            )
+        _warn_docker_callback_http_once()
+        return True
+
+    is_loopback = host in _LOOPBACK_HTTP_HOSTS
+    if scheme == "http" and is_credentialed:
+        if not is_loopback:
+            raise InsecureCredentialTransportError(
+                "credentialed SPL HTTP requests require HTTPS; plaintext is allowed only for the explicit "
+                "loopback development hosts 127.0.0.1, ::1, and localhost"
+            )
+        _warn_loopback_http_once()
+
+    return is_loopback
+
+
+class ConnectionPhaseError(OSError):
+    """Wrap a failure proven to occur while establishing the connection."""
+
+    def __init__(self, cause: OSError):
+        self.cause = cause
+        super().__init__(str(cause))
+
+
+class _PhaseAwareHTTPConnection(http.client.HTTPConnection):
+    def __init__(
+        self,
+        host: str,
+        *,
+        read_timeout: float | None,
+        **kwargs: Any,
+    ) -> None:
+        self._spl_read_timeout = read_timeout
+        super().__init__(host, **kwargs)
+
+    def connect(self) -> None:
+        try:
+            super().connect()
+        except OSError as exc:
+            raise ConnectionPhaseError(exc) from exc
+        if self.sock is not None:
+            self.sock.settimeout(self._spl_read_timeout)
+
+
+class _PhaseAwareHTTPSConnection(http.client.HTTPSConnection):
+    def __init__(
+        self,
+        host: str,
+        *,
+        read_timeout: float | None,
+        **kwargs: Any,
+    ) -> None:
+        self._spl_read_timeout = read_timeout
+        super().__init__(host, **kwargs)
+
+    def connect(self) -> None:
+        try:
+            # HTTPSConnection.connect includes TCP connect, any proxy tunnel,
+            # and the TLS handshake. No application request has been sent yet.
+            super().connect()
+        except OSError as exc:
+            raise ConnectionPhaseError(exc) from exc
+        if self.sock is not None:
+            self.sock.settimeout(self._spl_read_timeout)
+
+
+class _SplitTimeoutHTTPHandler(HTTPHandler):
+    def __init__(self, *, read_timeout: float | None) -> None:
+        super().__init__()
+        self._spl_read_timeout = read_timeout
+
+    def http_open(self, request: Request) -> Any:
+        connection = partial(
+            _PhaseAwareHTTPConnection,
+            read_timeout=self._spl_read_timeout,
+        )
+        return self.do_open(connection, request)
+
+
+class _SplitTimeoutHTTPSHandler(HTTPSHandler):
+    def __init__(
+        self,
+        *,
+        context: ssl.SSLContext,
+        read_timeout: float | None,
+    ) -> None:
+        super().__init__(context=context)
+        self._spl_context = context
+        self._spl_read_timeout = read_timeout
+
+    def https_open(self, request: Request) -> Any:
+        connection = partial(
+            _PhaseAwareHTTPSConnection,
+            read_timeout=self._spl_read_timeout,
+        )
+        return self.do_open(connection, request, context=self._spl_context)
+
+
+@lru_cache(maxsize=1)
+def verified_https_context() -> ssl.SSLContext:
+    """Return a verifying context with environment/system and certifi roots."""
+
+    import certifi
+
+    context = ssl.create_default_context()
+    context.load_verify_locations(cafile=certifi.where())
+    return context
+
+
+def urlopen_verified(
+    request: Request,
+    *,
+    timeout: float | None = DEFAULT_HTTP_TIMEOUT_SECONDS,
+    connect_timeout: float | None = DEFAULT_CONNECT_TIMEOUT_SECONDS,
+    allow_docker_callback_http: bool = False,
+    contains_credentials: bool = False,
+) -> Any:
+    """Open a policy-checked request with verified TLS and split timeout budgets.
+
+    ``contains_credentials`` marks authentication material carried in a body
+    instead of a recognized header. ``allow_docker_callback_http`` is a narrow
+    exception for the run-scoped worker callback route only.
+    """
+
+    bypass_environment_proxies = _validate_transport(
+        request,
+        allow_docker_callback_http=allow_docker_callback_http,
+        contains_credentials=contains_credentials,
+    )
+
+    handlers: list[Any] = [
+        _NoRedirectHandler(),
+        _SplitTimeoutHTTPHandler(read_timeout=timeout),
+        _SplitTimeoutHTTPSHandler(
+            context=verified_https_context(),
+            read_timeout=timeout,
+        ),
+    ]
+    if bypass_environment_proxies:
+        handlers.insert(0, ProxyHandler({}))
+    opener = build_opener(*handlers)
+    return opener.open(request, timeout=connect_timeout)

splime-0.4.5/src/spl/_process.py

@@ -0,0 +1,128 @@
+"""Bounded subprocess execution with POSIX process-tree termination."""
+
+from __future__ import annotations
+
+import os
+import signal
+import subprocess
+import time
+from collections.abc import Mapping, Sequence
+from pathlib import Path
+from typing import IO
+
+from spl._timeout import TimeoutDomain, validate_timeout_seconds
+
+DEFAULT_TERMINATION_GRACE_SECONDS = 0.5
+WINDOWS_TIMEOUT_UNSUPPORTED = (
+    "finite subprocess timeouts require Windows Job Object support; "
+    "timed worker execution is currently supported only on POSIX"
+)
+
+
+def run_process_tree(
+    command: Sequence[str],
+    *,
+    cwd: Path | None = None,
+    env: Mapping[str, str] | None = None,
+    timeout: float | None = None,
+    termination_grace_seconds: float = DEFAULT_TERMINATION_GRACE_SECONDS,
+    stdout_target: int | IO[str] | None = subprocess.PIPE,
+    stderr_target: int | IO[str] | None = subprocess.PIPE,
+) -> subprocess.CompletedProcess[str]:
+    """Run ``command`` and terminate its complete POSIX process group on timeout.
+
+    A finite timeout fails closed on non-POSIX systems. Windows process groups
+    do not contain arbitrary descendants; equivalent support requires assigning
+    the child to a Job Object at creation time.
+    """
+
+    validate_timeout_seconds(
+        timeout,
+        name="process timeout",
+        domain=TimeoutDomain.FINITE,
+        allow_none=True,
+    )
+    validate_timeout_seconds(
+        termination_grace_seconds,
+        name="termination_grace_seconds",
+        domain=TimeoutDomain.NON_NEGATIVE,
+        allow_none=False,
+    )
+    process_groups_supported = _posix_process_groups_supported()
+    if timeout is not None and not process_groups_supported:
+        raise RuntimeError(WINDOWS_TIMEOUT_UNSUPPORTED)
+
+    process = subprocess.Popen(
+        list(command),
+        cwd=cwd,
+        env=env,
+        text=True,
+        stdout=stdout_target,
+        stderr=stderr_target,
+        start_new_session=process_groups_supported,
+    )
+    try:
+        stdout_text, stderr_text = process.communicate(timeout=timeout)
+    except subprocess.TimeoutExpired as exc:
+        stdout_text, stderr_text = _terminate_timed_out_process(
+            process,
+            grace_seconds=termination_grace_seconds,
+        )
+        assert timeout is not None
+        raise subprocess.TimeoutExpired(
+            command,
+            timeout,
+            output=stdout_text,
+            stderr=stderr_text,
+        ) from exc
+
+    return subprocess.CompletedProcess(
+        list(command),
+        process.returncode,
+        stdout=stdout_text,
+        stderr=stderr_text,
+    )
+
+
+def _terminate_timed_out_process(
+    process: subprocess.Popen[str],
+    *,
+    grace_seconds: float,
+) -> tuple[str, str]:
+    """Terminate, force-kill, and reap one timed-out POSIX process group."""
+
+    _signal_process_group(process.pid, signal.SIGTERM)
+    deadline = time.monotonic() + grace_seconds
+    while time.monotonic() < deadline:
+        if not _process_group_exists(process.pid):
+            break
+        time.sleep(min(0.01, max(0.0, deadline - time.monotonic())))
+    _signal_process_group(process.pid, signal.SIGKILL)
+    stdout, stderr = process.communicate()
+    return stdout or "", stderr or ""
+
+
+def _posix_process_groups_supported() -> bool:
+    return os.name == "posix"
+
+
+def _process_group_exists(process_group_id: int) -> bool:
+    try:
+        os.killpg(process_group_id, 0)
+    except ProcessLookupError:
+        return False
+    except PermissionError:
+        return True
+    return True
+
+
+def _signal_process_group(process_group_id: int, sig: signal.Signals) -> None:
+    try:
+        os.killpg(process_group_id, sig)
+    except ProcessLookupError:
+        return
+    except PermissionError:
+        # macOS can report EPERM for a process group containing only a zombie
+        # leader between termination and the final wait. The leader is still
+        # reaped by ``communicate`` below; a live same-user group is killable.
+        return

splime-0.4.5/src/spl/_timeout.py

@@ -0,0 +1,49 @@
+"""Shared validation for timeout values that must be safe for runtime APIs."""
+
+from __future__ import annotations
+
+import math
+from enum import StrEnum
+
+
+class TimeoutDomain(StrEnum):
+    """Supported numeric domains for timeout settings."""
+
+    FINITE = "finite"
+    NON_NEGATIVE = "non-negative"
+    POSITIVE = "positive"
+
+
+def validate_timeout_seconds(
+    value: object,
+    *,
+    name: str,
+    domain: TimeoutDomain,
+    allow_none: bool,
+) -> float | None:
+    """Return ``value`` as a finite float when it satisfies ``domain``."""
+
+    requirement = {
+        TimeoutDomain.FINITE: "a finite number",
+        TimeoutDomain.NON_NEGATIVE: "a finite non-negative number",
+        TimeoutDomain.POSITIVE: "positive and finite",
+    }[domain]
+    suffix = " or None" if allow_none else ""
+    if value is None:
+        if allow_none:
+            return None
+        raise ValueError(f"{name} must be {requirement}")
+    if isinstance(value, bool) or not isinstance(value, (int, float)):
+        raise ValueError(f"{name} must be {requirement}{suffix}")
+    try:
+        normalized = float(value)
+    except (OverflowError, ValueError) as exc:
+        raise ValueError(f"{name} must be {requirement}{suffix}") from exc
+    invalid = (
+        not math.isfinite(normalized)
+        or (domain is TimeoutDomain.NON_NEGATIVE and normalized < 0)
+        or (domain is TimeoutDomain.POSITIVE and normalized <= 0)
+    )
+    if invalid:
+        raise ValueError(f"{name} must be {requirement}{suffix}")
+    return normalized