juniper-data-client 0.4.1__tar.gz → 0.5.0__tar.gz

juniper_data_client-0.5.0/PKG-INFO

@@ -0,0 +1,119 @@
+Metadata-Version: 2.4
+Name: juniper-data-client
+Version: 0.5.0
+Summary: HTTP client for the JuniperData dataset generation service
+Author: Paul Calnon
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/pcalnon/juniper-data-client
+Project-URL: Repository, https://github.com/pcalnon/juniper-data-client
+Project-URL: Documentation, https://github.com/pcalnon/juniper-data-client#readme
+Project-URL: Issues, https://github.com/pcalnon/juniper-data-client/issues
+Keywords: juniper,juniper-ml,juniper-ai,artificial-intelligence,dataset,machine-learning,api-client
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: requests>=2.28.0
+Requires-Dist: urllib3>=2.0.0
+Provides-Extra: test
+Requires-Dist: pytest>=7.0.0; extra == "test"
+Requires-Dist: pytest-cov>=4.0.0; extra == "test"
+Requires-Dist: pytest-timeout>=2.2.0; extra == "test"
+Requires-Dist: responses>=0.23.0; extra == "test"
+Requires-Dist: juniper-observability>=0.1.1; extra == "test"
+Provides-Extra: observability
+Requires-Dist: juniper-observability>=0.1.1; extra == "observability"
+Provides-Extra: dev
+Requires-Dist: juniper-data-client[test]; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: isort>=5.12.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: flake8>=7.0.0; extra == "dev"
+Requires-Dist: types-requests>=2.28.0; extra == "dev"
+Dynamic: license-file
+
+# juniper-data-client
+
+[![PyPI](https://img.shields.io/pypi/v/juniper-data-client)](https://pypi.org/project/juniper-data-client/)
+[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-blue)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)
+
+**Python HTTP client for the juniper-data dataset-generation service.**
+
+`juniper-data-client` is a thin, synchronous HTTP client for fetching datasets from a `juniper-data`
+instance. It exposes a single `JuniperDataClient` class whose methods correspond to the REST surface:
+generator schemas, dataset creation, tag-based filtering, batch operations, the named-version
+registry, and download of the resulting NPZ artifacts as ready-to-train `numpy` arrays. It is the
+canonical dataset-fetch interface used by `juniper-cascor` (training) and `juniper-canopy`
+(visualisation), and it reads the platform's shared `X_train` / `y_train` / `X_val` / `y_val` /
+`X_test` / `y_test` NPZ schema (all `float32`). `val` is
+presence-conditional: `validate_npz_contract` skips a split the artifact does not
+carry. Fake producers emit it (#187); live artifacts may omit it.
+
+> **Part of the Juniper platform.** juniper-data-client is the Python HTTP client for the juniper-data
+> dataset service in [Juniper](https://github.com/pcalnon/juniper-ml) — a multi-package ML research
+> platform built around constructive (Cascade-Correlation) and recurrent neural networks. Point it at
+> any running juniper-data instance.
+
+## Install
+
+```bash
+pip install juniper-data-client
+```
+
+Optional extras: `[observability]` adds X-Request-ID propagation via `juniper-observability`; `[test]`
+and `[dev]` install the test and lint/type-check toolchains.
+
+## Quick start
+
+```python
+from juniper_data_client import JuniperDataClient
+
+with JuniperDataClient("http://localhost:8100") as client:
+    print(client.health_check()["status"])
+
+    result = client.create_spiral_dataset(n_spirals=2, n_points_per_spiral=100, noise=0.1, seed=42)
+    arrays = client.download_artifact_npz(result["dataset_id"])
+
+    print("train:", len(arrays["X_train"]), "test:", len(arrays["X_test"]))
+```
+
+The artifacts come back as a dict of `float32` `numpy` arrays, ready for `torch.from_numpy(...)`.
+
+## API
+
+| Method | Purpose |
+|--------|---------|
+| `health_check()` / `is_ready()` / `wait_for_ready(...)` | Liveness / readiness probes |
+| `list_generators()` / `get_generator_schema(name)` | Discover available generators + their schemas |
+| `create_dataset(generator, params, ...)` | Create a dataset from any generator |
+| `create_spiral_dataset(...)` | Convenience wrapper for the spiral generator |
+| `download_artifact_npz(dataset_id)` / `download_artifact_bytes(dataset_id)` | Download as numpy arrays / raw bytes |
+| `get_dataset_metadata(id)` / `get_preview(id, n)` / `delete_dataset(id)` | Inspect / preview / delete |
+| `list_datasets(limit, offset)` | Paginated dataset listing |
+| `list_versions(name)` / `get_latest(name)` | Named-version registry |
+| `batch_create(...)` / `batch_delete(...)` / `batch_update_tags(...)` / `batch_export(...)` | Batch operations |
+
+## Status
+
+**Beta** on PyPI. The current version is shown by the badge above; see [`CHANGELOG.md`](./CHANGELOG.md).
+
+## Documentation
+
+- [`docs/QUICK_START.md`](./docs/QUICK_START.md) — installation and verification guide
+- [`docs/REFERENCE.md`](./docs/REFERENCE.md) — full API reference, configuration, and error model
+
+## License
+
+MIT — see [LICENSE](./LICENSE).

juniper_data_client-0.5.0/README.md

@@ -0,0 +1,74 @@
+# juniper-data-client
+
+[![PyPI](https://img.shields.io/pypi/v/juniper-data-client)](https://pypi.org/project/juniper-data-client/)
+[![Python 3.12+](https://img.shields.io/badge/python-3.12%2B-blue)](https://www.python.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](./LICENSE)
+
+**Python HTTP client for the juniper-data dataset-generation service.**
+
+`juniper-data-client` is a thin, synchronous HTTP client for fetching datasets from a `juniper-data`
+instance. It exposes a single `JuniperDataClient` class whose methods correspond to the REST surface:
+generator schemas, dataset creation, tag-based filtering, batch operations, the named-version
+registry, and download of the resulting NPZ artifacts as ready-to-train `numpy` arrays. It is the
+canonical dataset-fetch interface used by `juniper-cascor` (training) and `juniper-canopy`
+(visualisation), and it reads the platform's shared `X_train` / `y_train` / `X_val` / `y_val` /
+`X_test` / `y_test` NPZ schema (all `float32`). `val` is
+presence-conditional: `validate_npz_contract` skips a split the artifact does not
+carry. Fake producers emit it (#187); live artifacts may omit it.
+
+> **Part of the Juniper platform.** juniper-data-client is the Python HTTP client for the juniper-data
+> dataset service in [Juniper](https://github.com/pcalnon/juniper-ml) — a multi-package ML research
+> platform built around constructive (Cascade-Correlation) and recurrent neural networks. Point it at
+> any running juniper-data instance.
+
+## Install
+
+```bash
+pip install juniper-data-client
+```
+
+Optional extras: `[observability]` adds X-Request-ID propagation via `juniper-observability`; `[test]`
+and `[dev]` install the test and lint/type-check toolchains.
+
+## Quick start
+
+```python
+from juniper_data_client import JuniperDataClient
+
+with JuniperDataClient("http://localhost:8100") as client:
+    print(client.health_check()["status"])
+
+    result = client.create_spiral_dataset(n_spirals=2, n_points_per_spiral=100, noise=0.1, seed=42)
+    arrays = client.download_artifact_npz(result["dataset_id"])
+
+    print("train:", len(arrays["X_train"]), "test:", len(arrays["X_test"]))
+```
+
+The artifacts come back as a dict of `float32` `numpy` arrays, ready for `torch.from_numpy(...)`.
+
+## API
+
+| Method | Purpose |
+|--------|---------|
+| `health_check()` / `is_ready()` / `wait_for_ready(...)` | Liveness / readiness probes |
+| `list_generators()` / `get_generator_schema(name)` | Discover available generators + their schemas |
+| `create_dataset(generator, params, ...)` | Create a dataset from any generator |
+| `create_spiral_dataset(...)` | Convenience wrapper for the spiral generator |
+| `download_artifact_npz(dataset_id)` / `download_artifact_bytes(dataset_id)` | Download as numpy arrays / raw bytes |
+| `get_dataset_metadata(id)` / `get_preview(id, n)` / `delete_dataset(id)` | Inspect / preview / delete |
+| `list_datasets(limit, offset)` | Paginated dataset listing |
+| `list_versions(name)` / `get_latest(name)` | Named-version registry |
+| `batch_create(...)` / `batch_delete(...)` / `batch_update_tags(...)` / `batch_export(...)` | Batch operations |
+
+## Status
+
+**Beta** on PyPI. The current version is shown by the badge above; see [`CHANGELOG.md`](./CHANGELOG.md).
+
+## Documentation
+
+- [`docs/QUICK_START.md`](./docs/QUICK_START.md) — installation and verification guide
+- [`docs/REFERENCE.md`](./docs/REFERENCE.md) — full API reference, configuration, and error model
+
+## License
+
+MIT — see [LICENSE](./LICENSE).

{juniper_data_client-0.4.1 → juniper_data_client-0.5.0}/juniper_data_client/__init__.py

@@ -5,20 +5,27 @@ dataset generation service, used by both JuniperCascor and juniper-canopy.
 """
 
 from juniper_data_client.client import JuniperDataClient, RequestHook
-from juniper_data_client.exceptions import JuniperDataClientError, JuniperDataConfigurationError, JuniperDataConnectionError, JuniperDataNotFoundError, JuniperDataTimeoutError, JuniperDataValidationError
+from juniper_data_client.constants import ContractKind
+from juniper_data_client.contract import validate_npz_contract
+from juniper_data_client.exceptions import JuniperDataClientError, JuniperDataConfigurationError, JuniperDataConnectionError, JuniperDataContractError, JuniperDataNotFoundError, JuniperDataTimeoutError, JuniperDataValidationError
 
-__version__ = "0.4.1"
+__version__ = "0.5.0"
 
 __all__ = [
     "JuniperDataClient",
+    "validate_npz_contract",
     "JuniperDataClientError",
     "JuniperDataConfigurationError",
     "JuniperDataConnectionError",
+    "JuniperDataContractError",
     "JuniperDataNotFoundError",
     "JuniperDataTimeoutError",
     "JuniperDataValidationError",
     # METRICS-MON R4.3 / seed-13: instrumentation hook type alias
     # exported so consumers can type their hook closures.
     "RequestHook",
+    # APD-DCLIENT-006: the validate_npz_contract return Literal, exported so
+    # consumers can annotate and exhaustively match on the result.
+    "ContractKind",
     "__version__",
 ]

{juniper_data_client-0.4.1 → juniper_data_client-0.5.0}/juniper_data_client/client.py

@@ -8,6 +8,7 @@ import io
 import logging
 import os
 import time
+from pathlib import Path
 from typing import Any, Callable, Dict, List, Optional
 from urllib.parse import urlparse
 
@@ -18,9 +19,11 @@ from urllib3.util.retry import Retry
 
 from juniper_data_client.constants import (
     API_KEY_ENV_VAR,
+    API_KEY_FILE_ENV_VAR,
     API_KEY_HEADER_NAME,
     API_VERSION_PATH_SUFFIX,
     DEFAULT_BACKOFF_FACTOR,
+    DEFAULT_BACKOFF_JITTER,
     DEFAULT_BASE_URL,
     DEFAULT_LIST_LIMIT,
     DEFAULT_LIST_OFFSET,
@@ -60,11 +63,31 @@ from juniper_data_client.constants import (
     SPIRAL_TRAIN_RATIO_DEFAULT,
     URL_SCHEME_PREFIXES,
 )
-from juniper_data_client.exceptions import JuniperDataClientError, JuniperDataConnectionError, JuniperDataNotFoundError, JuniperDataTimeoutError, JuniperDataValidationError
+from juniper_data_client.exceptions import JuniperDataClientError, JuniperDataConfigurationError, JuniperDataConnectionError, JuniperDataNotFoundError, JuniperDataTimeoutError, JuniperDataValidationError
 
 logger = logging.getLogger("juniper_data_client.client")
 
 
+def _resolve_api_key_from_env() -> Optional[str]:
+    """Resolve the juniper-data API key from the environment.
+
+    Honors the Docker-secret ``JUNIPER_DATA_API_KEY_FILE`` indirection (a file whose
+    stripped contents are the key, e.g. ``/run/secrets/juniper_data_api_keys``) before the
+    plain ``JUNIPER_DATA_API_KEY`` env var, so a consumer that mounts the key as a file and
+    leaves ``api_key`` unset still authenticates. Mirrors how the Juniper services resolve
+    their own secrets (e.g. cascor ``api.secrets.get_secret``).
+    """
+    file_path = os.environ.get(API_KEY_FILE_ENV_VAR)
+    if file_path:
+        try:
+            content = Path(file_path).read_text(encoding="utf-8").strip()
+        except OSError:
+            content = ""
+        if content:
+            return content
+    return os.environ.get(API_KEY_ENV_VAR)
+
+
 # METRICS-MON R4.3 / seed-13: type alias for the optional instrumentation
 # hook. Consumers (canopy, cascor) pass a callable matching this shape;
 # default in :class:`JuniperDataClient` is no-op so standalone users
@@ -100,6 +123,42 @@ def _noop_request_hook(
     """
 
 
+def _render_error_detail(detail: Any) -> str:
+    """Render a server ``detail`` payload as readable text for an error message.
+
+    FastAPI answers a validation failure with a *list* of error objects, e.g.::
+
+        [{"type": "missing", "loc": ["body", "seed"], "msg": "Field required"}]
+
+    Interpolating that list into an f-string produced a Python repr -- valid
+    text, but nothing a caller could parse and nothing a human could read at a
+    glance (defect-register ``APD-DCLIENT-003``). This renders the same content
+    as ``body.seed: Field required``, while the untouched structure stays
+    available on ``exc.detail``.
+
+    Anything that is not a list of mappings is returned via ``str`` unchanged,
+    so plain-string details and unexpected shapes both survive intact.
+    """
+    if not isinstance(detail, list):
+        return str(detail)
+
+    rendered: list[str] = []
+    for item in detail:
+        if not isinstance(item, dict):
+            rendered.append(str(item))
+            continue
+        msg = item.get("msg", "")
+        loc = item.get("loc")
+        if isinstance(loc, (list, tuple)) and loc:
+            location = ".".join(str(part) for part in loc)
+            rendered.append(f"{location}: {msg}" if msg else location)
+        else:
+            rendered.append(str(msg) if msg else str(item))
+    # An empty list is a degenerate but legal payload; ``str`` keeps it visible
+    # rather than collapsing it to an empty message.
+    return "; ".join(rendered) if rendered else str(detail)
+
+
 class JuniperDataClient:
     """Client for interacting with the JuniperData REST API.
 
@@ -151,7 +210,7 @@ class JuniperDataClient:
         # guards — the no-op call is a single attribute load + return).
         self._on_request: RequestHook = on_request or _noop_request_hook
 
-        resolved_api_key = api_key or os.environ.get(API_KEY_ENV_VAR)
+        resolved_api_key = api_key or _resolve_api_key_from_env()
         if resolved_api_key:
             self.session.headers[API_KEY_HEADER_NAME] = resolved_api_key
 
@@ -163,13 +222,28 @@ class JuniperDataClient:
 
         Returns:
             Normalized URL with scheme, no trailing slash, no /v1 suffix
+
+        Raises:
+            JuniperDataConfigurationError: when the URL carries no host (an
+                empty string, a bare scheme, a path-only value, or a
+                userinfo-only authority like ``http://user:secret@``) — a
+                misconfiguration that would otherwise normalize to a broken,
+                hostless URL and fail opaquely on the first request
+                (``APD-DCLIENT-004``; mirrors juniper-recurrence-client).
         """
         url = url.strip()
 
-        if not url.startswith(URL_SCHEME_PREFIXES):
+        # Scheme matching is case-insensitive (RFC 3986 §3.1): a case-sensitive
+        # startswith would re-prefix "HTTPS://host" into "http://HTTPS://host",
+        # silently downgrading TLS and sending the API key to hostname "https".
+        if not url.lower().startswith(URL_SCHEME_PREFIXES):
             url = f"{DEFAULT_URL_SCHEME_PREFIX}{url}"
 
         parsed = urlparse(url)
+        # hostname, not netloc: netloc accepts a userinfo-only authority
+        # ("user:secret@") as truthy while hostname is None for it.
+        if not parsed.hostname:
+            raise JuniperDataConfigurationError(f"base_url must include a host; got {url!r}")
         normalized = f"{parsed.scheme}://{parsed.netloc}{parsed.path}"
         normalized = normalized.rstrip("/")
 
@@ -189,6 +263,8 @@ class JuniperDataClient:
         retry_strategy = Retry(
             total=self.retries,
             backoff_factor=self.backoff_factor,
+            # APD-ECO-002: decorrelate retry schedules across client instances.
+            backoff_jitter=DEFAULT_BACKOFF_JITTER,
             status_forcelist=RETRYABLE_STATUS_CODES,
             allowed_methods=RETRY_ALLOWED_METHODS,
         )
@@ -287,14 +363,46 @@ class JuniperDataClient:
                 # fall back to using the raw response text as the error detail.
                 error_detail = response.text
 
+            # APD-DCLIENT-003: ``error_detail`` is whatever the server sent --
+            # a str for most handlers, but a list[dict] for FastAPI's 422. It is
+            # attached to the exception UNMODIFIED, and rendered separately for
+            # the human-readable message. Interpolating the list directly
+            # produced a Python repr no caller could parse, and which lost
+            # nothing only because there was nowhere else for the structure to
+            # go. Now there is: ``exc.detail``.
+            rendered_detail = _render_error_detail(error_detail)
+
+            # The context is passed explicitly at each site rather than unpacked
+            # from a shared ``**kwargs`` dict: mypy --strict infers such a dict
+            # as ``dict[str, object]`` and cannot match it against the typed
+            # keyword-only parameters, so the dict form silently gives up the
+            # type checking on exactly the arguments being added here.
             if response.status_code == HTTP_404_NOT_FOUND:
-                outgoing_error = JuniperDataNotFoundError(f"Resource not found: {error_detail}")
+                outgoing_error = JuniperDataNotFoundError(
+                    f"Resource not found: {rendered_detail}",
+                    status_code=response.status_code,
+                    detail=error_detail,
+                    response=response,
+                )
                 raise outgoing_error
             elif response.status_code in (HTTP_400_BAD_REQUEST, HTTP_422_UNPROCESSABLE_ENTITY):
-                outgoing_error = JuniperDataValidationError(f"Validation error: {error_detail}")
+                # APD-DCLIENT-001: the status code is what separates these two.
+                # It rides on the exception now, so a caller no longer has to
+                # substring-match the message to tell them apart.
+                outgoing_error = JuniperDataValidationError(
+                    f"Validation error ({response.status_code}): {rendered_detail}",
+                    status_code=response.status_code,
+                    detail=error_detail,
+                    response=response,
+                )
                 raise outgoing_error
             else:
-                outgoing_error = JuniperDataClientError(f"Request failed ({response.status_code}): {error_detail}")
+                outgoing_error = JuniperDataClientError(
+                    f"Request failed ({response.status_code}): {rendered_detail}",
+                    status_code=response.status_code,
+                    detail=error_detail,
+                    response=response,
+                )
                 raise outgoing_error
         finally:
             duration_ms = (time.monotonic() - start) * 1000.0
@@ -391,6 +499,7 @@ class JuniperDataClient:
         self,
         generator: str,
         params: Dict[str, Any],
+        *,
         persist: bool = True,
         name: Optional[str] = None,
         description: Optional[str] = None,
@@ -401,6 +510,12 @@ class JuniperDataClient:
     ) -> Dict[str, Any]:
         """Create a new dataset via the JuniperData API.
 
+        ``persist`` and everything after it are keyword-only (defect-register
+        ``APD-DCLIENT-008``): with nine positional-or-keyword parameters the
+        call ``create_dataset("spiral", p, False)`` was legal and unreadable,
+        and any signature reordering would silently rebind arguments. Only
+        ``generator`` and ``params`` — the universal pair — stay positional.
+
         If a dataset with the same parameters already exists, the existing
         dataset is returned (caching behavior).
 
@@ -525,12 +640,24 @@ class JuniperDataClient:
     def download_artifact_npz(self, dataset_id: str) -> Dict[str, np.ndarray]:
         """Download and load an NPZ artifact for a dataset.
 
-        The returned dictionary contains numpy arrays with the standard keys:
+        The returned dictionary contains every array in the NPZ. Classification
+        artifacts carry the standard keys:
         - X_train, y_train: Training features and one-hot labels
         - X_test, y_test: Test features and one-hot labels
-        - X_full, y_full: Full dataset features and one-hot labels
 
-        All arrays are float32 dtype.
+        Additional keys may be present depending on the generator (WS-1 /
+        juniper-data#168):
+        - y_reg_{split}: regression target(s) (e.g. equities next-day close).
+        - 3-D sequence artifacts (X.ndim == 3, shape (W, L, F)) add a per-step
+          dt_{split} (or absolute t_{split}) elapsed-time channel plus
+          target_dt_{split} and optional observed_mask_{split} /
+          padding_mask_{split}. Pass the result to
+          :func:`juniper_data_client.validate_npz_contract` to classify
+          ("tabular" / "sequence") and validate it. The matching ``task_type``
+          ("classification" / "regression") and ``time_unit`` live in the
+          dataset metadata (see :meth:`get_dataset_metadata`).
+
+        Feature and label arrays are float32.
 
         Args:
             dataset_id: ID of the dataset whose artifact to download