juniper-data-client 0.4.1__tar.gz → 0.4.2__tar.gz

juniper_data_client-0.4.2/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: juniper-data-client
+Version: 0.4.2
+Summary: HTTP client for the JuniperData dataset generation service
+Author: Paul Calnon
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/pcalnon/juniper-data-client
+Project-URL: Repository, https://github.com/pcalnon/juniper-data-client
+Project-URL: Documentation, https://github.com/pcalnon/juniper-data-client#readme
+Project-URL: Issues, https://github.com/pcalnon/juniper-data-client/issues
+Keywords: juniper,juniper-ml,juniper-ai,artificial-intelligence,dataset,machine-learning,api-client
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Typing :: Typed
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: requests>=2.28.0
+Requires-Dist: urllib3>=2.0.0
+Provides-Extra: test
+Requires-Dist: pytest>=7.0.0; extra == "test"
+Requires-Dist: pytest-cov>=4.0.0; extra == "test"
+Requires-Dist: pytest-timeout>=2.2.0; extra == "test"
+Requires-Dist: responses>=0.23.0; extra == "test"
+Requires-Dist: juniper-observability>=0.1.1; extra == "test"
+Provides-Extra: observability
+Requires-Dist: juniper-observability>=0.1.1; extra == "observability"
+Provides-Extra: dev
+Requires-Dist: juniper-data-client[test]; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: isort>=5.12.0; extra == "dev"
+Requires-Dist: mypy>=1.0.0; extra == "dev"
+Requires-Dist: flake8>=7.0.0; extra == "dev"
+Requires-Dist: types-requests>=2.28.0; extra == "dev"
+Dynamic: license-file
+
+<!-- markdownlint-disable MD013 MD033 MD041 -->
+<!--
+  MD013 (line-length): README contains prose paragraphs that intentionally
+                       exceed the 512-char ecosystem limit (canonical
+                       §4 layout). Wrapping harms PyPI rendering.
+  MD033 (no-inline-html): The right-aligned logo + spacing rely on HTML.
+  MD041 (first-line-heading): The HTML logo is the first line by design.
+  Mirrors juniper-ml #283, juniper-cascor #276, juniper-cascor-worker #71.
+-->
+<div align="right" width="150px" height="150px" align="right" valign="top"> <img src="images/Juniper_Logo_150px.png" alt="Juniper" align="right" valign="top" width="150px" /></div>
+<br /> <br /> <br /> <br />
+
+# Juniper: Dynamic Neural Network Research Platform
+
+Juniper is an AI/ML research platform for investigating dynamic neural network architectures and novel learning paradigms.  The project emphasizes ground-up implementations from primary literature, enabling a more transparent exploration of fundamental algorithms.
+
+## Juniper Data Client
+
+`juniper-data-client` is the **Python HTTP client library** for the `juniper-data` dataset-generation service. The package exposes a single `JuniperDataClient` class whose methods correspond to the REST surface of `juniper-data`: generator schemas, dataset creation, tag-based filtering, batch operations, the named-version registry, and download of the resulting NPZ artifacts as ready-to-train `numpy` arrays. It is the canonical dataset-fetch interface used by both `juniper-cascor` (for training) and `juniper-canopy` (for visualisation), and it serializes the platform's shared `X_train` / `y_train` / `X_test` / `y_test` / `X_full` / `y_full` NPZ schema (all `float32`).
+
+## Distribution
+
+`juniper-data-client` is published on PyPI as **[`juniper-data-client`](https://pypi.org/project/juniper-data-client/)**.
+The package is also surfaced through the platform meta-distribution
+**[`juniper-ml`](https://pypi.org/project/juniper-ml/)**, which installs
+the full client stack via `pip install juniper-ml[all]`.
+
+```bash
+pip install juniper-data-client
+```
+
+## Ecosystem Compatibility
+
+This client library is part of the [Juniper](https://github.com/pcalnon/juniper-ml) ecosystem.
+Verified compatible versions:
+
+| juniper-data | juniper-cascor | juniper-canopy | data-client | cascor-client | cascor-worker |
+|--------------|----------------|----------------|-------------|---------------|---------------|
+| 0.6.x        | 0.5.x          | 0.5.x          | >=0.4.1     | >=0.4.0       | >=0.4.0       |
+
+For full-stack Docker deployment and integration tests, see [`juniper-deploy`](https://github.com/pcalnon/juniper-deploy).
+
+## Architecture
+
+`juniper-data-client` is a thin synchronous HTTP client. It does not embed any dataset-generation logic of its own: every call resolves to an HTTP request against a `juniper-data` instance, and every NPZ artifact returned has been produced server-side.
+
+```text
+┌────────────────────────┐                     ┌──────────────────┐
+│     Caller (e.g.       │  HTTP (requests +   │  juniper-data    │
+│ juniper-cascor /       │  urllib3 Retry)     │  REST service    │
+│ juniper-canopy /       │ ──────────────────► │  Port 8100       │
+│ research notebook)     │ ◄────────────────── │  /v1/...         │
+└──────────┬─────────────┘   JSON + NPZ        └──────────────────┘
+           │ uses
+           ▼
+┌────────────────────────┐
+│  juniper-data-client   │
+│  JuniperDataClient     │
+│  (this package)        │
+└────────────────────────┘
+```
+
+The client retries idempotent verbs (GET) on transient failures via `urllib3.Retry` with exponential backoff; mutating verbs (POST, PATCH, DELETE) are **not** auto-retried, to avoid duplicate-side-effect bugs against the dataset registry. Optional `X-Request-ID` propagation and a `RequestHook` instrumentation surface are available for callers that integrate with the platform's observability stack.
+
+## Related Services
+
+| Service | Relationship | Notes |
+|---------|-------------|-------|
+| [juniper-data](https://github.com/pcalnon/juniper-data) | The HTTP service this client targets | Set `base_url` to the service's URL (default `http://localhost:8100`) |
+| [juniper-cascor](https://github.com/pcalnon/juniper-cascor) | Primary consumer; uses this client to fetch training datasets | Reads `JUNIPER_DATA_URL` |
+| [juniper-canopy](https://github.com/pcalnon/juniper-canopy) | Secondary consumer; uses this client to fetch visualisation data | Reads `JUNIPER_DATA_URL` |
+
+## Active Research Components
+
+`juniper-data-client` does not host research components of its own; it is the surface through which other components of the Juniper platform reach the research components hosted by `juniper-data`. Through this client, callers access the **ARC-AGI dataset families** (ARC-AGI-1 and ARC-AGI-2), the **named-version dataset registry** (`list_versions`, `get_latest`), the **batch dataset operations** (`batch_create`, `batch_delete`, `batch_update_tags`, `batch_export`), and the **NPZ artifact contract** that the rest of the platform consumes. Treating these as research artifacts is appropriate: each is a stable interface around which comparative experiments are composed.
+
+## Quick Start Guide
+
+### Prerequisites
+
+- Python ≥ 3.12
+- A running `juniper-data` instance reachable at the URL passed as `base_url` (typically `http://localhost:8100`)
+
+### Installation
+
+```bash
+pip install juniper-data-client
+```
+
+Optional extras: `[observability]` enables `X-Request-ID` propagation through `juniper-observability`; `[test]` installs the testing dependencies; `[dev]` adds linting and type-checking tools.
+
+### Verification
+
+```python
+from juniper_data_client import JuniperDataClient
+
+with JuniperDataClient("http://localhost:8100") as client:
+    health = client.health_check()
+    print(f"Service status: {health['status']}")
+
+    result = client.create_spiral_dataset(
+        n_spirals=2,
+        n_points_per_spiral=100,
+        noise=0.1,
+        seed=42,
+    )
+    arrays = client.download_artifact_npz(result["dataset_id"])
+
+    print(f"Training samples: {len(arrays['X_train'])}")
+    print(f"Test samples:     {len(arrays['X_test'])}")
+```
+
+For PyTorch consumers, the returned arrays convert directly with `torch.from_numpy`:
+
+```python
+import torch
+X_train = torch.from_numpy(arrays["X_train"])  # torch.float32
+y_train = torch.from_numpy(arrays["y_train"])  # torch.float32
+```
+
+### Next Steps
+
+- [`docs/QUICK_START.md`](docs/QUICK_START.md) — complete installation and verification guide
+- [`docs/REFERENCE.md`](docs/REFERENCE.md) — full API reference, configuration, and error model
+- [`docs/DEVELOPER_CHEATSHEET.md`](docs/DEVELOPER_CHEATSHEET.md) — quick-reference card for development tasks
+- [`juniper-data`](https://github.com/pcalnon/juniper-data) — the upstream dataset service
+- [`juniper-ml`](https://pypi.org/project/juniper-ml/) — platform meta-package on PyPI
+
+## Research Philosophy
+
+The Juniper platform exists to study learning algorithms whose network architecture is not fixed in advance. Its initial anchor is the Cascade-Correlation algorithm of Fahlman and Lebiere (1990), implemented from the primary literature without recourse to higher-level abstractions that elide the algorithm's operational detail. The organising commitment is that algorithm implementations remain inspectable at the level at which they were originally specified: candidate units, correlation objectives, weight-freezing semantics, and the structural events that grow the network are first-class artifacts of the codebase rather than internal details of a library wrapper. This permits comparative work — across algorithms, datasets, and hyperparameter regimes — to be conducted on a known and reproducible substrate.
+
+The current platform comprises a Cascade-Correlation training service exposing a REST and WebSocket interface, a dataset-generation service with a named-version registry that includes the ARC-AGI families, a real-time monitoring dashboard for inspecting training dynamics as they occur, and a distributed worker that parallelises candidate-unit training across hosts. Near-term work extends the architectural-growth catalogue beyond Cascade-Correlation, introduces multi-network orchestration for comparative experiments at the level of network populations rather than individual runs, and tightens the dataset–training–monitoring loop into a reproducible research workbench. The longer-term direction is the systematic empirical study of constructive and architecture-growing learning algorithms, with first-class infrastructure for the ablation, comparison, and replication that such a study requires.
+
+Within this programme, `juniper-data-client` is the canonical integration boundary between the training and data services. Its contract is the dataset-fetch interface used by every training-side consumer; changes to its surface are therefore changes to the platform's shared dataset semantics.
+
+## Documentation
+
+| Document | Purpose |
+|----------|---------|
+| [`docs/DOCUMENTATION_OVERVIEW.md`](docs/DOCUMENTATION_OVERVIEW.md) | Navigation index for all `juniper-data-client` documentation |
+| [`docs/QUICK_START.md`](docs/QUICK_START.md) | Complete installation and verification guide |
+| [`docs/REFERENCE.md`](docs/REFERENCE.md) | Full API reference, configuration, error model, and NPZ schema |
+| [`docs/DEVELOPER_CHEATSHEET.md`](docs/DEVELOPER_CHEATSHEET.md) | Quick-reference card for development tasks |
+| [`CHANGELOG.md`](CHANGELOG.md) | Version history |
+
+## License
+
+MIT License — see [`LICENSE`](LICENSE) for details.

juniper_data_client-0.4.2/README.md

@@ -0,0 +1,149 @@
+<!-- markdownlint-disable MD013 MD033 MD041 -->
+<!--
+  MD013 (line-length): README contains prose paragraphs that intentionally
+                       exceed the 512-char ecosystem limit (canonical
+                       §4 layout). Wrapping harms PyPI rendering.
+  MD033 (no-inline-html): The right-aligned logo + spacing rely on HTML.
+  MD041 (first-line-heading): The HTML logo is the first line by design.
+  Mirrors juniper-ml #283, juniper-cascor #276, juniper-cascor-worker #71.
+-->
+<div align="right" width="150px" height="150px" align="right" valign="top"> <img src="images/Juniper_Logo_150px.png" alt="Juniper" align="right" valign="top" width="150px" /></div>
+<br /> <br /> <br /> <br />
+
+# Juniper: Dynamic Neural Network Research Platform
+
+Juniper is an AI/ML research platform for investigating dynamic neural network architectures and novel learning paradigms.  The project emphasizes ground-up implementations from primary literature, enabling a more transparent exploration of fundamental algorithms.
+
+## Juniper Data Client
+
+`juniper-data-client` is the **Python HTTP client library** for the `juniper-data` dataset-generation service. The package exposes a single `JuniperDataClient` class whose methods correspond to the REST surface of `juniper-data`: generator schemas, dataset creation, tag-based filtering, batch operations, the named-version registry, and download of the resulting NPZ artifacts as ready-to-train `numpy` arrays. It is the canonical dataset-fetch interface used by both `juniper-cascor` (for training) and `juniper-canopy` (for visualisation), and it serializes the platform's shared `X_train` / `y_train` / `X_test` / `y_test` / `X_full` / `y_full` NPZ schema (all `float32`).
+
+## Distribution
+
+`juniper-data-client` is published on PyPI as **[`juniper-data-client`](https://pypi.org/project/juniper-data-client/)**.
+The package is also surfaced through the platform meta-distribution
+**[`juniper-ml`](https://pypi.org/project/juniper-ml/)**, which installs
+the full client stack via `pip install juniper-ml[all]`.
+
+```bash
+pip install juniper-data-client
+```
+
+## Ecosystem Compatibility
+
+This client library is part of the [Juniper](https://github.com/pcalnon/juniper-ml) ecosystem.
+Verified compatible versions:
+
+| juniper-data | juniper-cascor | juniper-canopy | data-client | cascor-client | cascor-worker |
+|--------------|----------------|----------------|-------------|---------------|---------------|
+| 0.6.x        | 0.5.x          | 0.5.x          | >=0.4.1     | >=0.4.0       | >=0.4.0       |
+
+For full-stack Docker deployment and integration tests, see [`juniper-deploy`](https://github.com/pcalnon/juniper-deploy).
+
+## Architecture
+
+`juniper-data-client` is a thin synchronous HTTP client. It does not embed any dataset-generation logic of its own: every call resolves to an HTTP request against a `juniper-data` instance, and every NPZ artifact returned has been produced server-side.
+
+```text
+┌────────────────────────┐                     ┌──────────────────┐
+│     Caller (e.g.       │  HTTP (requests +   │  juniper-data    │
+│ juniper-cascor /       │  urllib3 Retry)     │  REST service    │
+│ juniper-canopy /       │ ──────────────────► │  Port 8100       │
+│ research notebook)     │ ◄────────────────── │  /v1/...         │
+└──────────┬─────────────┘   JSON + NPZ        └──────────────────┘
+           │ uses
+           ▼
+┌────────────────────────┐
+│  juniper-data-client   │
+│  JuniperDataClient     │
+│  (this package)        │
+└────────────────────────┘
+```
+
+The client retries idempotent verbs (GET) on transient failures via `urllib3.Retry` with exponential backoff; mutating verbs (POST, PATCH, DELETE) are **not** auto-retried, to avoid duplicate-side-effect bugs against the dataset registry. Optional `X-Request-ID` propagation and a `RequestHook` instrumentation surface are available for callers that integrate with the platform's observability stack.
+
+## Related Services
+
+| Service | Relationship | Notes |
+|---------|-------------|-------|
+| [juniper-data](https://github.com/pcalnon/juniper-data) | The HTTP service this client targets | Set `base_url` to the service's URL (default `http://localhost:8100`) |
+| [juniper-cascor](https://github.com/pcalnon/juniper-cascor) | Primary consumer; uses this client to fetch training datasets | Reads `JUNIPER_DATA_URL` |
+| [juniper-canopy](https://github.com/pcalnon/juniper-canopy) | Secondary consumer; uses this client to fetch visualisation data | Reads `JUNIPER_DATA_URL` |
+
+## Active Research Components
+
+`juniper-data-client` does not host research components of its own; it is the surface through which other components of the Juniper platform reach the research components hosted by `juniper-data`. Through this client, callers access the **ARC-AGI dataset families** (ARC-AGI-1 and ARC-AGI-2), the **named-version dataset registry** (`list_versions`, `get_latest`), the **batch dataset operations** (`batch_create`, `batch_delete`, `batch_update_tags`, `batch_export`), and the **NPZ artifact contract** that the rest of the platform consumes. Treating these as research artifacts is appropriate: each is a stable interface around which comparative experiments are composed.
+
+## Quick Start Guide
+
+### Prerequisites
+
+- Python ≥ 3.12
+- A running `juniper-data` instance reachable at the URL passed as `base_url` (typically `http://localhost:8100`)
+
+### Installation
+
+```bash
+pip install juniper-data-client
+```
+
+Optional extras: `[observability]` enables `X-Request-ID` propagation through `juniper-observability`; `[test]` installs the testing dependencies; `[dev]` adds linting and type-checking tools.
+
+### Verification
+
+```python
+from juniper_data_client import JuniperDataClient
+
+with JuniperDataClient("http://localhost:8100") as client:
+    health = client.health_check()
+    print(f"Service status: {health['status']}")
+
+    result = client.create_spiral_dataset(
+        n_spirals=2,
+        n_points_per_spiral=100,
+        noise=0.1,
+        seed=42,
+    )
+    arrays = client.download_artifact_npz(result["dataset_id"])
+
+    print(f"Training samples: {len(arrays['X_train'])}")
+    print(f"Test samples:     {len(arrays['X_test'])}")
+```
+
+For PyTorch consumers, the returned arrays convert directly with `torch.from_numpy`:
+
+```python
+import torch
+X_train = torch.from_numpy(arrays["X_train"])  # torch.float32
+y_train = torch.from_numpy(arrays["y_train"])  # torch.float32
+```
+
+### Next Steps
+
+- [`docs/QUICK_START.md`](docs/QUICK_START.md) — complete installation and verification guide
+- [`docs/REFERENCE.md`](docs/REFERENCE.md) — full API reference, configuration, and error model
+- [`docs/DEVELOPER_CHEATSHEET.md`](docs/DEVELOPER_CHEATSHEET.md) — quick-reference card for development tasks
+- [`juniper-data`](https://github.com/pcalnon/juniper-data) — the upstream dataset service
+- [`juniper-ml`](https://pypi.org/project/juniper-ml/) — platform meta-package on PyPI
+
+## Research Philosophy
+
+The Juniper platform exists to study learning algorithms whose network architecture is not fixed in advance. Its initial anchor is the Cascade-Correlation algorithm of Fahlman and Lebiere (1990), implemented from the primary literature without recourse to higher-level abstractions that elide the algorithm's operational detail. The organising commitment is that algorithm implementations remain inspectable at the level at which they were originally specified: candidate units, correlation objectives, weight-freezing semantics, and the structural events that grow the network are first-class artifacts of the codebase rather than internal details of a library wrapper. This permits comparative work — across algorithms, datasets, and hyperparameter regimes — to be conducted on a known and reproducible substrate.
+
+The current platform comprises a Cascade-Correlation training service exposing a REST and WebSocket interface, a dataset-generation service with a named-version registry that includes the ARC-AGI families, a real-time monitoring dashboard for inspecting training dynamics as they occur, and a distributed worker that parallelises candidate-unit training across hosts. Near-term work extends the architectural-growth catalogue beyond Cascade-Correlation, introduces multi-network orchestration for comparative experiments at the level of network populations rather than individual runs, and tightens the dataset–training–monitoring loop into a reproducible research workbench. The longer-term direction is the systematic empirical study of constructive and architecture-growing learning algorithms, with first-class infrastructure for the ablation, comparison, and replication that such a study requires.
+
+Within this programme, `juniper-data-client` is the canonical integration boundary between the training and data services. Its contract is the dataset-fetch interface used by every training-side consumer; changes to its surface are therefore changes to the platform's shared dataset semantics.
+
+## Documentation
+
+| Document | Purpose |
+|----------|---------|
+| [`docs/DOCUMENTATION_OVERVIEW.md`](docs/DOCUMENTATION_OVERVIEW.md) | Navigation index for all `juniper-data-client` documentation |
+| [`docs/QUICK_START.md`](docs/QUICK_START.md) | Complete installation and verification guide |
+| [`docs/REFERENCE.md`](docs/REFERENCE.md) | Full API reference, configuration, error model, and NPZ schema |
+| [`docs/DEVELOPER_CHEATSHEET.md`](docs/DEVELOPER_CHEATSHEET.md) | Quick-reference card for development tasks |
+| [`CHANGELOG.md`](CHANGELOG.md) | Version history |
+
+## License
+
+MIT License — see [`LICENSE`](LICENSE) for details.

{juniper_data_client-0.4.1 → juniper_data_client-0.4.2}/juniper_data_client/__init__.py

@@ -5,12 +5,14 @@ dataset generation service, used by both JuniperCascor and juniper-canopy.
 """
 
 from juniper_data_client.client import JuniperDataClient, RequestHook
+from juniper_data_client.contract import validate_npz_contract
 from juniper_data_client.exceptions import JuniperDataClientError, JuniperDataConfigurationError, JuniperDataConnectionError, JuniperDataNotFoundError, JuniperDataTimeoutError, JuniperDataValidationError
 
-__version__ = "0.4.1"
+__version__ = "0.4.2"
 
 __all__ = [
     "JuniperDataClient",
+    "validate_npz_contract",
     "JuniperDataClientError",
     "JuniperDataConfigurationError",
     "JuniperDataConnectionError",

{juniper_data_client-0.4.1 → juniper_data_client-0.4.2}/juniper_data_client/client.py

@@ -8,6 +8,7 @@ import io
 import logging
 import os
 import time
+from pathlib import Path
 from typing import Any, Callable, Dict, List, Optional
 from urllib.parse import urlparse
 
@@ -18,6 +19,7 @@ from urllib3.util.retry import Retry
 
 from juniper_data_client.constants import (
     API_KEY_ENV_VAR,
+    API_KEY_FILE_ENV_VAR,
     API_KEY_HEADER_NAME,
     API_VERSION_PATH_SUFFIX,
     DEFAULT_BACKOFF_FACTOR,
@@ -65,6 +67,26 @@ from juniper_data_client.exceptions import JuniperDataClientError, JuniperDataCo
 logger = logging.getLogger("juniper_data_client.client")
 
 
+def _resolve_api_key_from_env() -> Optional[str]:
+    """Resolve the juniper-data API key from the environment.
+
+    Honors the Docker-secret ``JUNIPER_DATA_API_KEY_FILE`` indirection (a file whose
+    stripped contents are the key, e.g. ``/run/secrets/juniper_data_api_keys``) before the
+    plain ``JUNIPER_DATA_API_KEY`` env var, so a consumer that mounts the key as a file and
+    leaves ``api_key`` unset still authenticates. Mirrors how the Juniper services resolve
+    their own secrets (e.g. cascor ``api.secrets.get_secret``).
+    """
+    file_path = os.environ.get(API_KEY_FILE_ENV_VAR)
+    if file_path:
+        try:
+            content = Path(file_path).read_text(encoding="utf-8").strip()
+        except OSError:
+            content = ""
+        if content:
+            return content
+    return os.environ.get(API_KEY_ENV_VAR)
+
+
 # METRICS-MON R4.3 / seed-13: type alias for the optional instrumentation
 # hook. Consumers (canopy, cascor) pass a callable matching this shape;
 # default in :class:`JuniperDataClient` is no-op so standalone users
@@ -151,7 +173,7 @@ class JuniperDataClient:
         # guards — the no-op call is a single attribute load + return).
         self._on_request: RequestHook = on_request or _noop_request_hook
 
-        resolved_api_key = api_key or os.environ.get(API_KEY_ENV_VAR)
+        resolved_api_key = api_key or _resolve_api_key_from_env()
         if resolved_api_key:
             self.session.headers[API_KEY_HEADER_NAME] = resolved_api_key
 
@@ -525,12 +547,25 @@ class JuniperDataClient:
     def download_artifact_npz(self, dataset_id: str) -> Dict[str, np.ndarray]:
         """Download and load an NPZ artifact for a dataset.
 
-        The returned dictionary contains numpy arrays with the standard keys:
+        The returned dictionary contains every array in the NPZ. Classification
+        artifacts carry the standard keys:
         - X_train, y_train: Training features and one-hot labels
         - X_test, y_test: Test features and one-hot labels
         - X_full, y_full: Full dataset features and one-hot labels
 
-        All arrays are float32 dtype.
+        Additional keys may be present depending on the generator (WS-1 /
+        juniper-data#168):
+        - y_reg_{split}: regression target(s) (e.g. equities next-day close).
+        - 3-D sequence artifacts (X.ndim == 3, shape (W, L, F)) add a per-step
+          dt_{split} (or absolute t_{split}) elapsed-time channel plus
+          target_dt_{split} and optional observed_mask_{split} /
+          padding_mask_{split}. Pass the result to
+          :func:`juniper_data_client.validate_npz_contract` to classify
+          ("tabular" / "sequence") and validate it. The matching ``task_type``
+          ("classification" / "regression") and ``time_unit`` live in the
+          dataset metadata (see :meth:`get_dataset_metadata`).
+
+        Feature and label arrays are float32.
 
         Args:
             dataset_id: ID of the dataset whose artifact to download

{juniper_data_client-0.4.1 → juniper_data_client-0.4.2}/juniper_data_client/constants.py

@@ -15,6 +15,36 @@ License: MIT License
 
 from typing import List, Tuple
 
+__all__ = [
+    "DEFAULT_BASE_URL",
+    "FAKE_BASE_URL",
+    "DEFAULT_TIMEOUT",
+    "DEFAULT_RETRIES",
+    "DEFAULT_BACKOFF_FACTOR",
+    "RETRYABLE_STATUS_CODES",
+    "RETRY_ALLOWED_METHODS",
+    "HTTP_POOL_CONNECTIONS",
+    "HTTP_POOL_MAXSIZE",
+    "URL_SCHEME_PREFIXES",
+    "DEFAULT_URL_SCHEME_PREFIX",
+    "API_VERSION_PATH_SUFFIX",
+    "DEFAULT_READY_TIMEOUT",
+    "DEFAULT_READY_POLL_INTERVAL",
+    "HEALTH_READY_STATUS",
+    "API_KEY_HEADER_NAME",
+    "API_KEY_ENV_VAR",
+    "API_KEY_FILE_ENV_VAR",
+    "ENDPOINT_HEALTH",
+    "ENDPOINT_HEALTH_READY",
+    "ENDPOINT_GENERATORS",
+    "ENDPOINT_GENERATOR_SCHEMA_TEMPLATE",
+    "ENDPOINT_DATASETS",
+    "ENDPOINT_DATASETS_VERSIONS",
+    "ENDPOINT_DATASETS_LATEST",
+    "ENDPOINT_DATASET_BY_ID_TEMPLATE",
+    "ENDPOINT_DATASET_ARTIFACT_TEMPLATE",
+]
+
 # ─── Service Configuration ───────────────────────────────────────────────────
 
 DEFAULT_BASE_URL: str = "http://localhost:8100"
@@ -53,6 +83,11 @@ HEALTH_READY_STATUS: str = "ready"
 
 API_KEY_HEADER_NAME: str = "X-API-Key"
 API_KEY_ENV_VAR: str = "JUNIPER_DATA_API_KEY"
+# Docker-secret indirection: when set, points at a file whose stripped contents are the
+# API key (e.g. /run/secrets/juniper_data_api_keys). The client honors this before the
+# plain API_KEY_ENV_VAR, so a deployment that mounts the key as a file (and sets only the
+# _FILE form) still authenticates -- mirrors how the Juniper services read their secrets.
+API_KEY_FILE_ENV_VAR: str = f"{API_KEY_ENV_VAR}_FILE"
 
 # ─── REST Endpoints ──────────────────────────────────────────────────────────
 
@@ -215,3 +250,28 @@ FAKE_SERVICE_STATUS: str = "ok"
 FAKE_SERVICE_NAME: str = "juniper-data"
 FAKE_SERVICE_VERSION: str = "fake"
 FAKE_SERVICE_UPTIME_SECONDS: float = 0.0
+
+# ─── NPZ Artifact Contract (WS-1 / juniper-data#168) ─────────────────────────
+
+# Split suffixes used for every per-split NPZ key (suffix a stem with
+# f"{stem}_{split}", e.g. f"{NPZ_KEY_X}_train").
+NPZ_SPLITS: Tuple[str, ...] = ("train", "test", "full")
+
+# Canonical per-split key stems.
+NPZ_KEY_X: str = "X"  # features: (N, F) tabular or (W, L, F) sequence
+NPZ_KEY_Y: str = "y"  # one-hot classification target
+NPZ_KEY_Y_REG: str = "y_reg"  # regression target (e.g. next-day close)
+NPZ_KEY_T: str = "t"  # absolute per-step time (sequence)
+NPZ_KEY_DT: str = "dt"  # per-step elapsed time / Δt (sequence)
+NPZ_KEY_TARGET_DT: str = "target_dt"  # irregular forecast horizon (sequence)
+NPZ_KEY_OBSERVED_MASK: str = "observed_mask"  # 1=real, 0=imputed (sequence)
+NPZ_KEY_PADDING_MASK: str = "padding_mask"  # 1=valid, 0=structural padding (sequence)
+NPZ_KEY_SEQ_LENGTHS: str = "seq_lengths"  # valid step count per window (sequence)
+
+# Contract discriminators returned by ``validate_npz_contract``.
+CONTRACT_KIND_TABULAR: str = "tabular"
+CONTRACT_KIND_SEQUENCE: str = "sequence"
+
+# ``task_type`` values carried in dataset metadata (meta.json, not the NPZ).
+TASK_TYPE_CLASSIFICATION: str = "classification"
+TASK_TYPE_REGRESSION: str = "regression"

juniper_data_client-0.4.2/juniper_data_client/contract.py

@@ -0,0 +1,125 @@
+"""NPZ contract validation for tabular and 3-D sequence dataset artifacts.
+
+WS-1 (juniper-data#168) adds an additive, ``X.ndim``-dispatched NPZ contract: a
+2-D ``X`` is the legacy tabular artifact (unchanged), while a 3-D ``X``
+``(W, L, F)`` is a time-series / irregular-Δt sequence artifact carrying a
+per-step ``dt`` (or absolute ``t``) channel plus optional ``observed_mask`` /
+``padding_mask``.
+
+``validate_npz_contract`` classifies a loaded artifact as ``"tabular"`` or
+``"sequence"`` and enforces the sequence rules: at least one of ``t`` / ``dt``;
+``dt >= 0`` with ``dt[:, 0] == 0``; consistent ``t`` / ``dt``; binary masks of the
+right shape; and ``observed_mask`` only meaningful where ``padding_mask == 1``.
+The 2-D path is untouched and returns immediately.
+
+Reference: ``juniper-ml/notes/JUNIPER_RECURSE_DELTA_T_HANDLING_2026-06-05.md`` §6.
+
+Project: Juniper
+Sub-Project: juniper-data-client
+Application: JuniperDataClient
+Author: Paul Calnon
+Version: 0.4.1
+License: MIT License
+"""
+
+from typing import Dict
+
+import numpy as np
+
+from juniper_data_client.constants import (
+    CONTRACT_KIND_SEQUENCE,
+    CONTRACT_KIND_TABULAR,
+    NPZ_KEY_DT,
+    NPZ_KEY_OBSERVED_MASK,
+    NPZ_KEY_PADDING_MASK,
+    NPZ_KEY_T,
+    NPZ_KEY_X,
+    NPZ_SPLITS,
+)
+
+
+def validate_npz_contract(arrays: Dict[str, np.ndarray], *, dt_atol: float = 1e-6) -> str:
+    """Classify and validate a loaded NPZ artifact's contract.
+
+    Args:
+        arrays: NPZ array mapping (e.g. the dict returned by
+            :meth:`JuniperDataClient.download_artifact_npz`). An
+            ``np.lib.npyio.NpzFile`` also works (it supports ``in`` / ``[]``).
+        dt_atol: absolute tolerance for the ``t`` / ``dt`` consistency check.
+
+    Returns:
+        ``"tabular"`` for a 2-D ``X`` (legacy path, no further checks), or
+        ``"sequence"`` for a validated 3-D artifact.
+
+    Raises:
+        ValueError: if ``X`` is neither 2-D nor 3-D, or any 3-D sequence rule is
+            violated (missing ``t`` / ``dt``; bad ``dt`` shape, sign, or
+            ``dt[:, 0]``; inconsistent ``t`` / ``dt``; a non-binary or mis-shaped
+            mask; or ``observed_mask`` set on a padded step).
+    """
+    x_full_key = f"{NPZ_KEY_X}_full"
+    x_train_key = f"{NPZ_KEY_X}_train"
+    x = arrays[x_full_key] if x_full_key in arrays else arrays[x_train_key]
+    if x.ndim == 2:
+        return CONTRACT_KIND_TABULAR
+    if x.ndim != 3:
+        raise ValueError(f"X must be 2-D (tabular) or 3-D (sequence), got {x.ndim}-D")
+
+    for split in NPZ_SPLITS:
+        x_key = f"{NPZ_KEY_X}_{split}"
+        if x_key in arrays:
+            xs = arrays[x_key]
+            _validate_sequence_split(arrays, split, int(xs.shape[0]), int(xs.shape[1]), dt_atol)
+    return CONTRACT_KIND_SEQUENCE
+
+
+def _validate_sequence_split(arrays: Dict[str, np.ndarray], split: str, n_windows: int, lookback: int, dt_atol: float) -> None:
+    """Enforce the 3-D sequence rules for one split's keys."""
+    t_key = f"{NPZ_KEY_T}_{split}"
+    dt_key = f"{NPZ_KEY_DT}_{split}"
+    has_t = t_key in arrays
+    has_dt = dt_key in arrays
+    if not (has_t or has_dt):
+        raise ValueError(f"{split}: a 3-D artifact needs at least one of {t_key!r} / {dt_key!r}")
+    if has_dt:
+        _validate_dt(arrays[dt_key], n_windows, lookback, dt_key)
+    if has_t and has_dt:
+        _validate_t_dt_consistency(arrays[t_key], arrays[dt_key], split, dt_atol)
+    _validate_masks(arrays, split, n_windows, lookback)
+
+
+def _validate_dt(dt: np.ndarray, n_windows: int, lookback: int, dt_key: str) -> None:
+    """``dt`` must be ``(W, L)``, non-negative, with a zero first column."""
+    if dt.shape != (n_windows, lookback):
+        raise ValueError(f"{dt_key} shape {dt.shape} != {(n_windows, lookback)}")
+    if np.any(dt < 0):
+        raise ValueError(f"{dt_key} has negative gaps")
+    if n_windows and np.any(dt[:, 0] != 0):
+        raise ValueError(f"{dt_key}[:, 0] must be 0 by convention")
+
+
+def _validate_t_dt_consistency(t: np.ndarray, dt: np.ndarray, split: str, dt_atol: float) -> None:
+    """When both ``t`` and ``dt`` are present they must agree to tolerance."""
+    recon = np.zeros_like(t)
+    recon[:, 1:] = np.diff(t, axis=1)
+    if not np.allclose(recon, dt, atol=dt_atol):
+        raise ValueError(f"{split}: t_ and dt_ are inconsistent")
+
+
+def _validate_masks(arrays: Dict[str, np.ndarray], split: str, n_windows: int, lookback: int) -> None:
+    """Masks (if present) must be binary, correctly shaped, and consistent."""
+    for mask_key in (f"{NPZ_KEY_OBSERVED_MASK}_{split}", f"{NPZ_KEY_PADDING_MASK}_{split}"):
+        if mask_key in arrays:
+            mask = arrays[mask_key]
+            if mask.shape != (n_windows, lookback):
+                raise ValueError(f"{mask_key} shape {mask.shape} != {(n_windows, lookback)}")
+            if not np.isin(mask, (0, 1)).all():
+                raise ValueError(f"{mask_key} must be binary (0/1)")
+
+    observed_key = f"{NPZ_KEY_OBSERVED_MASK}_{split}"
+    padding_key = f"{NPZ_KEY_PADDING_MASK}_{split}"
+    if observed_key in arrays and padding_key in arrays:
+        observed = arrays[observed_key]
+        padding = arrays[padding_key]
+        if np.any((padding == 0) & (observed == 1)):
+            raise ValueError(f"{split}: observed_mask=1 on a padded (padding_mask=0) step")