@datalayer/core 1.1.66 → 1.2.10

package/README.md

@@ -18,7 +18,7 @@
 
 ## Overview
 
-Datalayer Core is the foundational package that powers the [Datalayer AI Platform](https://datalayer.ai). It provides TypeScript and Python packages as a Command Line Interface (CLI) for AI engineers, data scientists, and researchers to seamlessly integrate scalable compute runtimes into their workflows.
+Datalayer Core is the foundational package that powers the [Datalayer AI Platform](https://datalayer.app/). It provides a TypesScript and Python packages as a Command Line Interface (CLI) for AI engineers, data scientists, and researchers to seamlessly integrate scalable compute runtimes into their workflows.
 
 This package serves as the base foundation used by many other Datalayer packages, containing core application classes, configuration, and unified APIs for authentication, runtime management, and code execution in cloud-based environments.
 
@@ -66,6 +66,8 @@ npm install
 
 ## Quick Start with Python
 
+### 1. Authentication
+
 Set your Datalayer token as an environment variable:
 
 ```bash
@@ -87,19 +89,31 @@ if client.authenticate():
     print("Successfully authenticated!")
 ```
 
-## Architecture
+### 2. Runtime, Snapshots, and Evals
 
-Datalayer Core serves as the foundation for the entire Datalayer ecosystem:
+Runtime execution, snapshot workflows, and evals CLI are now documented and maintained in:
 
-- **Base Classes**: Core application classes inherited by other Datalayer packages
-- **Configuration Management**: Centralized configuration system for all Datalayer components
-- **Authentication Layer**: Unified authentication across all Datalayer services
-- **Runtime Abstraction**: Common interface for different types of compute runtimes
-- **Resource Management**: Automatic cleanup and lifecycle management
+- [agent-runtimes README](https://github.com/datalayer/agent-runtimes/blob/main/README.md)
+- [agent-runtimes CLI docs](https://agent-runtimes.datalayer.tech/cli)
+
+Use `agent-runtimes` for runtime workloads (`RuntimeClient`, `@datalayer`, snapshots, and evals), and use `datalayer-core` for account/platform operations (authentication, secrets, API keys, usage, profile).
 
 ## Examples
 
-- [OTEL example README](https://github.com/datalayer/core/blob/main/examples/otel/README.md)
+Examples have moved to Agent Runtimes:
+
+- [Agent Runtimes examples README](https://github.com/datalayer/agent-runtimes/blob/main/examples/README.md)
+- [Agent Runtimes examples directory](https://github.com/datalayer/agent-runtimes/tree/main/examples)
+
+## Platform Integration
+
+Datalayer adds AI capabilities and scalable compute runtimes to your development workflows. The platform is designed to seamlessly integrate into your existing processes and supercharge your computations with the processing power you need.
+
+Key platform features accessible through this Client and CLI:
+
+- **Remote Runtimes**: Execute code on powerful remote machines with CPU, RAM, and GPU resources
+- **Multiple Interfaces**: Access and consume runtimes through Python Client, CLI, or other integrated tools
+- **Scalable Compute**: Dynamically scale your computational resources based on workload requirements
 
 ## Documentation
 
@@ -179,6 +193,16 @@ This Client is designed to be simple and extensible. We welcome contributions! P
 
 For issues and enhancement requests, please use the [GitHub issue tracker](https://github.com/datalayer/core/issues).
 
+## Architecture
+
+Datalayer Core serves as the foundation for the entire Datalayer ecosystem:
+
+- **Base Classes**: Core application classes inherited by other Datalayer packages
+- **Configuration Management**: Centralized configuration system for all Datalayer components
+- **Authentication Layer**: Unified authentication across all Datalayer services
+- **Runtime Abstraction**: Common interface for different types of compute runtimes
+- **Resource Management**: Automatic cleanup and lifecycle management
+
 ## Use Cases
 
 - **AI/ML Development**: Scale your machine learning workflows with cloud compute using Client or CLI
@@ -195,16 +219,15 @@ This project is licensed under the [BSD 3-Clause License](https://github.com/dat
 
 - **Documentation**: [Datalayer Platform Documentation](https://datalayer.ai/docs/)
 - **Issues**: [GitHub Issues](https://github.com/datalayer/core/issues)
-- **Community**: [Datalayer Platform](https://datalayer.ai)
+- **Community**: [Datalayer Platform](https://datalayer.app/)
 
 ---
 
-<div align="center">
-
-**If this project is helpful to you, please give us a ⭐️**
-
-Made with ❤️ by [Datalayer](https://datalayer.ai)
-
-<img src="https://assets.datalayer.tech/datalayer-25.svg" alt="Datalayer Logo" width="200"></img>
+<p align="center">
+  <img src="https://assets.datalayer.tech/datalayer-25.svg" alt="Datalayer Logo" width="200"></img>
+</p>
 
-</div>
+<p align="center">
+  <strong>🚀 AI Agents for Data Analysis</strong><br></br>
+  <a href="https://datalayer.app/">Get started with Datalayer today!</a>
+</p>

package/lib/api/constants.d.ts

@@ -16,7 +16,7 @@ export declare const API_BASE_PATHS: {
  */
 export declare const DEFAULT_SERVICE_URLS: {
     /** Default URL for AI Agents (durable agent management) service */
-    readonly AI_AGENTS: "https://prod1.datalayer.run";
+    readonly AI_AGENTS: "https://r1.datalayer.run";
     /**
      * Default URL for the Contents service.
      *

package/lib/api/constants.js

@@ -20,7 +20,7 @@ export const API_BASE_PATHS = {
  */
 export const DEFAULT_SERVICE_URLS = {
     /** Default URL for AI Agents (durable agent management) service */
-    AI_AGENTS: 'https://prod1.datalayer.run',
+    AI_AGENTS: 'https://r1.datalayer.run',
     /**
      * Default URL for the Contents service.
      *

package/lib/api/contents/attachments.js

@@ -4,8 +4,7 @@
  */
 import { requestDatalayerAPI } from '../DatalayerApi.js';
 import { API_BASE_PATHS, DEFAULT_SERVICE_URLS } from '../constants.js';
-import { contentsToCamelCase, contentsToSnakeCase } from '../../models/contents/index.js';
-import { assertSandboxUid } from './sandboxUid.js';
+import { contentsToCamelCase, contentsToSnakeCase, } from '../../models/contents/index.js';
 const convert = (value) => contentsToCamelCase(value);
 const url = (baseUrl, suffix = '') => `${baseUrl}${API_BASE_PATHS.CONTENTS}/attachments${suffix}`;
 export const createAttachment = async (token, request, idempotencyKey, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
@@ -18,7 +17,7 @@ export const createAttachment = async (token, request, idempotencyKey, baseUrl =
 export const listAttachments = async (token, options = {}, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => {
     const parameters = new URLSearchParams();
     if (options.sandboxUid)
-        parameters.set('sandbox_uid', assertSandboxUid(options.sandboxUid));
+        parameters.set('sandbox_uid', options.sandboxUid);
     if (options.sourceUid)
         parameters.set('source_uid', options.sourceUid);
     parameters.set('active', String(options.active ?? false));
@@ -29,10 +28,7 @@ export const listAttachments = async (token, options = {}, baseUrl = DEFAULT_SER
     }));
 };
 export const getAttachmentManifest = async (token, sandboxUid, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
-    // Throws on a Pod name. Contents validates this while building its
-    // reply, so sending the wrong spelling produced a 500 the caller could
-    // not read; this fails where the caller can be seen.
-    url: url(baseUrl, `/manifest/${encodeURIComponent(assertSandboxUid(sandboxUid))}`),
+    url: url(baseUrl, `/manifest/${encodeURIComponent(sandboxUid)}`),
     method: 'GET',
     token,
 }));

package/lib/api/contents/datasets.d.ts

@@ -5,6 +5,25 @@ export declare const getDatasetRevision: (token: string, sourceUid: string, revi
 export declare const createDatasetPublication: (token: string, sourceUid: string, request: DatasetPublicationCreate, idempotencyKey: string, baseUrl?: string) => Promise<DatasetPublication>;
 export declare const listDatasetPublications: (token: string, sourceUid: string, baseUrl?: string) => Promise<DatasetPublicationList>;
 export declare const unpublishDataset: (token: string, sourceUid: string, publicationUid: string, baseUrl?: string) => Promise<DatasetPublication>;
+/** The Dataset a clone produced, and where it came from. */
+export type DatasetClone = {
+    uid: string;
+    name: string;
+    sourceUid: string;
+    publicationUid: string;
+    revisionUid: string;
+    fileCount: number;
+    totalSize: number;
+};
+/**
+ * Take a published Dataset into your own catalog.
+ *
+ * A publication names one immutable revision, so the clone is a Dataset of
+ * the caller's own whose first revision names the same objects. Nothing is
+ * copied: the bytes cannot change, which is what makes sharing them safe, and
+ * the record and its permissions belong to the caller from that moment.
+ */
+export declare const cloneDatasetPublication: (token: string, sourceUid: string, publicationUid: string, idempotencyKey: string, baseUrl?: string) => Promise<DatasetClone>;
 /**
  * Every Dataset currently published, across sources.
  *
@@ -13,3 +32,32 @@ export declare const unpublishDataset: (token: string, sourceUid: string, public
  * Any signed-in caller may read it — published means public.
  */
 export declare const listPublishedDatasets: (token: string, baseUrl?: string) => Promise<DatasetPublicationList>;
+/** A published Data Server's catalog, as the library shows it. */
+export type DataServerPublication = {
+    uid: string;
+    sourceUid: string;
+    actorUid: string;
+    ownerUid: string;
+    name: string;
+    description?: string | null;
+    tags: string[];
+    connectors: string[];
+    relations: string[];
+    status: string;
+    createdAt: string;
+};
+/** The Datasource an attach produced, naming the published Data Server. */
+export type DataServerAttachment = {
+    uid: string;
+    name: string;
+    sourceUid: string;
+    publicationUid: string;
+};
+/**
+ * Attach a published Data Server as a Datasource of your own.
+ *
+ * The Datasource names it and the relations it publishes, and carries no
+ * credential: the Data Server's own mTLS still decides whether a query
+ * reaches it. That is what makes publishing a catalog safe.
+ */
+export declare const attachDataServerPublication: (token: string, sourceUid: string, publicationUid: string, idempotencyKey: string, baseUrl?: string) => Promise<DataServerAttachment>;

package/lib/api/contents/datasets.js

@@ -4,32 +4,56 @@
  */
 import { requestDatalayerAPI } from '../DatalayerApi.js';
 import { API_BASE_PATHS, DEFAULT_SERVICE_URLS } from '../constants.js';
-import { contentsToCamelCase, contentsToSnakeCase } from '../../models/contents/index.js';
+import { contentsToCamelCase, contentsToSnakeCase, } from '../../models/contents/index.js';
 const convert = (value) => contentsToCamelCase(value);
 const resourceUrl = (baseUrl, sourceUid, resource, suffix = '') => `${baseUrl}${API_BASE_PATHS.CONTENTS}/sources/${encodeURIComponent(sourceUid)}/${resource}${suffix}`;
 export const createDatasetRevision = async (token, sourceUid, request, idempotencyKey, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
-    url: resourceUrl(baseUrl, sourceUid, 'revisions'), method: 'POST', token,
+    url: resourceUrl(baseUrl, sourceUid, 'revisions'),
+    method: 'POST',
+    token,
     headers: { 'Idempotency-Key': idempotencyKey },
     body: contentsToSnakeCase(request),
 }));
 export const listDatasetRevisions = async (token, sourceUid, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
-    url: resourceUrl(baseUrl, sourceUid, 'revisions'), method: 'GET', token,
+    url: resourceUrl(baseUrl, sourceUid, 'revisions'),
+    method: 'GET',
+    token,
 }));
 export const getDatasetRevision = async (token, sourceUid, revisionUid, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
     url: resourceUrl(baseUrl, sourceUid, 'revisions', `/${encodeURIComponent(revisionUid)}`),
-    method: 'GET', token,
+    method: 'GET',
+    token,
 }));
 export const createDatasetPublication = async (token, sourceUid, request, idempotencyKey, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
-    url: resourceUrl(baseUrl, sourceUid, 'publications'), method: 'POST', token,
+    url: resourceUrl(baseUrl, sourceUid, 'publications'),
+    method: 'POST',
+    token,
     headers: { 'Idempotency-Key': idempotencyKey },
     body: contentsToSnakeCase(request),
 }));
 export const listDatasetPublications = async (token, sourceUid, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
-    url: resourceUrl(baseUrl, sourceUid, 'publications'), method: 'GET', token,
+    url: resourceUrl(baseUrl, sourceUid, 'publications'),
+    method: 'GET',
+    token,
 }));
 export const unpublishDataset = async (token, sourceUid, publicationUid, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
     url: resourceUrl(baseUrl, sourceUid, 'publications', `/${encodeURIComponent(publicationUid)}`),
-    method: 'DELETE', token,
+    method: 'DELETE',
+    token,
+}));
+/**
+ * Take a published Dataset into your own catalog.
+ *
+ * A publication names one immutable revision, so the clone is a Dataset of
+ * the caller's own whose first revision names the same objects. Nothing is
+ * copied: the bytes cannot change, which is what makes sharing them safe, and
+ * the record and its permissions belong to the caller from that moment.
+ */
+export const cloneDatasetPublication = async (token, sourceUid, publicationUid, idempotencyKey, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
+    url: resourceUrl(baseUrl, sourceUid, 'publications', `/${encodeURIComponent(publicationUid)}/clone`),
+    method: 'POST',
+    token,
+    headers: { 'Idempotency-Key': idempotencyKey },
 }));
 /**
  * Every Dataset currently published, across sources.
@@ -43,3 +67,16 @@ export const listPublishedDatasets = async (token, baseUrl = DEFAULT_SERVICE_URL
     method: 'GET',
     token,
 }));
+/**
+ * Attach a published Data Server as a Datasource of your own.
+ *
+ * The Datasource names it and the relations it publishes, and carries no
+ * credential: the Data Server's own mTLS still decides whether a query
+ * reaches it. That is what makes publishing a catalog safe.
+ */
+export const attachDataServerPublication = async (token, sourceUid, publicationUid, idempotencyKey, baseUrl = DEFAULT_SERVICE_URLS.CONTENTS) => convert(await requestDatalayerAPI({
+    url: `${baseUrl}/api/contents/v1/sources/${encodeURIComponent(sourceUid)}/dataserver-publications/${encodeURIComponent(publicationUid)}/attach`,
+    method: 'POST',
+    token,
+    headers: { 'Idempotency-Key': idempotencyKey },
+}));

package/lib/api/contents/runtimeMounts.js

@@ -8,7 +8,7 @@
  * A Pod's volumes are fixed when it is created, so for most of a sandbox's
  * life its mounts were decided before it existed. The mount gateway lifts
  * that for the Home Folder: the platform binds a folder into a running pod
- * and the sandbox sees it at `/home/jovyan/{handle}` within a second or two,
+ * and the sandbox sees it at `/home/datalayer/{handle}` within a second or two,
  * with no restart.
  *
  * Not every Runtime can take one — a pod created before the gateway cannot,

package/lib/api/contents/sandboxUid.d.ts

@@ -1,53 +1,21 @@
 /**
- * A sandbox is identified by its uid, never by its runtime's Pod name.
+ * A sandbox is identified by its uid, a ULID.
  *
- * A Datalayer runtime has a ULID; its Kubernetes Pod is named
- * `runtime-<ulid>`, because a Pod name has to be a DNS label. Route
- * parameters and `runtime_name` fields carry the Pod name, and every Contents
- * record keyed on a sandbox is keyed on the ULID.
- *
- * Passing the Pod name where a uid belongs used to *work* at the call site
- * and fail far away: Contents validates `sandbox_uid` while building its
- * reply, so a Pod name became an unhandled 500 that reached the browser as
- * "Network Error", with nothing readable in it and nothing pointing at the
- * caller. Throwing here names the caller in the stack instead.
+ * A Datalayer runtime has one name: its uid. The Pod is called by it, the
+ * `runtime_name` field and every `/runtimes/{uid}` route carry it, and
+ * Contents keys an attachment on it. There used to be a second spelling — a
+ * Pod name `runtime-<ulid>` — and every place the two met needed a
+ * converter; the converters went with the prefix.
  *
  * The server half of this is `datalayer_common.runtime_names`.
  */
-/** What a runtime Pod's name is prefixed with. */
-export declare const RUNTIME_NAME_PREFIX = "runtime-";
-/** Thrown when a Pod name is given where a sandbox uid is required. */
-export declare class RuntimeNameNotAUid extends Error {
-    readonly value: string;
-    constructor(value: string);
-}
-/**
- * Assert that a value is a sandbox uid, and return it.
- *
- * Use this wherever a `sandbox_uid` is sent. A non-Datalayer identifier —
- * Daytona, E2B and Modal name their own sandboxes — passes through: there is
- * no ULID to check it against, and inventing one would be worse.
- */
-export declare function assertSandboxUid(value: string): string;
-/**
- * The Pod name for a sandbox uid, converting on purpose.
- *
- * The mirror of {@link sandboxUidFromRuntimeName}, and the direction that was
- * missing: route parameters and `runtime_name` fields carry the **Pod name**,
- * so anything addressing `/runtimes/{name}/…` needs one. Without this, a
- * caller holding only a uid either wrote the prefix itself or — the version
- * that was here — passed the uid and hoped, which is a `404` from a route
- * whose parameter looked plausible.
- *
- * Idempotent, like the server's `runtime_name()`: a name that has been
- * through here stays one name rather than growing a second prefix.
- */
-export declare function runtimeNameFromSandboxUid(value: string): string;
+/** Crockford base32, 26 characters — a ULID, in either case. */
+export declare const SANDBOX_UID_PATTERN: RegExp;
 /**
- * The sandbox uid for a Pod name, converting on purpose.
+ * Whether a value is a sandbox uid.
  *
- * For the few callers that genuinely only hold the Pod name — a route
- * parameter, a label read off a Pod. Everywhere else should already have the
- * uid, and {@link assertSandboxUid} is what says so.
+ * What tells a runtime apart from anything else that can stand in a route:
+ * an agentspec id, a word the listing itself addresses. A sandbox at an
+ * external provider is named `external-<provider>-<uid>` and is not one.
  */
-export declare function sandboxUidFromRuntimeName(value: string): string;
+export declare function isSandboxUid(value: string | undefined | null): boolean;

package/lib/api/contents/sandboxUid.js

@@ -3,76 +3,25 @@
  * Distributed under the terms of the Modified BSD License.
  */
 /**
- * A sandbox is identified by its uid, never by its runtime's Pod name.
+ * A sandbox is identified by its uid, a ULID.
  *
- * A Datalayer runtime has a ULID; its Kubernetes Pod is named
- * `runtime-<ulid>`, because a Pod name has to be a DNS label. Route
- * parameters and `runtime_name` fields carry the Pod name, and every Contents
- * record keyed on a sandbox is keyed on the ULID.
- *
- * Passing the Pod name where a uid belongs used to *work* at the call site
- * and fail far away: Contents validates `sandbox_uid` while building its
- * reply, so a Pod name became an unhandled 500 that reached the browser as
- * "Network Error", with nothing readable in it and nothing pointing at the
- * caller. Throwing here names the caller in the stack instead.
+ * A Datalayer runtime has one name: its uid. The Pod is called by it, the
+ * `runtime_name` field and every `/runtimes/{uid}` route carry it, and
+ * Contents keys an attachment on it. There used to be a second spelling — a
+ * Pod name `runtime-<ulid>` — and every place the two met needed a
+ * converter; the converters went with the prefix.
  *
  * The server half of this is `datalayer_common.runtime_names`.
  */
-/** What a runtime Pod's name is prefixed with. */
-export const RUNTIME_NAME_PREFIX = 'runtime-';
-/** Thrown when a Pod name is given where a sandbox uid is required. */
-export class RuntimeNameNotAUid extends Error {
-    value;
-    constructor(value) {
-        super(`'${value}' is a runtime Pod name, not a sandbox uid. A sandbox is ` +
-            `identified by its ULID ('${value.slice(RUNTIME_NAME_PREFIX.length)}'). ` +
-            'If all you hold is the Pod name, convert it deliberately with ' +
-            'sandboxUidFromRuntimeName().');
-        this.name = 'RuntimeNameNotAUid';
-        this.value = value;
-    }
-}
-/**
- * Assert that a value is a sandbox uid, and return it.
- *
- * Use this wherever a `sandbox_uid` is sent. A non-Datalayer identifier —
- * Daytona, E2B and Modal name their own sandboxes — passes through: there is
- * no ULID to check it against, and inventing one would be worse.
- */
-export function assertSandboxUid(value) {
-    if (value.startsWith(RUNTIME_NAME_PREFIX)) {
-        throw new RuntimeNameNotAUid(value);
-    }
-    return value;
-}
-/**
- * The Pod name for a sandbox uid, converting on purpose.
- *
- * The mirror of {@link sandboxUidFromRuntimeName}, and the direction that was
- * missing: route parameters and `runtime_name` fields carry the **Pod name**,
- * so anything addressing `/runtimes/{name}/…` needs one. Without this, a
- * caller holding only a uid either wrote the prefix itself or — the version
- * that was here — passed the uid and hoped, which is a `404` from a route
- * whose parameter looked plausible.
- *
- * Idempotent, like the server's `runtime_name()`: a name that has been
- * through here stays one name rather than growing a second prefix.
- */
-export function runtimeNameFromSandboxUid(value) {
-    if (!value || value.startsWith(RUNTIME_NAME_PREFIX)) {
-        return value;
-    }
-    return `${RUNTIME_NAME_PREFIX}${value}`;
-}
+/** Crockford base32, 26 characters — a ULID, in either case. */
+export const SANDBOX_UID_PATTERN = /^[0-9A-HJKMNP-TV-Za-hjkmnp-tv-z]{26}$/;
 /**
- * The sandbox uid for a Pod name, converting on purpose.
+ * Whether a value is a sandbox uid.
  *
- * For the few callers that genuinely only hold the Pod name — a route
- * parameter, a label read off a Pod. Everywhere else should already have the
- * uid, and {@link assertSandboxUid} is what says so.
+ * What tells a runtime apart from anything else that can stand in a route:
+ * an agentspec id, a word the listing itself addresses. A sandbox at an
+ * external provider is named `external-<provider>-<uid>` and is not one.
  */
-export function sandboxUidFromRuntimeName(value) {
-    return value.startsWith(RUNTIME_NAME_PREFIX)
-        ? value.slice(RUNTIME_NAME_PREFIX.length)
-        : value;
+export function isSandboxUid(value) {
+    return !!value && SANDBOX_UID_PATTERN.test(value);
 }