@evolvingmachines/evolve 0.0.65 → 0.0.67

package/skills/evolve-evals/references/sdk-reference/methods/meta.md

@@ -0,0 +1,209 @@
+---
+title: "Capabilities: method and fields"
+description: "Read harness, model, provider, status, and limit information."
+---
+
+Read live capability values rather than storing a fixed harness, model, or provider list. No API key is needed.
+
+Python examples run inside an async function.
+
+## meta
+
+Fetch the capability document from the selected API origin.
+
+```ts TypeScript
+import { meta } from "@evolvingmachines/evolve";
+
+const capabilities = await meta();
+console.log(
+  capabilities.agents,
+  capabilities.limits,
+);
+```
+
+```python Python
+from evolve import meta
+
+capabilities = await meta()
+print(
+    capabilities.agents,
+    capabilities.limits,
+)
+```
+
+**Returns:** `CapabilityDocument`. Also available as `hosted().meta()`.
+
+### Parameters and behavior
+
+| Parameter | Type and meaning |
+| --- | --- |
+| TypeScript `meta(config?)` | `HostedClientConfig`; optional `baseUrl` chooses the origin. |
+| Python `meta(config=None)` | `HostedClientConfig`; optional `base_url` chooses the origin. |
+
+Origin precedence: explicit config, `EVOLVE_DASHBOARD_URL`, then `https://dashboard.evolvingmachines.ai`. `apiKey` / `api_key` and `org` are not sent. `schema_version` identifies the capability schema, not the SDK release.
+
+## Capability fields
+
+TypeScript returns the typed object below. Python uses dataclasses for the document, agents, providers, and status vocabularies. Nested policy objects such as `limits`, `agent_registration`, `analyze`, provider `sizing`, and provider `gpus` are Python dictionaries.
+
+### CapabilityDocument: SDK field definitions
+
+`limits` reports current values for this deployment. `default_model` is a picker suggestion; managed evaluation jobs still require a model. Optional fields may be absent on older servers; Python maps missing `analyze` and `gpu_concurrency_cap` to `None`.
+
+```ts Fields
+interface CapabilityDocument {
+  schema_version: number;
+  agents: AgentCapability[];
+  agent_registration: {
+    name_pattern: string;
+    max_name_length: number;
+    max_run_command_length: number;
+    max_install_script_length: number;
+    max_env_entries: number;
+    max_per_user: number;
+    max_upload_bytes: number;
+    reserved_names: string[];
+    reserved_env_keys: string[];
+  };
+  sandbox_providers: ProviderCapability[];
+  gpu_concurrency_cap?: number;
+  managed_providers: ManagedProviderCapability[];
+  platform_constraints: { capability: string; reason: string }[];
+  network_modes: string[];
+  statuses: {
+    job: StatusVocabulary;
+    trial: StatusVocabulary;
+    import: StatusVocabulary;
+    dataset_version: StatusVocabulary;
+  };
+  analyze?: {
+    default_model: string;
+    reasoning_efforts: string[];
+    models: { alias: string; model_id: string; default_reasoning_effort: string }[];
+  };
+  limits: {
+    job: {
+      n_concurrent_trials: { default: number; max: number };
+      default_max_trial_spend_usd: number;
+      default_max_retries: number;
+      default_sandbox_provider: string;
+      default_sizing: { cpus: number; memory_mb: number; storage_mb: number };
+      model_required: boolean;
+      default_agent_timeout_sec: number;
+      default_verifier_timeout_sec: number;
+      reasoning_efforts: string[];
+      default_reasoning_effort: string;
+    };
+    compare: { min_ids: number; max_ids: number };
+    pagination: {
+      collections: { default: number; max: number };
+      dataset_tasks: { default: number; max: number };
+      trace_events: { default: number; max: number };
+    };
+    uploads: {
+      dataset_archive_bytes: number;
+      job_archive_bytes: number;
+      archive_bytes_source: string;
+      agent_tarball_bytes: number;
+      skill_archive_bytes: number;
+      skill_uploads_per_user: number;
+    };
+    dataset_names: {
+      pattern: string;
+      max_name_length: number;
+      max_version_length: number;
+      max_git_url_length: number;
+      max_git_ref_length: number;
+      max_git_path_length: number;
+    };
+    max_items_named_in_error_message: number;
+  };
+  import_warning_codes: string[];
+  error_codes: string[];
+}
+```
+
+### Additional fields returned by the API
+
+The API also returns the fields below. TypeScript keeps them in the response, although this SDK's `CapabilityDocument` type does not declare them. Python keeps the nested `limits` and `sizing` dictionaries, but its status objects omit `description`.
+
+| Field | Meaning |
+| --- | --- |
+| `limits.job.default_timeout_multiplier` | Default multiplier for phase time limits. |
+| `limits.job.binary_effort_values` | Accepted values for harnesses whose reasoning can only be on or off. |
+| `limits.job.max_job_name_length` | Maximum job-name length. |
+| `limits.stop.max_trial_ids`, `max_run_ids` | Maximum IDs in the respective stop requests. Both fields belong to `limits.stop`. |
+| `limits.dataset_names.reserved_names` | Names reserved for dataset API routes. |
+| `sandbox_providers[].sizing.storage_ceiling_source` | Where the storage limit came from: `live-quota`, `fallback`, or `provider-constant`. |
+| `statuses.*.description` | Description of each status set; available in the HTTP response and TypeScript object. |
+
+### Harnesses and models
+
+`effort_support` is `level`, `binary`, or `none`. `runnable: false` includes a reason. `supports_config` controls native settings support; `presets` lists accepted named presets. Python defaults missing `supports_config` to `False` and `presets` to an empty list. `latest_version: null` means unknown, not current.
+
+```ts Fields
+interface AgentModelOption {
+  alias: string;
+  model_id: string;
+  description: string | null;
+}
+
+interface AgentCapability {
+  name: string;
+  runnable: boolean;
+  reason: string | null;
+  default_model: string | null;
+  models: AgentModelOption[];
+  effort_support: AgentEffortSupport;
+  default_effort: string | null;
+  version_pinnable: boolean;
+  supports_config?: boolean;
+  presets?: string[];
+  latest_version?: string | null;
+}
+```
+
+### Evaluation sandbox providers
+
+`sizing.storage` distinguishes chosen storage sizes from a fixed allocation. GPU support may declare a fallback through `degrades_to`. `gpus.source` distinguishes a live quota result, fallback, or provider constant. `refuses` names unsupported capabilities.
+
+```ts Fields
+interface ProviderCapability {
+  name: string;
+  default: boolean;
+  sizing: {
+    max_cpus: number;
+    max_memory_mb: number;
+    max_storage_mb: number;
+    storage: "sized" | "fixed";
+  };
+  gpus?: {
+    supported: boolean;
+    max_gpus: number;
+    degrades_to?: "modal";
+    reason?: string;
+    source?: "live-quota" | "fallback" | "provider-constant";
+  };
+  refuses: { capability: string; reason: string }[];
+}
+```
+
+### Managed runtime providers and status vocabularies
+
+`configured` checks required operator configuration; it is not a service health check. `agent_sessions` says whether the separate managed-agent runtime supports full sessions through that provider. Status vocabularies list all values and their terminal subset.
+
+```ts Fields
+interface ManagedProviderCapability {
+  name: string;
+  configured: boolean;
+  requires_config: string[];
+  missing_config: string[];
+  agent_sessions: boolean;
+  agent_sessions_reason: string | null;
+}
+
+interface StatusVocabulary {
+  values: string[];
+  terminal: string[];
+}
+```

package/skills/evolve-evals/references/sdk-reference/methods/secrets.md

@@ -0,0 +1,157 @@
+---
+title: "Secrets: methods"
+description: "Store credentials and read or delete their metadata."
+---
+
+Use `managedSecrets()` in TypeScript or `managed_secrets()` in Python. This client has its own configuration and is separate from `hosted()`. Examples use `store` for that client.
+
+Managed evaluations accept `direct` secrets. Their values enter the sandbox as environment variables. The storage client also supports brokered secrets for the separate managed-agent runtime.
+
+Python examples run inside an async function.
+
+## list
+
+Read stored secret metadata. Secret values are never returned.
+
+```ts TypeScript
+const secrets = await store.list();
+```
+
+```python Python
+secrets = await store.list()
+```
+
+**Returns:** `ManagedSecretMetadata[]` / `List[ManagedSecretMetadata]`. No arguments; no pagination.
+
+## set
+
+Store a value under a name and optional label.
+
+```ts TypeScript
+const result = await store.set({
+  name: "SERVICE_TOKEN",
+  label: "v2",
+  value: process.env.SERVICE_TOKEN!,
+  delivery: "direct",
+});
+```
+
+```python Python
+import os
+
+result = await store.set(
+    name="SERVICE_TOKEN",
+    label="v2",
+    value=os.environ["SERVICE_TOKEN"],
+    delivery="direct",
+)
+```
+
+**Returns:** `ManagedSecretWriteResult`: `status` (`created` or `updated`) and `secret` metadata.
+
+### Parameters and behavior
+
+| Parameter | Type and meaning |
+| --- | --- |
+| `name` | Required string. Trimmed and converted to uppercase; 1–128 letters, digits, or underscores. Must start with a letter or underscore; `EVOLVE_` is reserved. |
+| `value` | Required, non-empty string; at most 190 UTF-8 bytes. |
+| `delivery` | Required `direct` or `brokered`. |
+| `label?` | String, at most 80 letters, digits, dots, underscores, or hyphens. Whitespace is trimmed; omitted or empty means `default`. |
+| `allowedHosts?` / `allowed_hosts?` | String list of hostnames or wildcard hostnames, such as `api.example.com` or `*.example.com`. No scheme, port, or path. |
+| `allowedPathPrefixes?` / `allowed_path_prefixes?` | String list of paths starting with `/`. No `..` segments or encoded slashes. |
+| `allowedMethods?` / `allowed_methods?` | String list: `GET`, `POST`, `PUT`, `PATCH`, `DELETE`, or `HEAD`. Converted to uppercase. |
+
+Each scope list accepts at most 50 entries after trimming and removing blank or duplicate strings. Hostnames allow at most 253 characters; path prefixes allow at most 512.
+
+Direct delivery accepts omitted or empty scope lists and refuses non-empty lists. Brokered delivery requires all three lists to be non-empty. Managed eval jobs currently cannot attach brokered secrets.
+
+Restating the same name, label, and value returns `updated` and can update delivery/scope. A different value at the same identity returns `secret_exists`. Rotate using a new label, or delete and set that identity.
+
+## delete
+
+Delete the stored row for a name and label.
+
+```ts TypeScript
+const result = await store.delete({
+  name: "SERVICE_TOKEN",
+  label: "v1",
+});
+```
+
+```python Python
+result = await store.delete(
+    name="SERVICE_TOKEN",
+    label="v1",
+)
+```
+
+**Returns:** `ManagedSecretDeleteResult`: `ok: boolean`, `name: string`, and the deleted `label: string`.
+
+### Parameters and behavior
+
+| Parameter | Type and meaning |
+| --- | --- |
+| `name` | Required string, normalized by the same rules as `set`. |
+| `label?` | If provided: 1–80 allowed characters after trimming; empty labels are refused. If omitted: choose the `default` row, otherwise the only row, otherwise refuse the ambiguous choice. |
+
+## Configuration
+
+```ts TypeScript
+import { managedSecrets } from "@evolvingmachines/evolve";
+
+const store = managedSecrets({
+  apiKey: process.env.EVOLVE_API_KEY,
+  dashboardUrl: "https://dashboard.evolvingmachines.ai",
+});
+```
+
+```python Python
+from evolve import managed_secrets, ManagedSecretsClientConfig
+
+store = managed_secrets(ManagedSecretsClientConfig(
+    dashboard_url="https://dashboard.evolvingmachines.ai",
+))
+```
+
+| TypeScript config | Python config | Default |
+| --- | --- | --- |
+| `apiKey` | `api_key` | `EVOLVE_API_KEY`; required when making a request. |
+| `dashboardUrl` | `dashboard_url` | `EVOLVE_DASHBOARD_URL`, then `https://dashboard.evolvingmachines.ai`. |
+
+HTTP failures raise plain `Error` in TypeScript and `RuntimeError` in Python, rather than hosted `EvolveApiError` / `EvolveAPIError`.
+
+## Secret fields
+
+### Metadata, write result, and delete result
+
+Python returns dataclasses. Its metadata uses `allowed_hosts`, `allowed_path_prefixes`, `allowed_methods`, `created_at`, `updated_at`, and `last_used_at` instead of their camelCase TypeScript names. Other fields keep their names. Missing optional `label` / `delivery` is `None` in Python. No result exposes `value`.
+
+```ts Fields
+interface ManagedSecretMetadata {
+  id: string;
+  name: string;
+  label?: string;
+  delivery?: ManagedSecretDelivery;
+  allowedHosts: string[];
+  allowedPathPrefixes: string[];
+  allowedMethods: string[];
+  createdAt: string;
+  updatedAt: string;
+  lastUsedAt: string | null;
+}
+
+type ManagedSecretDelivery = "brokered" | "direct";
+
+interface ManagedSecretWriteResult {
+  status: "created" | "updated";
+  secret: ManagedSecretMetadata;
+}
+
+interface ManagedSecretDeleteResult {
+  ok: boolean;
+  name: string;
+  label: string;
+}
+```
+
+See [attach a stored secret](/sdk-reference/secrets#attach-it-to-a-job) for the job input. Provider API keys are managed separately in the dashboard.

package/skills/evolve-evals/references/sdk-reference/methods/skills.md

@@ -0,0 +1,125 @@
+---
+title: "Uploaded skills: methods"
+description: "Upload, read, and delete skill content used by evaluation agents."
+---
+
+Use `skills()` or `hosted().skills`. These are skill folders mounted into runs; `evolve skills` separately serves documentation. Examples use a configured `client`. Python examples run inside an async function.
+
+## upload
+
+Store one skill folder or a root containing child skill folders.
+
+```ts TypeScript
+const uploaded = await client.upload(
+  "./my-skill",
+  {
+    org: "my-team",
+  },
+);
+```
+
+```python Python
+uploaded = await client.upload(
+    "./my-skill",
+    org="my-team",
+)
+```
+
+**Returns:** An array/list of `SkillUpload`, one per discovered skill.
+
+### Parameters and behavior
+
+| Parameter | Type and meaning |
+| --- | --- |
+| `directory` | Required local path. Contains `SKILL.md`, or immediate child folders that contain it. |
+| `org?` | Organization slug or id. Overrides the client default; otherwise uses your personal organization. |
+
+Identical content under the same name reuses a record. Different content creates a new immutable `upload:<id>` reference and moves `name:<skill-name>` to that record. Existing jobs keep their locked content.
+
+## list
+
+Read one page or iterate all visible uploaded skills.
+
+```ts TypeScript
+const page = await client.list({
+  scope: "org",
+  limit: 20,
+});
+```
+
+```python Python
+page = await client.list(
+    scope="org",
+    limit=20,
+)
+```
+
+**Returns:** `SkillUploadPage` when awaited; a `SkillUpload` per iteration.
+
+### Parameters and behavior
+
+| Parameter | Type and meaning |
+| --- | --- |
+| `scope?` | `my` (default), `shared` (other members’ uploads in your organizations), or `org` (all uploads in your organizations). |
+| `limit?`, `cursor?` | Collection pagination; default 50, maximum 200. `for await` / `async for` follows pages. |
+
+Page fields are `items`, `nextCursor`, `hasMore` in TypeScript; `items`, `next_cursor`, `has_more` in Python.
+
+## get
+
+Read uploaded metadata and its `SKILL.md` content.
+
+```ts TypeScript
+const skill = await client.get("name:my-skill");
+console.log(skill.skill_md);
+```
+
+```python Python
+skill = await client.get("name:my-skill")
+print(skill.skill_md)
+```
+
+**Returns:** TypeScript `SkillUpload & { skill_md: string | null }`; Python `SkillUpload` with `skill_md` populated.
+
+### Parameters and behavior
+
+Pass a record id or `name:<skill-name>` as the required argument (`id` in TypeScript, `skill_id` in Python). A name reference resolves only your own current upload. Use a record id for an upload shared through your organization. An unknown name returns `skill_name_not_found`.
+
+## delete
+
+Delete an uploaded record you own.
+
+```ts TypeScript
+await client.delete(skillId);
+```
+
+```python Python
+await client.delete(skill_id)
+```
+
+**Returns:** No value (`void` / `None`).
+
+### Parameters and behavior
+
+Pass the record id, not a `name:` reference. A non-terminal job using the record prevents deletion. Completed jobs retain their recorded skill locks.
+
+## Skill fields
+
+### SkillUpload response
+
+`digest` is `sha256:<hex>`. `ref` is the immutable `upload:<id>` handle used by job agent arms. `description` is extracted from `SKILL.md`.
+
+Python returns these fields as dataclass attributes and adds `skill_md: Optional[str]`, defaulting to `None` for list/upload results. TypeScript adds `skill_md` only to the `get` result.
+
+```ts Fields
+interface SkillUpload {
+  id: string;
+  name: string;
+  org: string | null;
+  digest: string;
+  size_bytes: number;
+  description: string | null;
+  ref: string;
+  created_at: string;
+}
+```