@evolvingmachines/evolve 0.0.65 → 0.0.66

package/skills/evolve-evals/references/sdk-reference/errors.mdx

@@ -1,210 +0,0 @@
----
-title: "Errors"
-description: "The error class every refused request raises, and every code it can carry."
----
-
-A refused request raises one class. Branch on its `code`, never on the message.
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    import { EvolveApiError, HOSTED_ERROR_CODES, isHostedErrorCode } from "@evolvingmachines/evolve";
-
-    class EvolveApiError extends Error {
-      status: number;                 // the HTTP status
-      code: HostedErrorCode | string; // the stable code below; "unknown_error" when the server sent none, or one a newer server added
-      message: string;                // the server's sentence; may be shortened when a list is long
-      param?: string;                 // the input the refusal is about: a body path, a query parameter, or a multipart part
-      details?: Record<string, unknown>; // the complete machine-readable data; never shortened
-      retryAfterSec?: number;         // on 429 and 503, from the body first, the Retry-After header second
-      requestId?: string;             // the server's id for this failure, from x-request-id; quote it in support
-      isKnownCode(): boolean;
-    }
-    ```
-
-    ```ts
-    try {
-      await jobs().start({ datasets: [{ name: "terminal-bench-4" }], agents: [{ name: "codex", model_name: "gpt-6-astra" }], sandbox_provider: "modal" });
-    } catch (err) {
-      if (err instanceof EvolveApiError && err.code === "provider_unsupported") {
-        const { refused_tasks } = err.details as { refused_tasks: { task_name: string; reason: string }[] };
-      }
-    }
-    ```
-
-    `HOSTED_ERROR_CODES` is the closed list as a runtime array and `isHostedErrorCode(value)` narrows a string to it. The same list is published as `error_codes` in the [capability document](/sdk-reference/meta).
-  </Tab>
-  <Tab title="Python">
-    ```python
-    from evolve import EvolveAPIError, HOSTED_ERROR_CODES, is_hosted_error_code
-
-    class EvolveAPIError(Exception):
-        status: int                        # the HTTP status
-        code: str                          # the stable code below; "unknown_error" when the server sent none
-        param: Optional[str]               # the input the refusal is about
-        details: Optional[Dict[str, Any]]  # the complete machine-readable data; never shortened
-        retry_after_sec: Optional[float]   # on 429 and 503
-        request_id: Optional[str]          # the server's id for this failure; quote it in support
-        def is_known_code(self) -> bool: ...
-    ```
-
-    ```python
-    try:
-        await jobs().start(datasets=[{"name": "terminal-bench-4"}], agents=[{"name": "codex", "model_name": "gpt-6-astra"}], sandbox_provider="modal")
-    except EvolveAPIError as err:
-        if err.code == "provider_unsupported":
-            refused = (err.details or {}).get("refused_tasks", [])
-    ```
-
-    `str(err)` is the server's sentence, which may be shortened when a list is long. `HOSTED_ERROR_CODES` is the closed list and `is_hosted_error_code(value)` tests a string against it.
-  </Tab>
-</Tabs>
-
-Every response, success or failure, carries the same id in its `x-request-id` header. Of the 429 codes only `rate_limited` carries a retry delay; `quota_exceeded` and every `too_many_concurrent_*` code carry none.
-
-## Errors that are not API refusals
-
-These mean the request succeeded, or never happened, and something else went wrong. They are separate classes, not codes.
-
-- `NoActiveVersionError`: `datasets().getActive()` (Python `get_active`) on a dataset with no active version. Carries `dataset`.
-- `ImportSettleError`: `watchImport()` (Python `watch_import`) reached its settle timeout before the version settled. `code` is `settle_timeout`; carries `importId` (Python `import_id`), `dataset`, `version` and the last observed `state`. When `state` is `FAILED` the version did settle; read the failure with `getImport()` (Python `get_import`).
-- `EvolveDigestMismatchError`: a download's bytes do not match the digest the server stated. Carries `expected` and `actual`. Do not trust the stored object.
-- `EvolveIncompleteDownloadError`: fewer bytes arrived than `Content-Length` promised. Carries `expectedBytes` and `receivedBytes` (Python `expected_bytes`, `received_bytes`). Retry the download.
-- `EvolveUploadTimeoutError`, TypeScript only: an upload saw no socket activity for 600 seconds. Retry the upload.
-
-## The codes
-
-Grouped by the door that returns them. The HTTP status is stated where it is fixed.
-
-### Keys and limits
-
-| Code | When |
-| --- | --- |
-| `missing_authorization` | No API key on the request. |
-| `invalid_api_key` | The key is not valid. |
-| `read_only_key` | 403. A read-only key on a mutating route. |
-| `credential_service_unavailable` | The key could not be checked; retry. |
-| `rate_limited` | 429. Wait `retryAfterSec`; every SDK watch loop does so on its own. |
-| `insufficient_credits` | 402. The account's credit balance is zero at job create, resume or retry. |
-| `quota_exceeded` | 429, no retry delay. The organization's `max_queued_trials` would be crossed. `details` carry `quota`, `limit`, `used`, `requested` and `org`. The CLI prints `Launch quota exceeded:` and exits 2. |
-
-### Files and logs
-
-| Code | When |
-| --- | --- |
-| `not_found` | 404. A path the run's file system does not hold. |
-| `not_captured` | 404. A read of a file on the kept tree that the run never touched; only the image holds it. |
-| `filesystem_state` | 409. `source=live` while the box is gone, or `source=capture` before the kept tree exists; `details.state` says what the run has. |
-| `feature_unsupported` | 422. The provider or the run's shape has no live view of what was asked; `details.cause` names it. |
-| `provider_unreachable` | 503. The box's provider did not answer; wait `retryAfterSec`. |
-| `task_package_not_retained` | 409. The dataset version keeps no task package, so its task files cannot be served. |
-
-### Input
-
-| Code | When |
-| --- | --- |
-| `invalid_json` | The body is not JSON. |
-| `invalid_input` | A field is malformed; the message names it and `param` names it when the server can. Also the 413 for a trial home too large to serve as one text view (`details`: `size_bytes`, `max_unranged_bytes`). |
-| `invalid_limit`, `invalid_cursor`, `invalid_after` | A paging parameter is malformed. |
-| `invalid_status`, `invalid_visibility`, `invalid_format` | A filter or selector names a value outside its closed set. |
-| `invalid_ids` | 400. A stop batch that is empty or over 100 ids. |
-| `invalid_multipart` | 400. An upload that is not `multipart/form-data`, or is malformed. |
-| `idempotency_key_reused` | 409. The key already stands for a different request. |
-| `concurrent_update` | The row changed under the request; read it again. |
-
-### Datasets and publishing
-
-| Code | When |
-| --- | --- |
-| `dataset_not_found` | The name does not exist, or belongs to someone else; the two are never told apart. |
-| `dataset_version_not_found` | The version does not exist. |
-| `dataset_name_taken` | 409. The name belongs to another publisher. |
-| `dataset_in_use` | 409. Jobs reference the dataset; `details` name them. |
-| `dataset_not_owned` | The caller does not own it; a platform dataset has no owner. |
-| `dataset_import_in_progress` | 409. A visibility change while a version is importing; `details` name the version. |
-| `upstream_not_watchable` | Auto-import on a dataset with no moving git ref. |
-| `no_active_version` | A bare name on a dataset with no active version. |
-| `version_not_ready` | 409. A job or an activation on a version that is not READY. |
-| `version_not_activatable` | A FAILED or ARCHIVED version. |
-| `unknown_task_names`, `no_tasks` | A selector names tasks the version lacks, or filters every task away. |
-| `task_not_found` | 404. `getTaskBuild()` on a name with no recorded outcome. |
-| `task_failed_to_build` | A job create that names a task whose build FAILED; `details.failed_tasks` quote each reason. |
-| `unpinned_git_ref` | A git publish on a branch; `details.commit` is the sha to pin. |
-| `hub_package_not_found` | 400. The hub package does not exist or is private; `details.hub_package`. |
-| `hub_unreachable` | 502. The hub could not be asked; retry the publish. |
-| `invalid_archive` | The tarball is unreadable or unsafe. |
-| `import_too_large` | 413. A dataset archive over the published ceiling. |
-| `import_not_found` | The import id does not exist or is not yours. |
-| `too_many_concurrent_imports` | 429, no retry delay. Too many uploads in flight; retry when one finishes. `details.max_concurrent`. |
-| `package_not_retained`, `package_corrupt`, `package_missing` | 409. The original package was never retained, is corrupt, or is gone. Re-publish. |
-| `too_many_concurrent_package_downloads` | 429, no retry delay. `details.max_concurrent`. |
-| `upload_session_not_found`, `upload_offset_mismatch`, `upload_chunk_digest_mismatch`, `upload_incomplete`, `upload_archive_digest_mismatch`, `upload_session_failed`, `too_many_concurrent_upload_chunks` | A large upload's session failed; the SDK surfaces the refusal as is. Retry the publish or upload. |
-
-### Jobs, trials, regrades
-
-| Code | When |
-| --- | --- |
-| `provider_unsupported` | 422. A selected task is refused on the chosen provider; `details.refused_tasks` carry each reason. A GPU task that moves to `modal` is not refused; the trial records the move as `sandbox_provider_degrade`. |
-| `job_not_found` | The job does not exist or is not visible to you. A regrade job id on delete. |
-| `job_not_terminal` | 409. Resume, retry, regrade, analyze or delete on a live job; on delete also a live regrade, listed in `details.regrade_job_ids`. |
-| `no_failed_trials` | 409. Resume, or retry with `failed_only`, found nothing. |
-| `trial_not_found` | 404. A retry names a trial the job does not own; a trace feed id no run owns. |
-| `trial_not_settled` | 409. A retry names a trial still running. |
-| `regrade_source_ineligible` | 409. The trial recorded no verifier inputs; the message says why. |
-| `no_regradable_trials` | 409. A whole-job regrade found nothing eligible. |
-| `org_forbidden` | 403. A member who did not create the job tried to delete it. |
-| `job_uploaded` | 409. Resume, retry or regrade on an uploaded job. |
-| `secret_not_found`, `secret_ambiguous`, `secret_brokered_unsupported`, `secret_exists`, `secret_not_attached` | See [secrets](/core-concepts/secrets). |
-| `agent_version_not_found` | A version pin that does not resolve. |
-| `agent_version_unresolvable` | 502. The latest version could not be looked up; retry the create. |
-| `agent_kwarg_unsupported`, `agent_config_unsupported`, `agent_config_key_refused`, `agent_preset_unsupported` | See [agents](/core-concepts/agents). |
-
-### Analyze and check
-
-| Code | When |
-| --- | --- |
-| `invalid_rubric` | 400. Unknown keys, empty or duplicate criteria, or a length bound exceeded. |
-| `analysis_already_running` | 409. One analysis wave at a time; also blocks delete. |
-| `no_analyzable_trials` | 409. Every trial is CANCELLED, or the filter selects nothing. |
-| `analysis_not_found` | 404. An analysis you cannot read or that never existed. |
-| `analysis_not_terminal` | 409. Download of an analysis still queued or running. |
-| `check_not_found` | 404. Neither a check nor a task check you can read. |
-| `check_not_terminal` | 409. Download of a check or task check not yet settled. |
-| `no_checkable_tasks` | 400. No task directory in the archive, or the globs and the cap left nothing. |
-| `too_many_concurrent_check_uploads` | 429, no retry delay. `details.max_concurrent`. |
-
-### Job upload
-
-The upload door itself answers four codes: `invalid_multipart` (400), `invalid_input` (400), `upload_too_large` (413) and `too_many_concurrent_job_uploads` (429). Everything else arrives on the import's `failure`, with no HTTP status.
-
-| Code | When |
-| --- | --- |
-| `not_a_job_dir` | No `result.json` and `config.json` at the root, or they do not parse. |
-| `invalid_archive` | The tarball is unreadable or unsafe. |
-| `invalid_trial` | One trial's `result.json` cannot be ingested; the failure names the trial. |
-| `trial_too_large` | Not a failure: the trial is skipped and listed on `skipped_trials`; `details` carry `file`, `bytes`, `max_bytes`. |
-| `upload_too_large` | The archive is over the published ceiling, at the door or when extracted. |
-| `job_already_uploaded` | You already uploaded this archive's job; `details.existing_job_id`. Delete it to replace it. |
-| `too_many_concurrent_job_uploads` | 429, no retry delay. `details.max_concurrent`. |
-| `job_import_not_found` | 404. An import that is not yours or does not exist. |
-
-### Registered agents and skills
-
-| Code | When |
-| --- | --- |
-| `agent_not_found` | A name outside your organizations reads the same as none. |
-| `agent_name_taken`, `agent_name_reserved`, `agent_invalid_name` | The name is used, collides with a built-in, or breaks the published pattern. |
-| `agent_source_required`, `agent_source_conflict` | Neither an install script nor a directory, or both. |
-| `agent_invalid_env` | The declared env overrides a run-contract key, or looks like a credential. |
-| `agent_too_large`, `agent_limit_reached` | The tarball or the per-user count is over the published limit. |
-| `skill_not_found`, `skill_name_not_found` | An `upload:<id>` or a `name:<skill-name>` that does not resolve. |
-| `skill_ref_invalid`, `skill_unresolvable`, `skill_invalid` | A reference that does not parse, cannot be fetched, or holds no valid skill. |
-| `skill_in_use` | 409. A running job references the upload. |
-| `skill_too_large`, `skill_limit_reached` | The tarball or the per-user count is over the published limit. |
-| `too_many_concurrent_skill_uploads` | 429, no retry delay. `details.max_concurrent`. |
-
-### Organizations
-
-`org_not_found`, `org_slug_taken`, `org_forbidden`, `org_personal_immutable`, `org_last_owner`, `org_in_use`, `org_member_not_found`, `invite_not_found` and `invite_invalid` belong to team accounts. The SDK reads organizations only, so only `org_not_found` and `org_forbidden` can reach it today.
-
-`internal_error` is the server's own failure; quote `requestId`.

package/skills/evolve-evals/references/sdk-reference/index.mdx

@@ -1,60 +0,0 @@
----
-title: "Clients"
-description: "How the hosted clients are built and configured."
----
-
-One client per noun, each built by a factory. Every factory reads `EVOLVE_API_KEY` from the environment unless a config names the key. Create a key on the dashboard's [API keys page](https://dashboard.evolvingmachines.ai/api-keys).
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    import { jobs, trials, datasets, analyses, checks, skills, agents, auth, orgs, hosted, meta } from "@evolvingmachines/evolve";
-
-    interface HostedClientConfig {
-      apiKey?: string;   // default: process.env.EVOLVE_API_KEY
-      baseUrl?: string;  // default: the Evolve dashboard API
-      org?: string;      // default organization for jobs().start() and datasets().publish(); omitted: your personal one
-    }
-
-    function jobs(config?: HostedClientConfig): JobsClient
-    function hosted(config?: HostedClientConfig): HostedEvolve
-    ```
-
-    `datasets()`, `trials()`, `analyses()`, `checks()`, `skills()`, `agents()`, `auth()` and `orgs()` take the same config. `hosted()` builds every client from one config and exposes them as properties: `datasets`, `agents`, `jobs`, `skills`, `trials`, `analyses`, `checks`, `orgs`, plus `meta()`.
-
-    ```ts
-    const evolve = hosted({ apiKey: process.env.EVOLVE_API_KEY });
-    const job = await evolve.jobs.get("3e1f9a2c-…");
-    ```
-
-    Every list method returns a handle you either await for one page (`{ items, nextCursor, hasMore }`) or iterate with `for await` to walk every page.
-  </Tab>
-  <Tab title="Python">
-    ```python
-    from evolve import jobs, trials, datasets, analyses, checks, skills, agents, auth, orgs, hosted, meta
-    from evolve import HostedClientConfig
-
-    @dataclass
-    class HostedClientConfig:
-        api_key: Optional[str] = None   # default: EVOLVE_API_KEY
-        base_url: Optional[str] = None  # default: the Evolve dashboard API
-        org: Optional[str] = None       # default organization for jobs().start() and datasets().publish(); None: your personal one
-
-    def jobs(config: Optional[HostedClientConfig] = None) -> JobsClient
-    def hosted(config: Optional[HostedClientConfig] = None) -> HostedEvolve
-    ```
-
-    `datasets()`, `trials()`, `analyses()`, `checks()`, `skills()`, `agents()`, `auth()` and `orgs()` take the same config. `hosted()` builds every client from one config and exposes them as properties: `datasets`, `agents`, `jobs`, `skills`, `trials`, `analyses`, `checks`, `orgs`, plus `await meta()`.
-
-    ```python
-    evolve = hosted(HostedClientConfig(api_key=os.environ["EVOLVE_API_KEY"]))
-    job = await evolve.jobs.get("3e1f9a2c-…")
-    ```
-
-    Every method is a coroutine. Each client is an async context manager and has `close()`. Every list method returns a handle you either await for one page (`items`, `next_cursor`, `has_more`) or iterate with `async for` to walk every page.
-  </Tab>
-</Tabs>
-
-The config has no timeout setting. The clients behind `hosted()` are built on first access, so `meta()` works before a key is set. Neither client follows a redirect with your key: Python raises the 3xx, TypeScript drops the `Authorization` header on a cross-origin redirect.
-
-Every object the clients return is catalogued on [types](/sdk-reference/types), and every refusal on [errors](/sdk-reference/errors).