@evolvingmachines/evolve 0.0.65 → 0.0.66

package/skills/evolve-evals/references/sdk-reference/agents.mdx

@@ -1,108 +0,0 @@
----
-title: "agents"
-description: "Register and manage your own agents."
----
-
-`agents()` returns the registered-agents client. A registered agent's name works in a job's `agents[].name` like a built-in.
-
-## create
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    create(input: { name: string; org?: string; run_command: string; env?: Record<string, string> } & ({ install_script: string } | { directory: string })): Promise<Agent>
-    ```
-
-    Register an agent from an install script's text, or from a local directory. `run_command` runs with `sh -c` at the task's working directory; what it can rely on, and the registration limit, are on [agents](/core-concepts/agents#register-your-own-agent). The registration belongs to you and to an organization (`org`, a slug or id; omitted, the client's `org` default, else your personal one): its members may name the agent in their own jobs, and only you can change or delete it. A name stays unique per person, so two teammates may each register their own `acme-cli`.
-
-    ```ts
-    const agent = await agents().create({
-      name: "acme-cli",
-      install_script: "curl -fsSL https://acme.dev/install.sh | sh",
-      run_command: "acme-cli --headless",
-    });
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    async def create(
-        *,
-        name: str,
-        install_script: Optional[str] = None,
-        directory: Optional[str] = None,
-        run_command: str,
-        env: Optional[Dict[str, str]] = None,
-        org: Optional[str] = None,
-    ) -> Agent
-    ```
-
-    Register an agent from an install script's text, or from a local directory. `run_command` runs with `sh -c` at the task's working directory; what it can rely on, and the registration limit, are on [agents](/core-concepts/agents#register-your-own-agent). The registration belongs to you and to an organization (`org`, a slug or id; omitted, the client's `org` default, else your personal one): its members may name the agent in their own jobs, and only you can change or delete it. A name stays unique per person, so two teammates may each register their own `acme-cli`.
-
-    ```python
-    agent = await agents().create(
-        name="acme-cli",
-        install_script="curl -fsSL https://acme.dev/install.sh | sh",
-        run_command="acme-cli --headless",
-    )
-    ```
-  </Tab>
-</Tabs>
-
-## list, get, upsert, delete
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    list(options?: { scope?: "my" | "shared" | "org"; limit?: number; cursor?: string }): AgentList
-    get(name: string): Promise<Agent>
-    upsert(
-      name: string,
-      input: { run_command: string; env?: Record<string, string> } & ({ install_script: string } | { directory: string }),
-    ): Promise<Agent>
-    delete(name: string): Promise<void>
-    ```
-
-    Registered agents, one agent, replace one without a gap, or delete one. `scope` is the lists' own vocabulary: `"my"` (yours, the default), `"shared"` (your organizations' other members') or `"org"` (every registration in your organizations, yours included); `get` takes a name, which is yours by construction. Past jobs keep the agent they recorded. `upsert` creates when the name is free and replaces when it is not, consuming no new slot; it is a full replacement, so an omitted `env` becomes empty.
-
-    ```ts
-    await agents().upsert(
-      "acme-cli",
-      {
-        install_script: "curl -fsSL https://acme.dev/install.sh | sh",
-        run_command: "acme-cli --headless --v2",
-      },
-    );
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    def list(
-        *,
-        scope: Optional[Literal['my', 'shared', 'org']] = None,
-        limit: Optional[int] = None,
-        cursor: Optional[str] = None,
-    )
-    async def get(name: str) -> Agent
-    async def upsert(
-        name: str,
-        *,
-        run_command: str,
-        install_script: Optional[str] = None,
-        directory: Optional[str] = None,
-        env: Optional[Dict[str, str]] = None,
-        org: Optional[str] = None,
-    ) -> Agent
-    async def delete(name: str) -> None
-    ```
-
-    Registered agents, one agent, replace one without a gap, or delete one. `scope` is the lists' own vocabulary: `"my"` (yours, the default), `"shared"` (your organizations' other members') or `"org"` (every registration in your organizations, yours included); `get` takes a name, which is yours by construction. Past jobs keep the agent they recorded. `upsert` creates when the name is free and replaces when it is not, consuming no new slot; it is a full replacement, so an omitted `env` becomes empty.
-
-    ```python
-    await agents().upsert(
-        "acme-cli",
-        install_script="curl -fsSL https://acme.dev/install.sh | sh",
-        run_command="acme-cli --headless --v2",
-    )
-    ```
-  </Tab>
-</Tabs>

package/skills/evolve-evals/references/sdk-reference/analyses.mdx

@@ -1,222 +0,0 @@
----
-title: "analyses"
-description: "Read trace-analysis runs."
----
-
-`analyses()` returns the analyses client. Analyses are started from a job, with `jobs().analyze()`; this client reads them back.
-
-## list
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    list(options?: { scope?: "my" | "shared" | "org"; job?: string; status?: AnalysisStatus[]; limit?: number; cursor?: string }): AnalysisList
-    ```
-
-    Analysis runs, newest first, each naming the trial, job and task it judged.
-
-    ```ts
-    for await (const a of analyses().list({
-      job: job.id,
-      status: ["failed"],
-    })) {
-      console.log(a.id, a.label);
-    }
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    def list(
-        *,
-        scope: Optional[JobListScope] = None,
-        job: Optional[str] = None,
-        status: Optional[List[AnalysisStatus]] = None,
-        limit: Optional[int] = None,
-        cursor: Optional[str] = None,
-    )
-    ```
-
-    Analysis runs, newest first, each naming the trial, job and task it judged.
-
-    ```python
-    async for a in analyses().list(
-        job=job.id,
-        status=["failed"],
-    ):
-        print(a["id"], a["label"])
-    ```
-  </Tab>
-</Tabs>
-
-## get, transcript, artifact
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    get(analysisId: string): Promise<TrialAnalysis>
-    transcript(
-      analysisId: string,
-      options?: { since?: number },
-    ): Promise<AnalysisTranscript>
-    artifact(
-      analysisId: string,
-      stream: "trace-stdout" | "trace-stderr",
-    ): Promise<string | null>
-    artifact(
-      analysisId: string,
-      stream: "agent-home",
-    ): Promise<Record<string, string> | null>
-    ```
-
-    The verdict document for every analysis, failed ones included, the analyzer's own transcript (everything after `since` in one read), or one stored artifact; an analysis has no verifier log and no trajectory. A run opens to its job's creator and every member of the organization; any other id answers `analysis_not_found` (`trial_not_found` from `transcript`).
-
-    ```ts
-    const verdict = await analyses().get("a0a1b2c3-…");
-    ```
-  </Tab>
-  <Tab title="Python">
-    The Python client has `list` and `download` only. Read a verdict from the analyzed trial: `(await trials().get(trial_id)).analysis`.
-  </Tab>
-</Tabs>
-
-## defaults
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    defaults(): Promise<AnalyzeDefaults>
-    ```
-
-    The defaults an analysis runs under when you name none: `model_name`, `reasoning_effort`, `sandbox_provider`, the platform's `rubric`, and `prompt`, the built-in analyze prompt as a template with `{trial_path}`, `{task_section}` and `{criteria_guidance}` unrendered. Edit the rubric or the prompt from here and pass it to `jobs().analyze()`; passed back unchanged, they run exactly as the defaults do, and the analysis records them as named, so a later change of the platform default does not move it.
-
-    ```ts
-    const { rubric } = await analyses().defaults();
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    async def defaults() -> AnalyzeDefaults
-    ```
-
-    The defaults an analysis runs under when you name none: `model_name`, `reasoning_effort`, `sandbox_provider`, the platform's `rubric`, and `prompt`, the built-in analyze prompt as a template with `{trial_path}`, `{task_section}` and `{criteria_guidance}` unrendered. Edit the rubric or the prompt from here and pass it to `jobs().analyze()`; passed back unchanged, they run exactly as the defaults do, and the analysis records them as named, so a later change of the platform default does not move it.
-
-    ```python
-    rubric = (await analyses().defaults())["rubric"]
-    ```
-  </Tab>
-</Tabs>
-
-## download
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    download(analysisId: string): Promise<Buffer>
-    download(
-      analysisId: string,
-      options: { to: string },
-    ): Promise<string>
-    download(
-      analysisId: string,
-      options: { stream: true },
-    ): Promise<ReadableStream<Uint8Array>>
-    ```
-
-    The run as one `.tar.gz`, `analyze-<analyzed trial>__<7 chars>/`: a trial tree of the analyzer's own run, with `artifacts/analysis.json` on a completed run. The bytes, the saved file's path, or a stream. `analysis_not_terminal` while the run is queued or running.
-
-    ```ts
-    const path = await analyses().download(
-      "a0a1b2c3-…",
-      { to: "./analyses" },
-    );
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    async def download(
-        analysis_id: str,
-        *,
-        to: Optional[str] = None,
-    )
-    ```
-
-    The run as one `.tar.gz`, `analyze-<analyzed trial>__<7 chars>/`: the bytes, or with `to` the saved file's path. `analysis_not_found` for an id you cannot read, `analysis_not_terminal` while the run is queued or running.
-
-    ```python
-    path = await analyses().download(
-        "a0a1b2c3-…",
-        to="./analyses",
-    )
-    ```
-  </Tab>
-</Tabs>
-
-## filesystem(analysisId: string)
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    filesystem(analysisId: string): RunFilesystem
-
-    interface RunFilesystem {
-      status(): Promise<FilesystemStatus>
-      list(options?: { path?: string; source?: "live" | "capture"; cursor?: string; limit?: number }): Promise<FilesystemListing>
-      read(path: string, options?: { source?: "live" | "capture"; range?: TrialFileRange }): Promise<Buffer>
-      search(options: { q: string; path?: string; regex?: boolean; limit?: number; source?: "live" | "capture" }): Promise<FilesystemSearchResult>
-      changes(options?: { source?: "live" | "capture"; phase?: "setup" | "agent" | "verifier" | "all"; cursor?: string; limit?: number }): Promise<FilesystemChanges>
-      archive(options?: { path?: string; source?: "live" | "capture" }): Promise<Buffer>
-      archive(options: { path?: string; source?: "live" | "capture"; to: string }): Promise<string>
-      watch(paths: string[]): Promise<FilesystemWatchResult>
-      events(options?: { lastEventId?: string; signal?: AbortSignal }): AsyncIterableIterator<FilesystemStreamEvent>
-      logs(options: { stream: SandboxLogStream; cursor?: string; limit?: number }): Promise<SandboxLogLines>
-      logEvents(options?: { lastEventId?: string; signal?: AbortSignal }): AsyncIterableIterator<SandboxLogEvent>
-      procs(): Promise<SandboxProcs>
-    }
-    ```
-
-    The analysis run's file system, sandbox logs and process list. While the box runs, every read sees it as it changes; after the run, the same reads answer from the kept tree. `status()` says which (`state`: `live`, `captured`, `capturing`, `none`), and `source` on a read forces one — a source that is not there is refused with `filesystem_state`.
-
-    `list` pages one folder by name; each entry says whether the run created or modified it and in which phase, and on the kept tree `captured: false` marks a file the run never touched (it lists but does not open — `not_captured`). `read` answers raw bytes, a slice with `range`. `search` finds text under a folder, or over the whole box when `path` is `/` (seconds); `truncated` says there was more. `changes` is the flat list of what the run created, modified or removed. `archive` is a `.tar.gz` of one subtree, in memory or saved under `to`; while the box runs, a subtree too large to read out in time is refused (`feature_unsupported`).
-
-    `events` yields change frames while the box lives (`state`, then one `fs` per change with its `seq`, `path` and `type`); on a provider without a native watcher, `watch(paths)` declares the folders you have open so their changes are reported (at most 8). `logs` pages one stream — `agent`, `verifier`, or `system` when the job asked for it; `setup` and `metrics` are named but not recorded today, so their page is empty with a `reason` — and `logEvents` follows every stream while the box lives. `procs` is the live process list.
-
-    ```ts
-    const fs = analyses().filesystem(analysis.id);
-    const { state } = await fs.status();
-    for (const entry of (await fs.list({ path: "/app/work" })).entries) console.log(entry.name, entry.changed);
-    const main = (await fs.read("/app/work/main.py")).toString("utf8");
-    for await (const frame of fs.events()) if (frame.event === "fs") console.log(frame.data.path, frame.data.type);
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    def filesystem(self, analysis_id: str) -> RunFilesystem
-
-    class RunFilesystem:
-        async def status(self) -> FilesystemStatus
-        async def list(self, *, path="/", source=None, cursor=None, limit=None) -> FilesystemListing
-        async def read(self, path, *, source=None, start=None, end=None, suffix=None) -> bytes
-        async def search(self, q, *, path="/", regex=False, limit=None, source=None) -> FilesystemSearchResult
-        async def changes(self, *, source=None, phase=None, cursor=None, limit=None) -> FilesystemChanges
-        async def archive(self, *, path="/", source=None, to=None)  # bytes, or the saved path with to=
-        async def watch(self, paths) -> FilesystemWatchResult
-        def events(self, *, last_event_id=None) -> AsyncIterator[FilesystemStreamEvent]
-        async def logs(self, stream, *, cursor=None, limit=None) -> SandboxLogLines
-        def log_events(self, *, last_event_id=None) -> AsyncIterator[SandboxLogEvent]
-        async def procs(self) -> SandboxProcs
-    ```
-
-    The same surface, the same words: `status().state` says whether the reads answer from the running box (`live`) or the kept tree (`captured`); `source=` forces one. `read` answers bytes, a slice with `start`/`end` or `suffix`. `events` and `log_events` are async iterators over the live streams.
-
-    ```python
-    fs = analyses().filesystem(analysis.id)
-    print((await fs.status()).state)
-    for entry in (await fs.list(path="/app/work")).entries:
-        print(entry.name, entry.changed)
-    main = (await fs.read("/app/work/main.py")).decode()
-    async for frame in fs.events():
-        if frame.event == "fs":
-            print(frame.data["path"], frame.data["type"])
-    ```
-  </Tab>
-</Tabs>

package/skills/evolve-evals/references/sdk-reference/auth.mdx

@@ -1,116 +0,0 @@
----
-title: "auth and orgs"
-description: "Who you are, and your organizations."
----
-
-`auth()` returns the identity client, `orgs()` the organizations client. Every job, dataset and analysis belongs to an organization: your personal one unless you name another. Name it per call (`org` on `jobs().start()` and `datasets().publish()`) or once for a client (`HostedClientConfig.org`); the call's own value wins.
-
-## auth.status
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    status(): Promise<AuthStatus>
-    ```
-
-    The caller and the API key in use: `user_id`, `email`, and `key` with `id`, `label`, `created_at` and `last_used_at`. The key's secret is never returned, and `last_used_at` is always null today.
-
-    ```ts
-    const me = await auth().status();
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    async def status() -> AuthStatus
-    ```
-
-    The caller and the API key in use. The key's secret is never returned.
-
-    ```python
-    me = await auth().status()
-    ```
-  </Tab>
-</Tabs>
-
-## orgs.list, orgs.get
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    list(): Promise<Organization[]>
-    get(org: string): Promise<OrganizationDetail>
-    ```
-
-    The organizations you belong to, or one organization with your role, its members, its quota and live usage.
-
-    ```ts
-    const org = await orgs().get("acme");
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    async def list() -> List[Organization]
-    async def get(org: str) -> OrganizationDetail
-    ```
-
-    The organizations you belong to, or one organization with your role, its members, its quota and live usage.
-
-    ```python
-    org = await orgs().get("acme")
-    ```
-  </Tab>
-</Tabs>
-
-## orgs.create, orgs.invite, orgs.join, orgs.members
-
-<Tabs>
-  <Tab title="TypeScript">
-    ```ts
-    create(name: string, options?: { displayName?: string }): Promise<Organization>
-    invite(org: string): Promise<OrgInviteCreated>
-    join(token: string): Promise<OrgJoined>
-    members(org: string): Promise<OrgMember[]>
-    ```
-
-    Create an organization (you become its owner; the name is its slug), mint an invite token for one you own (the token is in the response once), join the organization a token names (`already_member` says whether you were in it already), or list an organization's members with their `email` and `role`.
-
-    ```ts
-    const org = await orgs().create("acme", { displayName: "Acme Corp" });
-    const invite = await orgs().invite("acme");
-    // send invite.token to a teammate, who runs:
-    const joined = await orgs().join(invite.token);
-    const members = await orgs().members("acme");
-    ```
-  </Tab>
-  <Tab title="Python">
-    ```python
-    async def create(name: str, *, display_name: str | None = None) -> Organization
-    async def invite(org: str) -> OrgInviteCreated
-    async def join(token: str) -> OrgJoined
-    async def members(org: str) -> List[OrgMember]
-    ```
-
-    The same four verbs: create an organization, mint an invite token for one you own, join with a token, list members.
-
-    ```python
-    org = await orgs().create("acme", display_name="Acme Corp")
-    invite = await orgs().invite("acme")
-    joined = await orgs().join(invite.token)
-    members = await orgs().members("acme")
-    ```
-  </Tab>
-</Tabs>
-
-## Quota and usage
-
-`quota` is nine ceilings, each the value the platform administrator set or the fleet default; only the administrator's dashboard sets them, never a key or the SDK.
-
-- `max_queued_trials`: the one that refuses. A job whose trials would not fit is refused with `quota_exceeded`.
-- `max_concurrent_trials`, `max_concurrent_imports`, `max_concurrent_analyses` (trace analyses and task checks under one count): work beyond them waits.
-- `monthly_budget_usd`: model spend allowed per UTC calendar month; a trial it refuses settles `BUDGET` with `team:` in its message. Null means no monthly budget.
-- `max_concurrent_sandboxes_e2b`, `_daytona`, `_modal`: the organization's sandboxes in flight per provider, trials, analyses, regrade verifiers and managed sessions together; work beyond one waits, and an organization with no value of its own reads the platform's fleet ceiling.
-- `max_concurrent_sessions`: managed-agent sessions open at once; recorded, not yet enforced.
-
-A `0` pauses that kind of work: creates are refused at `max_queued_trials` 0, work waits at 0 on the concurrency ceilings, and `monthly_budget_usd` 0 pauses all model spend for the month.
-
-`usage` is the live load: `in_flight_trials`, `queued_trials`, `in_flight_imports`, `in_flight_analyses`, `active_sessions` (the organization's managed-agent sessions not yet ended), and `month_spend_usd` with `month_spend_as_of`, the month-to-date model spend as of that time (UTC calendar month, reset on the 1st); it may lag by a few minutes. It is null, never 0, when no reading is held; `evolve auth org show` prints `unavailable` then.