@evolvingmachines/evolve 0.0.65 → 0.0.67-train-harness-wave-2026-09-25.20260926.e00a8a6d

package/skills/evolve-evals/references/sdk-reference/methods/agents.md

@@ -0,0 +1,154 @@
+---
+title: "Agents: methods"
+description: "Register, read, replace, and delete custom agents."
+---
+
+Use `agents()` or `hosted().agents` for registered custom agents. Built-in harness capabilities come from [`meta()`](/sdk-reference/methods/meta). Examples use a configured `client`. Python examples run inside an async function.
+
+## create
+
+Register a custom agent from an install script or local directory.
+
+```ts TypeScript
+const agent = await client.create({
+  name: "my-agent",
+  directory: "./agent-source",
+  run_command: "my-agent --headless",
+});
+```
+
+```python Python
+agent = await client.create(
+    name="my-agent",
+    directory="./agent-source",
+    run_command="my-agent --headless",
+)
+```
+
+**Returns:** `Agent`. See [agent fields](#agent-fields).
+
+### Parameters and behavior
+
+| Parameter | Type and meaning |
+| --- | --- |
+| `name` | Required string. Read naming rules and reserved names from `meta().agent_registration`. |
+| `install_script` or `directory` | Exactly one. Script is its text, not a path. Directory is packed and uploaded. |
+| `run_command` | Required command string, run headlessly with `sh -c` in the task working directory. |
+| `env?` | String-to-string map. Injected at run time; platform-owned environment keys are refused. |
+| `org?` | Organization slug or id. Overrides client default, then falls back to your personal organization. |
+
+The build has internet access but no secrets. Its dependencies must be publicly fetchable, and installation must leave executables under `$PREFIX/bin`. Read [custom agents](/core-concepts/agents) for the run contract.
+
+## list
+
+Browse registered agents visible in the chosen scope.
+
+```ts TypeScript
+const page = await client.list({
+  scope: "org",
+  limit: 20,
+});
+```
+
+```python Python
+page = await client.list(
+    scope="org",
+    limit=20,
+)
+```
+
+**Returns:** `AgentPage` when awaited; an `Agent` per iteration.
+
+### Parameters and behavior
+
+| Parameter | Type and meaning |
+| --- | --- |
+| `scope?` | `my` (default): yours. `shared`: other members’ registrations in your organizations. `org`: all registrations in your organizations, including yours. |
+| `limit?`, `cursor?` | Collection pagination; default 50, maximum 200. Await one page or use `for await` / `async for`. |
+
+Page fields are `items`, `nextCursor`, `hasMore` in TypeScript; `items`, `next_cursor`, `has_more` in Python.
+
+## get
+
+Read a registered agent by name.
+
+```ts TypeScript
+const agent = await client.get("my-agent");
+```
+
+```python Python
+agent = await client.get("my-agent")
+```
+
+**Returns:** `Agent`.
+
+### Parameters and behavior
+
+`name` is a required string. This reads registered agents; use `meta()` to discover built-in harnesses.
+
+## upsert
+
+Create or fully replace the registration under a name.
+
+```ts TypeScript
+const agent = await client.upsert("my-agent", {
+  directory: "./agent-source",
+  run_command: "my-agent --headless",
+  env: { AGENT_MODE: "eval" },
+});
+```
+
+```python Python
+agent = await client.upsert(
+    "my-agent",
+    directory="./agent-source",
+    run_command="my-agent --headless",
+    env={"AGENT_MODE": "eval"},
+)
+```
+
+**Returns:** `Agent` after replacement.
+
+### Parameters and behavior
+
+The first argument is the required name. Remaining fields match [`create`](#create), except `name` is not repeated in the TypeScript input.
+
+This replaces the entire registration: omitting `env` clears it. The owning organization cannot change. Existing registrations can be updated without a delete-and-create gap.
+
+## delete
+
+Remove a registered agent.
+
+```ts TypeScript
+await client.delete("my-agent");
+```
+
+```python Python
+await client.delete("my-agent")
+```
+
+**Returns:** No value (`void` / `None`).
+
+### Parameters and behavior
+
+`name` is required. Past jobs retain their recorded agent identity.
+
+## Agent fields
+
+### Agent response
+
+Both SDKs use the same fields; Python returns an `Agent` dataclass. `source` identifies how it was registered. The response does not contain the install script or source archive.
+
+```ts Fields
+interface Agent {
+  name: string;
+  org: string | null;
+  source: AgentSource;
+  run_command: string;
+  env: Record<string, string>;
+  created_at: string;
+  updated_at: string;
+}
+
+type AgentSource = "install_script" | "tarball";
+```

package/skills/evolve-evals/references/sdk-reference/methods/analyses.md

@@ -0,0 +1,211 @@
+---
+title: "Analyses methods"
+description: "Read the analyzer’s result, transcript, files, and current defaults."
+---
+
+Create `client` with `analyses()`. Start a wave with [jobs.analyze](/sdk-reference/methods/jobs#analyze), then read its results here.
+
+**Note:**
+
+Python exposes `list`, `defaults`, `download`, and `filesystem`. Direct `get`, `transcript`, and `artifact` reads are TypeScript-only. In Python, `list(job=...)` or `Trial.analysis` gives the result; an analysis download gives its stored evidence.
+
+| Both SDKs | TypeScript only |
+| --- | --- |
+| [list](#list), [defaults](#defaults), [download](#download), [filesystem](#filesystem) | [get](#get), [transcript](#transcript), [artifact](#artifact) |
+
+## list
+
+List visible analyses, newest first. Returns one page or an iterable of `TrialAnalysis`.
+
+### Signature
+
+```ts TypeScript signature
+list(options?: ListAnalysesOptions): AnalysisList;
+```
+
+```python Python signature
+def list(
+    *,
+    scope: Optional[JobListScope] = None,
+    job: Optional[str] = None,
+    status: Optional[List[AnalysisStatus]] = None,
+    limit: Optional[int] = None,
+    cursor: Optional[str] = None,
+) -> _PaginatedList: ...
+```
+
+```ts TypeScript
+const page = await client.list({
+  job: jobId,
+  status: ["completed"]
+});
+```
+
+```python Python
+page = await client.list(
+    job=job_id,
+    status=['completed'],
+)
+```
+
+Optional `scope`: `my` (default), `shared`, or `org`; `job`: source job id; `status`: list of `queued`, `running`, `completed`, or `failed`. `limit` defaults to 50 (maximum 200); `cursor` continues a page.
+
+Python analysis rows are dictionaries: `row["id"]`, `row["checks"]`. [Analysis result fields](/sdk-reference/types#analysis-and-check-results) include failures and cost as well as the per-criterion checks.
+
+## defaults
+
+Read the current analysis defaults. Returns `AnalyzeDefaults`: model, effort, sandbox provider, rubric, and unrendered prompt template.
+
+### Signature
+
+```ts TypeScript signature
+defaults(): Promise<AnalyzeDefaults>;
+```
+
+```python Python signature
+async def defaults() -> AnalyzeDefaults: ...
+```
+
+```ts TypeScript
+const defaults = await client.defaults();
+```
+
+```python Python
+defaults = await client.defaults()
+```
+
+Python returns a dictionary. These are current defaults; an existing analysis records the policy it actually ran under.
+
+## get
+
+TypeScript only. Read one `TrialAnalysis` at any status, including an earlier analysis no longer attached to `Trial.analysis`.
+
+### Signature
+
+```ts TypeScript signature
+get(analysisId: string): Promise<TrialAnalysis>;
+```
+
+```ts TypeScript
+const analysis = await client.get(analysisId);
+```
+
+Takes one analysis id. Missing or inaccessible records return `analysis_not_found`.
+
+## transcript
+
+TypeScript only. Read the analyzer’s own activity, not the evaluated agent’s trace. Returns `AnalysisTranscript`.
+
+### Signature
+
+```ts TypeScript signature
+transcript(
+  analysisId: string,
+  options?: AnalysisTranscriptOptions,
+): Promise<AnalysisTranscript>;
+```
+
+```ts TypeScript
+const transcript = await client.transcript(analysisId, {
+  since: 0
+});
+```
+
+Optional `since` is the number of events already read (nonnegative integer, default 0). There is no server-side pagination. `total` counts all stored events; `events` starts at `since`. `gateway_calls` is returned separately in full on each read. See [transcript fields](/sdk-reference/types#analysis-and-check-results).
+
+## artifact
+
+TypeScript only. Read the analyzer’s stored stdout, stderr, or captured home.
+
+### Signature
+
+```ts TypeScript signature
+artifact(
+  analysisId: string,
+  stream: Exclude<AnalysisArtifactStream, "agent-home">
+): Promise<string | null>;
+artifact(
+  analysisId: string,
+  stream: "agent-home"
+): Promise<Record<string, string> | null>;
+```
+
+```ts TypeScript
+const stderr = await client.artifact(
+  analysisId,
+  "trace-stderr"
+);
+```
+
+`trace-stdout` and `trace-stderr` return `string | null`; `agent-home` returns `Record<string, string> | null`. Null means not stored. Analyses do not expose trial `verifier` or `trace-atif` artifact selectors.
+
+## download
+
+Download a settled analysis as a `.tar.gz` containing its wrapper trial, transcript, logs, and available result artifacts.
+
+### Signature
+
+```ts TypeScript signature
+download(analysisId: string): Promise<Buffer>;
+download(
+  analysisId: string,
+  options: { to: string }
+): Promise<string>;
+download(
+  analysisId: string,
+  options: { stream: true },
+): Promise<ReadableStream<Uint8Array>>;
+download(
+  analysisId: string,
+  options?: DownloadJobOptions
+): Promise<Buffer | string | ReadableStream<Uint8Array>>;
+```
+
+```python Python signature
+async def download(
+    analysis_id: str,
+    *,
+    to: Optional[str] = None,
+) -> bytes | str: ...
+```
+
+```ts TypeScript
+const path = await client.download(analysisId, {
+  to: "./results"
+});
+```
+
+```python Python
+path = await client.download(
+    analysis_id,
+    to='./results',
+)
+```
+
+Omit delivery options to return `Buffer` / `bytes`; `to` saves to a directory and returns a path. TypeScript also accepts `{ stream: true }`, returning a raw `ReadableStream<Uint8Array>` that the caller must verify. Python has no stream option. Queued or running analyses return `analysis_not_terminal`.
+
+## filesystem
+
+Get the analyzer’s `RunFilesystem`, including its live or captured files, sandbox logs, and process list.
+
+### Signature
+
+```ts TypeScript signature
+filesystem(analysisId: string): RunFilesystem;
+```
+
+```python Python signature
+def filesystem(analysis_id: str) -> 'RunFilesystem': ...
+```
+
+```ts TypeScript
+const fs = client.filesystem(analysisId);
+const status = await fs.status();
+```
+
+```python Python
+fs = client.filesystem(analysis_id)
+status = await fs.status()
+```
+
+This is the analyzer’s sandbox, separate from the trial it analyzed. See [filesystem methods](/sdk-reference/methods/filesystem).

package/skills/evolve-evals/references/sdk-reference/methods/auth.md

@@ -0,0 +1,257 @@
+---
+title: "Identity and teams: methods"
+description: "Read identity and manage organization membership."
+---
+
+Create `identity = auth()` and `teams = orgs()` for these examples. `hosted().orgs` is available; there is no `hosted().auth` property.
+
+Python examples run inside an async function.
+
+## status
+
+Identify the account and API key making the request.
+
+```ts TypeScript
+const user = await identity.status();
+console.log(user.user_id, user.email);
+```
+
+```python Python
+user = await identity.status()
+print(user.user_id, user.email)
+```
+
+**Returns:** `AuthStatus`. No arguments.
+
+### Parameters and behavior
+
+Returns key metadata only, never its secret value. `auth()` has no key-creation, listing, or revocation methods; see the [identity guide](/sdk-reference/auth).
+
+## list
+
+List organizations you belong to, with your personal organization first.
+
+```ts TypeScript
+const organizations = await teams.list();
+```
+
+```python Python
+organizations = await teams.list()
+```
+
+**Returns:** `Organization[]` / `List[Organization]`. No arguments; this is not a paginated result.
+
+## get
+
+Read an organization’s membership, limits, and current usage.
+
+```ts TypeScript
+const team = await teams.get("my-team");
+console.log(team.quota, team.usage);
+```
+
+```python Python
+team = await teams.get("my-team")
+print(team.quota, team.usage)
+```
+
+**Returns:** `OrganizationDetail`.
+
+### Parameters and behavior
+
+`org` is a required organization slug or id. Quotas are read-only through the SDK. A limit of zero pauses the corresponding enforced workload; see the quota notes below.
+
+## create
+
+Create a shared organization and become its owner.
+
+```ts TypeScript
+const team = await teams.create("my-team", {
+  displayName: "My team",
+});
+```
+
+```python Python
+team = await teams.create(
+    "my-team",
+    display_name="My team",
+)
+```
+
+**Returns:** `Organization`.
+
+### Parameters and behavior
+
+| Parameter | Type and meaning |
+| --- | --- |
+| `name` | Required slug: 1–48 lowercase letters, digits, or hyphens. No leading or trailing hyphen. |
+| `displayName?` / `display_name?` | Non-empty string, at most 120 characters after trimming. Defaults to the slug. TypeScript passes an options object; Python uses a keyword. |
+
+A taken slug returns `org_slug_taken`.
+
+## invite
+
+Create an invitation for an organization you own.
+
+```ts TypeScript
+const invite = await teams.invite("my-team");
+// Save invite.token securely; it is returned only here.
+```
+
+```python Python
+invite = await teams.invite("my-team")
+# Save invite.token securely; it is returned only here.
+```
+
+**Returns:** `OrgInviteCreated`: invitation metadata plus its one-time `token`.
+
+### Parameters and behavior
+
+`org` is a required slug or id. SDK calls use the server defaults: a 7-day expiry and unlimited uses. Neither SDK exposes invitation limits in this method. Personal organizations cannot issue invitations.
+
+## join
+
+Redeem an invitation token to join as a member.
+
+```ts TypeScript
+const result = await teams.join(inviteToken);
+console.log(result.org.slug, result.already_member);
+```
+
+```python Python
+result = await teams.join(invite_token)
+print(result.org.slug, result.already_member)
+```
+
+**Returns:** `OrgJoined`: `org: Organization` and `already_member: boolean`.
+
+### Parameters and behavior
+
+`token` is required. If you already belong to the organization, `already_member` is true and no invitation use is consumed.
+
+## members
+
+List an organization’s members, owners first.
+
+```ts TypeScript
+const members = await teams.members("my-team");
+```
+
+```python Python
+members = await teams.members("my-team")
+```
+
+**Returns:** `OrgMember[]` / `List[OrgMember]`.
+
+### Parameters and behavior
+
+`org` is a required slug or id. Any organization member can read the list; this method is not paginated.
+
+## Response fields
+
+Both languages use the field names below. Python returns dataclasses, with `None` for null. Optional TypeScript `role` is an optional Python value.
+
+### Identity and key metadata
+
+```ts Fields
+interface AuthStatus {
+  user_id: string;
+  email: string | null;
+  key: ApiKey;
+}
+
+interface ApiKey {
+  id: string;
+  label: string | null;
+  created_at: string;
+  last_used_at: string | null;
+}
+```
+
+`key.created_at` can be `null` / `None` when the credential service has no creation timestamp, despite the current SDK's string annotation. `last_used_at` is currently always null; the credential service does not record it.
+
+### Organization and members
+
+```ts Fields
+type OrgRole = "owner" | "member";
+
+interface Organization {
+  org_id: string;
+  slug: string;
+  display_name: string;
+  personal: boolean;
+  role?: OrgRole;
+  created_at: string;
+}
+
+interface OrganizationDetail extends Organization {
+  member_count: number;
+  quota: OrgQuota;
+  usage: OrgUsage;
+}
+
+interface OrgMember {
+  user_id: string;
+  email: string;
+  role: OrgRole;
+  joined_at: string;
+}
+```
+
+### Quotas and usage
+
+All quota values are effective limits after applying platform defaults. `monthly_budget_usd: null` means no monthly budget. `month_spend_usd: null` means no current meter copy is available; `month_spend_as_of` records when the copied meter was read.
+
+`max_concurrent_analyses` covers both trace analyses and task checks. Provider sandbox limits cover work sharing that provider. `max_concurrent_sessions` is reported but not yet enforced when creating managed sessions. `active_sessions` is zero for shared organizations because sessions carry no organization.
+
+```ts Fields
+interface OrgQuota {
+  max_concurrent_trials: number;
+  max_queued_trials: number;
+  max_concurrent_imports: number;
+  max_concurrent_analyses: number;
+  max_concurrent_sessions: number;
+  monthly_budget_usd: number | null;
+  max_concurrent_sandboxes_e2b: number;
+  max_concurrent_sandboxes_daytona: number;
+  max_concurrent_sandboxes_modal: number;
+}
+
+interface OrgUsage {
+  in_flight_trials: number;
+  queued_trials: number;
+  in_flight_imports: number;
+  in_flight_analyses: number;
+  active_sessions: number;
+  month_spend_usd: number | null;
+  month_spend_as_of: string | null;
+}
+```
+
+### Invitation and join result
+
+`expires_at: null` means no expiry; `max_uses: null` means unlimited uses. Tokens appear only when an invitation is created.
+
+```ts Fields
+interface OrgInvite {
+  invite_id: string;
+  expires_at: string | null;
+  max_uses: number | null;
+  uses: number;
+  revoked_at: string | null;
+  created_at: string;
+}
+
+interface OrgInviteCreated extends OrgInvite {
+  token: string;
+}
+
+interface OrgJoined {
+  org: Organization;
+  already_member: boolean;
+}
+```
+
+## Related operations
+
+SDK sharing methods belong to [jobs](/sdk-reference/methods/jobs) and [checks](/sdk-reference/methods/checks). Team rename/delete, role changes, and invitation revocation have no SDK methods; see [team HTTP operations](/sdk-reference/auth#manage-teams-over-http).