@benchsdk/client 0.2.1 → 0.4.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ComputeSDK
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -10,89 +10,69 @@ This package talks to the platform-owned benchmark/run/participant/worker API. I
 npm install @benchsdk/client
 ```
 
-## Define A Worker
+> Higher-level benchmark authoring (`defineBenchmarkConfig` / `defineTask` and
+> the local orchestrator) lives in
+> [`@benchsdk/runner`](../benchsdk-runner). This package is REST transport plus
+> the worker engine only.
+
+## Authentication
+
+`createBenchmarkClient` requires a platform API key or OAuth token. Provide
+`apiKey`/`token` directly, or set `BENCHMARKS_PLATFORM_API_KEY` or
+`BENCHMARKS_PLATFORM_TOKEN` in the environment.
+
+## Run A Worker
 
 ```ts
-import { defineStep, defineTask, defineWorker } from '@benchsdk/client';
+import { createBenchmarkClient } from '@benchsdk/client';
 import { compute } from 'computesdk';
 
-const worker = defineWorker({
+const client = createBenchmarkClient({
+  apiKey: process.env.BENCHMARKS_PLATFORM_API_KEY,
+});
+
+// `task` is a raw function; declare named steps imperatively via `step(...)`,
+// so values flow between steps with closures and cleanup runs in a `finally`.
+const { assignment, records } = await client.runWorker({
   benchmarkSlug: 'scale',
   runId: process.env.BENCHMARK_RUN_ID!,
   participantSlug: 'e2b',
   processKind: 'container',
   processKey: process.env.HOSTNAME,
   concurrency: 100,
-  task: defineTask('sandbox.lifecycle', [
-    defineStep('create', async ({ assignment, state }) => {
-      state.sandbox = await compute.sandbox.create({
-        provider: assignment.provider ?? 'e2b',
-      });
-    }),
-    defineStep('readiness', async ({ state }) => {
-      await (state.sandbox as any).runCommand('true');
-    }),
-    defineStep('exec.first-command', async ({ state }) => {
-      await (state.sandbox as any).runCommand('node -v');
-    }),
-    defineStep('pause', { readiness: 'poll' }, async () => {
-      // Every worker reports active pause concurrency and waits here until
-      // the platform reports the participant's pause step is ready.
-    }),
-    defineStep('destroy', async ({ state }) => {
-      await (state.sandbox as any).destroy();
-    }),
-  ]),
+  task: async ({ assignment, step }) => {
+    const sandbox = await step('create', () =>
+      compute.sandbox.create({ provider: assignment.provider ?? 'e2b' }),
+    );
+    try {
+      await step('readiness', () => sandbox.runCommand('true'), { readiness: 'internal' });
+      await step('exec.first-command', () => sandbox.runCommand('node -v'));
+      // A `readiness: 'poll'` step reports active concurrency and waits until
+      // the platform reports the participant's step is ready (a barrier).
+      await step('pause', () => {}, { readiness: 'poll' });
+      return { sandboxId: sandbox.id };
+    } finally {
+      await step('destroy', () => sandbox.destroy());
+    }
+  },
 });
-
-await worker.run();
 ```
 
-`worker.run()` claims the next pending platform assignment for the participant. If no work is available, it returns `{ assignment: null, records: [] }`.
+`client.runWorker(...)` claims the next pending platform assignment for the participant. If no work is available, it returns `{ assignment: null, records: [] }`.
 
 Task results are flushed to the platform in batches of 1,000 records by default. Set `batchSize` to tune this per worker; the SDK validates the platform limit of 5,000 records per batch. Workers also flush partial batches every 30 seconds by default via `flushIntervalMs`, and always flush pending records during final completion or shutdown.
 
-## Reuse A Bench Definition
-
-```ts
-import { defineBench, defineStep, defineTask } from '@benchsdk/client';
-
-const lifecycleTask = defineTask('sandbox.lifecycle', [
-  defineStep('create', async ({ state }) => {
-    state.sandboxId = 'sandbox_123';
-  }),
-  defineStep('exec.first-command', async ({ state }) => ({
-    sandboxId: String(state.sandboxId),
-  })),
-]);
-
-const bench = defineBench({
-  slug: 'scale',
-  participantSlug: 'e2b',
-  concurrency: 100,
-  task: lifecycleTask,
-});
-
-const worker = bench.defineWorker({
-  runId: process.env.BENCHMARK_RUN_ID!,
-  processKey: process.env.HOSTNAME,
-});
-
-await worker.run();
-```
-
 ## Create A Platform Run
 
 ```ts
 import { createBenchmarkClient } from '@benchsdk/client';
 
 const client = createBenchmarkClient({
-  apiKey: process.env.COMPUTESDK_ADMIN_API_KEY,
+  apiKey: process.env.BENCHMARKS_PLATFORM_API_KEY,
 });
 
 await client.upsertBenchmark('scale', {
   name: 'Scale',
-  kind: 'scale',
   config: { timeoutMs: 120_000 },
 });
 
@@ -110,55 +90,27 @@ await client.planWorkers('scale', run.id, 'modal');
 console.log(run.id);
 ```
 
-Workers must be planned before `worker.run()` can claim assignments.
+Workers must be planned before `client.runWorker(...)` can claim assignments.
 
 ## API
 
-### Definition Helpers
+### Worker Engine
 
 ```ts
-defineStep(name, fn)
-defineTask(name, steps)
-defineWorker(options)
-defineBench(options)
+client.runWorker(options)
 ```
 
-Step functions receive:
+The `task` function receives:
 
 | Field | Type | Description |
 |-------|------|-------------|
 | `assignment` | `BenchmarkAssignment` | Platform-owned assignment for this worker |
 | `taskIndex` | `number` | Deterministic task index within the benchmark run |
-| `state` | `Record<string, unknown>` | Mutable per-task state shared across steps |
-
-If a step returns a JSON object, it is merged into the task result `data` object. Defined tasks also include `taskName` in `data`.
-
-`defineTask(name, steps, options)` supports task cleanup:
+| `step` | `(name, fn, options?) => Promise<R>` | Runs `fn` as a named platform step and records its timing/outcome |
 
-| Option | Type | Description |
-|--------|------|-------------|
-| `cleanup` | `(context) => Promise<void> \| void` | Runs after the task finishes, whether steps succeeded or failed. Use shared `state` to tear down resources created by earlier steps. |
-
-```ts
-type SandboxState = {
-  sandbox?: Awaited<ReturnType<typeof compute.sandbox.create>>;
-};
-
-defineTask<SandboxState>('sandbox.lifecycle', [
-  defineStep<SandboxState>('create', async ({ state }) => {
-    state.sandbox = await compute.sandbox.create();
-  }),
-  defineStep<SandboxState>('exec', async ({ state }) => {
-    await state.sandbox.runCommand('node -v');
-  }),
-], {
-  cleanup: async ({ state }) => {
-    await state.sandbox?.destroy?.();
-  },
-});
-```
+If the task returns a JSON object, it is stored as the task result `data`.
 
-`defineStep(name, options, fn)` supports step-level progress coordination:
+`step(name, fn, options)` supports step-level progress coordination via `options`:
 
 | Option | Type | Description |
 |--------|------|-------------|
@@ -202,7 +154,7 @@ client.failWorker(benchmarkSlug, runId, workerId, attemptId, error)
 client.runWorker(options)
 ```
 
-For custom coordinators that do not fit `defineWorker`, use the best-effort reporter wrapper:
+For custom coordinators that do not fit `runWorker`, use the best-effort reporter wrapper:
 
 ```ts
 const reporter = await BenchmarkReporter.claim({
@@ -227,10 +179,10 @@ await reporter?.finish(false);
 
 `BenchmarkReporter` swallows platform telemetry failures for claim, heartbeat, result flushing, artifact upload, and finish calls. Benchmark work can continue even when reporting is temporarily unavailable.
 
-For `defineWorker` / `runWorker`, use `onFinish` to upload worker-level logs once, after final task results are flushed and before the worker attempt is completed or failed:
+For `runWorker`, use `onFinish` to upload worker-level logs once, after final task results are flushed and before the worker attempt is completed or failed:
 
 ```ts
-defineWorker({
+client.runWorker({
   benchmarkSlug: 'scale',
   runId,
   participantSlug: 'e2b',
@@ -250,7 +202,7 @@ For coordinator health artifacts, sample system metrics:
 
 ```ts
 const metrics = createSystemMetricsCollector();
-const samples = [metrics.sample()];
+const samples = [await metrics.sample()];
 metrics.stop();
 ```
 
@@ -269,7 +221,7 @@ console.log(participant?.tasks.completionRatio);
 console.log(participant?.concurrency.find((item) => item.step === 'pause')?.ready);
 ```
 
-Most workers should use `defineWorker(...).run()`.
+Most workers should use `client.runWorker(...)`.
 
 ## Task Result Shape
 
@@ -286,7 +238,6 @@ Most workers should use `defineWorker(...).run()`.
     { "name": "destroy", "status": "success", "startedAt": "...", "completedAt": "...", "latencyMs": 180 }
   ],
   "data": {
-    "taskName": "sandbox.lifecycle",
     "sandboxId": "..."
   }
 }