pareta 0.1.0 → 1.0.0

package/README.md

@@ -1,13 +1,13 @@
 # pareta
 
 [![npm](https://img.shields.io/npm/v/pareta)](https://www.npmjs.com/package/pareta)
-[![types](https://img.shields.io/npm/types/pareta)](https://www.npmjs.com/package/pareta)
-[![license](https://img.shields.io/npm/l/pareta)](https://github.com/Pareta-AI/pareta-js/blob/main/LICENSE)
+[![License](https://img.shields.io/npm/l/pareta)](https://github.com/Pareta-AI/pareta-js/blob/main/LICENSE)
 
-TypeScript/JavaScript client for [Pareta](https://pareta.ai) — deploy
-open-weights endpoints, run metered inference, browse the benchmark catalog, and
-evaluate models on your own data. The mirror of the Python [`pareta`](https://pypi.org/project/pareta/)
-package, re-expressed as one Promise-only client.
+TypeScript/JavaScript client for [Pareta](https://pareta.ai). One model id —
+`"auto"` — and Pareta plans each request, routes it to benchmark-proven open
+specialists, verifies the result, and falls back to a frontier model when
+that's the right call. One request, one bill; you never pay for Pareta's
+orchestration or cold starts.
 
 ```bash
 npm install pareta        # or: pnpm add pareta / yarn add pareta / bun add pareta
@@ -16,93 +16,82 @@ npm install pareta        # or: pnpm add pareta / yarn add pareta / bun add pare
 ```ts
 import { Pareta } from "pareta";
 
-const pa = Pareta.fromEnv(); // reads PARETA_API_KEY (+ optional PARETA_BASE_URL)
+const pa = Pareta.fromEnv();                 // reads PARETA_API_KEY
+// or: new Pareta({ apiKey: "pareta_sk_…", baseURL: "https://api.pareta.ai" })
 
 const res = await pa.chat.completions.create({
-  model: "ep_…", // an endpoint id from endpoints.deploy(...)
-  messages: [{ role: "user", content: "Extract the totals from this invoice." }],
+  model: "auto",                             // the routing brain — the product
+  messages: [{ role: "user", content: "Extract the total from this invoice: …" }],
 });
 console.log(res.choices[0].message.content);
-```
 
-Inference is OpenAI-compatible, so you can equally point the `openai` SDK at
-`baseURL` + your `pareta_sk_` key. The SDK's unique value is the control plane —
-**deploy**, **eval**, and **discovery**.
+// Streaming (progress while Pareta plans + executes, then tokens)
+for await (const chunk of await pa.chat.completions.create({
+  model: "auto", messages: [...], stream: true,
+})) {
+  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
+}
+```
 
-## Authenticate
+## Is it actually good? Measure it on YOUR data
 
-Every request uses a `pareta_sk_` secret key (mint one in the
-[dashboard](https://pareta.ai) — key management is browser-only). Put it in the
-environment and use `fromEnv()`, or pass it explicitly:
+Don't take the routing brain on faith — benchmark it. `pa.evals` runs `"auto"`
+head-to-head against frontier models on your own ground truth and prices every
+contender honestly:
 
 ```ts
-const pa = new Pareta({ apiKey: "pareta_sk_…" });
+const set = await pa.evals.sets.create({ task: "invoice-extraction", items: [...] });
+const run = await pa.evals.runs.create({
+  evalSet: set.id, models: ["auto"], frontier: ["claude-opus-4-7"], wait: true,
+});
 ```
 
-## Deploy → infer
+And watch what your live traffic is doing — spend, success rate, and the
+projected savings vs calling a frontier directly:
 
 ```ts
-// Pareta picks the GPU/serving class; model defaults to the task's best pick.
-const ep = await pa.endpoints.deploy({ task: "invoice-extraction", wait: true });
-
-const res = await pa.chat.completions.create({
-  model: ep.id!,
+await pa.auto.metrics();                     // requests, success, spend, savings
+await pa.auto.compareFrontier({              // one prompt, metered, side-by-side
+  model: "gpt-5.5",
   messages: [{ role: "user", content: "…" }],
 });
 ```
 
-Stream tokens:
-
-```ts
-for await (const chunk of pa.chat.completions.create({ model: ep.id!, messages, stream: true })) {
-  process.stdout.write(chunk.choices[0]?.delta.content ?? "");
-}
-```
+## Inference is OpenAI-compatible
 
-## Discover the best model for a task
+You don't even need this SDK to call Pareta — point the `openai` package at
+`baseURL` + your key and set `model: "auto"`:
 
 ```ts
-const lb = await pa.tasks.leaderboard("invoice-extraction");
-console.log(lb.recommended, lb.frontier?.name); // open pick + frontier baseline
+import OpenAI from "openai";
+const client = new OpenAI({ apiKey: "pareta_sk_…", baseURL: "https://api.pareta.ai/v1" });
+const res = await client.chat.completions.create({ model: "auto", messages: [...] });
 ```
 
-## Evaluate on your own data
+This SDK's unique value is everything AROUND that call — evals on your data,
+auto metrics, and the benchmark catalog.
 
-```ts
-const run = await pa.evals.runs.create({
-  task: "intent-classification",
-  items: [{ input: { text: "cancel my plan" }, expected: "cancellation" }],
-  models: ["qwen-1"],
-  frontier: "benchmarked", // also race the benchmarked frontier models
-  wait: true,
-});
-console.log(run.cost, run.results.map((r) => [r.modelId, r.qualityMean]));
-```
+Discovery (`pa.tasks.match`) tells you whether Pareta has a benchmark-proven
+specialist lane for your workload — describe the task in free text and get
+ranked matching tasks back.
 
-Eval runs are metered against your org balance; `run.cost` is the billed total
-in dollars (floored to cents).
+## Auth
 
-## Errors
-
-Status codes map to typed errors you can branch on:
+Mint a `pareta_sk_` key in the dashboard (key management is browser-only) and
+pass it as `apiKey` or via `PARETA_API_KEY`. The SDK only ever *consumes* a
+key; it never creates, lists, or revokes them.
 
-```ts
-import { InsufficientCreditsError, EndpointNotReadyError } from "pareta";
-
-try {
-  await pa.chat.completions.create({ model, messages });
-} catch (e) {
-  if (e instanceof InsufficientCreditsError) { /* top up in the dashboard */ }
-  if (e instanceof EndpointNotReadyError) { /* endpoint is cold/stopped */ }
-}
-```
-
-## Runtime
+## Errors
 
-Node 18+, modern browsers, and edge runtimes (Workers / Vercel Edge) — built on
-native `fetch` / `ReadableStream` / `FormData`, **zero runtime dependencies**.
-Ships ESM + CommonJS + `.d.ts`.
+All errors subclass `ParetaError` — `AuthenticationError` (401),
+`InsufficientCreditsError` (402), `NotFoundError` (404),
+`EndpointNotReadyError` (503), `RateLimitError` (429, auto-retried),
+`BadRequestError` (400/422), and `APIConnectionError` / `APITimeoutError`
+(transport, auto-retried). Idempotent GETs and 429/5xx/timeouts are retried
+with exponential backoff (`maxRetries`, default 2).
 
-## Docs
+## Python
 
-Full guide, reference, and examples: **[docs.pareta.ai](https://docs.pareta.ai)**.
+The same surface ships as a Python package: [`pip install pareta`](https://pypi.org/project/pareta/),
+plus a CLI (`pareta[cli]`) and an MCP server (`pareta[mcp]`). Docs for both:
+[docs.pareta.ai](https://docs.pareta.ai).

package/dist/index.cjs

@@ -174,7 +174,7 @@ function parseSSE(reader, opts) {
 }
 
 // src/version.ts
-var VERSION = "0.1.0";
+var VERSION = "1.0.0";
 
 // src/money.ts
 var MICRO_PER_CENT = 10000n;
@@ -280,33 +280,6 @@ var ModelList = class extends BaseModel {
     return this.data[Symbol.iterator]();
   }
 };
-var Endpoint = class extends BaseModel {
-  get id() {
-    return this.raw.id ?? this.raw.name ?? null;
-  }
-  get name() {
-    return this.raw.name ?? null;
-  }
-  get model() {
-    return this.raw.model ?? null;
-  }
-  get status() {
-    return this.raw.status ?? null;
-  }
-  // Live list sends camelCase `taskName`; the detail record sends `task`.
-  get task() {
-    return this.raw.taskName ?? this.raw.task ?? null;
-  }
-  get url() {
-    return this.raw.url ?? null;
-  }
-  get isLive() {
-    return this.raw.status === "live";
-  }
-};
-function endpointList(raw) {
-  return (raw ?? []).map((e) => new Endpoint(e));
-}
 var Task = class extends BaseModel {
   get id() {
     return this.raw.id ?? null;
@@ -375,6 +348,20 @@ function evalSetFromCreate(raw) {
 function evalSetList(raw) {
   return (raw?.eval_sets ?? []).map((e) => new EvalSet(e));
 }
+var EvalItemResult = class extends BaseModel {
+  get idx() {
+    return this.raw.idx ?? null;
+  }
+  get score() {
+    return this.raw.score ?? null;
+  }
+  get prediction() {
+    return this.raw.prediction ?? null;
+  }
+  get error() {
+    return this.raw.error ?? null;
+  }
+};
 var EvalResult = class extends BaseModel {
   get modelId() {
     return this.raw.model_id ?? null;
@@ -401,46 +388,9 @@ var EvalResult = class extends BaseModel {
   get errorCount() {
     return this.raw.error_count ?? null;
   }
-};
-var LeaderboardEntry = class extends BaseModel {
-  get name() {
-    return this.raw.name ?? null;
-  }
-  get kind() {
-    return this.raw.kind ?? null;
-  }
-  get quality() {
-    return this.raw.quality ?? null;
-  }
-  // Raw integer — sub-cent unit rate, NOT floored (see money.ts).
-  get costPerRequestMicroUsd() {
-    return this.raw.cost_per_request_micro_usd ?? null;
-  }
-  get contextK() {
-    return this.raw.context_k ?? null;
-  }
-  get runMode() {
-    return this.raw.run_mode ?? null;
-  }
-};
-var Leaderboard = class extends BaseModel {
-  get taskId() {
-    return this.raw.task_id ?? null;
-  }
-  get metric() {
-    return this.raw.metric ?? null;
-  }
-  get costUnit() {
-    return this.raw.cost_unit ?? null;
-  }
-  get recommended() {
-    return this.raw.recommended ?? null;
-  }
-  get models() {
-    return (this.raw.models ?? []).map((m) => new LeaderboardEntry(m));
-  }
-  get frontier() {
-    return this.raw.frontier ? new LeaderboardEntry(this.raw.frontier) : null;
+  /** Per-item rows (idx/score/prediction/error); empty when not persisted. */
+  get perItem() {
+    return (this.raw.per_item ?? []).map((it) => new EvalItemResult(it));
   }
 };
 var FrontierModel = class extends BaseModel {
@@ -453,14 +403,11 @@ var FrontierModel = class extends BaseModel {
   get vision() {
     return Boolean(this.raw.vision);
   }
-  /** Only meaningful when a task was given: it's on that task's leaderboard. */
+  /** Only meaningful when a task was given: it has benchmark results on that task. */
   get benchmarked() {
     return Boolean(this.raw.benchmarked);
   }
 };
-function leaderboardFrom(raw) {
-  return new Leaderboard(raw ?? {});
-}
 function frontierModels(raw) {
   return (raw?.frontier_models ?? []).map((m) => new FrontierModel(m));
 }
@@ -503,7 +450,7 @@ var EvalRun = class extends BaseModel {
 var PATH = "/v1/chat/completions";
 function buildBody(params) {
   const { model, messages, stream, ...extra } = params;
-  if (!model) throw new ParetaError("model is required (an endpoint id from endpoints.deploy)");
+  if (!model) throw new ParetaError('model is required (use "auto")');
   if (!messages || messages.length === 0) {
     throw new ParetaError("messages is required and must be non-empty");
   }
@@ -551,99 +498,19 @@ var Models = class {
   }
 };
 
-// src/resources/endpoints.ts
-var BASE = "/v1/endpoints";
-function buildDeployBody(params) {
-  const { task, model, name, wait: _wait, ...extra } = params;
-  if (!task) throw new ParetaError("task is required");
-  const body = { task, model: model || "recommended", ...extra };
-  if (name !== void 0) body.name = name;
-  return body;
-}
-function endpointFromComplete(data) {
-  const ep = data && typeof data === "object" ? data.endpoint : null;
-  return new Endpoint(ep ?? {});
-}
-function deployError(data) {
-  const msg = data && typeof data === "object" ? data.message : String(data);
-  return new ParetaError(`deploy failed: ${msg || "unknown error"}`);
-}
-var EndpointMetrics = class {
-  constructor(client, id) {
-    this.client = client;
-    this.id = id;
-  }
-  client;
-  id;
-  performance(params) {
-    return this.client.request("GET", `${BASE}/${this.id}/performance`, { params });
-  }
-  uptime(params) {
-    return this.client.request("GET", `${BASE}/${this.id}/uptime`, { params });
-  }
-  cost(params) {
-    return this.client.request("GET", `${BASE}/${this.id}/cost`, { params });
-  }
-  quality(params) {
-    return this.client.request("GET", `${BASE}/${this.id}/quality`, { params });
-  }
-  activity(params) {
-    return this.client.request("GET", `${BASE}/${this.id}/activity`, { params });
-  }
-};
-var Endpoints = class {
-  constructor(client) {
-    this.client = client;
-  }
-  client;
-  deploy(params) {
-    const body = buildDeployBody(params);
-    const stream = this.client.stream("POST", BASE, { body, events: true });
-    if (!params.wait) return stream;
-    return this.waitForDeploy(stream);
-  }
-  async waitForDeploy(stream) {
-    for await (const ev of stream) {
-      if (ev.event === "complete") return endpointFromComplete(ev.data);
-      if (ev.event === "error") throw deployError(ev.data);
-    }
-    throw new ParetaError("deploy stream ended without a 'complete' event");
-  }
-  list() {
-    return this.client.request("GET", BASE, { cast: endpointList });
-  }
-  retrieve(endpointId) {
-    return this.client.request("GET", `${BASE}/${endpointId}`, {
-      cast: (raw) => new Endpoint(raw)
-    });
-  }
-  start(endpointId) {
-    return this.client.request("POST", `${BASE}/${endpointId}/start`);
-  }
-  stop(endpointId) {
-    return this.client.request("POST", `${BASE}/${endpointId}/stop`);
-  }
-  async delete(endpointId) {
-    await this.client.request("DELETE", `${BASE}/${endpointId}`);
-  }
-  metrics(endpointId) {
-    return new EndpointMetrics(this.client, endpointId);
-  }
-};
-
 // src/resources/tasks.ts
-var BASE2 = "/v1/tasks";
+var BASE = "/v1/tasks";
 var Tasks = class {
   constructor(client) {
     this.client = client;
   }
   client;
   list() {
-    return this.client.request("GET", BASE2, { cast: taskList });
+    return this.client.request("GET", BASE, { cast: taskList });
   }
   retrieve(taskId, opts = {}) {
     const params = opts.examplesN !== void 0 ? { examples_n: opts.examplesN } : void 0;
-    return this.client.request("GET", `${BASE2}/${taskId}`, {
+    return this.client.request("GET", `${BASE}/${taskId}`, {
       params,
       cast: (raw) => new Task(raw)
     });
@@ -651,25 +518,15 @@ var Tasks = class {
   /** Free-text intent → ranked candidate tasks (the Step-0 backend matcher). */
   match(query, opts = {}) {
     if (!query || !query.trim()) throw new ParetaError("query is required");
-    return this.client.request("POST", `${BASE2}/match`, {
+    return this.client.request("POST", `${BASE}/match`, {
       body: { query, top_k: opts.topK ?? 5 },
       cast: (raw) => new TaskMatch(raw)
     });
   }
-  /** Models ranked by quality/cost for a task (+ recommended + frontier baseline). */
-  leaderboard(taskId) {
-    return this.client.request("GET", `${BASE2}/${taskId}/leaderboard`, {
-      cast: leaderboardFrom
-    });
-  }
-  /** The task's recommended deployable model — what deploy(model:"recommended") resolves to. */
-  async recommended(taskId) {
-    return (await this.leaderboard(taskId)).recommended;
-  }
 };
 
 // src/resources/evals.ts
-var BASE3 = "/v1/eval-sets";
+var BASE2 = "/v1/eval-sets";
 var RUNS = "/v1/eval-runs";
 var FRONTIER = "/v1/eval/frontier-models";
 var INLINE_MAX = 5 * 1024 * 1024;
@@ -743,17 +600,17 @@ var EvalSets = class {
   client;
   create(params) {
     const { files, data } = itemsJsonl(params.task, params.items, params.name);
-    return this.client.request("POST", BASE3, { files, data, cast: evalSetFromCreate });
+    return this.client.request("POST", BASE2, { files, data, cast: evalSetFromCreate });
   }
   list() {
-    return this.client.request("GET", BASE3, { cast: evalSetList });
+    return this.client.request("GET", BASE2, { cast: evalSetList });
   }
   async retrieve(evalSetId) {
-    const raw = await this.client.request("GET", `${BASE3}/${evalSetId}`);
+    const raw = await this.client.request("GET", `${BASE2}/${evalSetId}`);
     return new EvalSet((raw ?? {}).eval_set ?? {});
   }
   async delete(evalSetId) {
-    await this.client.request("DELETE", `${BASE3}/${evalSetId}`);
+    await this.client.request("DELETE", `${BASE2}/${evalSetId}`);
   }
   /**
    * Attach a binary doc (PDF/image) to one row's blob field. Collapses the
@@ -763,14 +620,14 @@ var EvalSets = class {
   async uploadDocument(evalSetId, file, opts) {
     const { blob, filename, size, mime } = await toBlob(file, opts.mime);
     if (size < INLINE_MAX) {
-      return this.client.request("POST", `${BASE3}/${evalSetId}/attach-blob`, {
+      return this.client.request("POST", `${BASE2}/${evalSetId}/attach-blob`, {
         files: { file: { filename, content: blob, contentType: mime } },
         data: { idx: String(opts.idx), field_name: opts.fieldName, mime }
       });
     }
     const minted = await this.client.request(
       "POST",
-      `${BASE3}/${evalSetId}/blob-upload-url`,
+      `${BASE2}/${evalSetId}/blob-upload-url`,
       { body: { idx: opts.idx, field_name: opts.fieldName, mime, file_size: size } }
     );
     const put = await globalThis.fetch(minted.upload_url, {
@@ -782,7 +639,7 @@ var EvalSets = class {
     if (put.status !== 200 && put.status !== 201) {
       throw new ParetaError(`blob upload PUT failed: ${put.status}`);
     }
-    return this.client.request("POST", `${BASE3}/${evalSetId}/blob-upload-complete`, {
+    return this.client.request("POST", `${BASE2}/${evalSetId}/blob-upload-complete`, {
       body: { idx: opts.idx, field_name: opts.fieldName, storage_uri: minted.storage_uri, mime }
     });
   }
@@ -865,9 +722,41 @@ var Evals = class {
   }
 };
 
+// src/resources/auto.ts
+var Auto = class {
+  constructor(client) {
+    this.client = client;
+  }
+  client;
+  /** Your org's `model: "auto"` traffic, rolled up. Read-only, free. */
+  metrics() {
+    return this.client.request("GET", "/v1/auto/metrics");
+  }
+  /**
+   * One prompt against a frontier vendor for a side-by-side with
+   * `model: "auto"` — METERED at the vendor's actual token cost (a failed
+   * vendor call bills $0). Allowed models: gpt-5.5, gemini-3-5-flash,
+   * gemini-3-1-pro, claude-sonnet-4-6.
+   */
+  compareFrontier(params) {
+    return this.client.request(
+      "POST",
+      "/v1/playground/frontier",
+      { body: params }
+    );
+  }
+};
+
 // src/client.ts
 var DEFAULT_BASE_URL = "https://api.pareta.ai";
-var DEFAULT_TIMEOUT_MS = 6e4;
+function randomKey() {
+  try {
+    return globalThis.crypto?.randomUUID?.().replace(/-/g, "") ?? Math.random().toString(36).slice(2) + Date.now().toString(36);
+  } catch {
+    return Math.random().toString(36).slice(2) + Date.now().toString(36);
+  }
+}
+var DEFAULT_TIMEOUT_MS = 6e5;
 var DEFAULT_MAX_RETRIES = 2;
 var RETRY_STATUSES = /* @__PURE__ */ new Set([408, 409, 429, 500, 502, 503, 504]);
 function normalizeBaseURL(url) {
@@ -882,9 +771,9 @@ var Pareta = class _Pareta {
   // Resource namespaces (assigned in the constructor; filled in slice by slice).
   chat;
   models;
-  endpoints;
   tasks;
   evals;
+  auto;
   constructor(options = {}) {
     if (!options.apiKey) {
       throw new ParetaError(
@@ -902,9 +791,9 @@ var Pareta = class _Pareta {
     this.fetchImpl = options.fetch ? f : f.bind(globalThis);
     this.chat = new Chat(this);
     this.models = new Models(this);
-    this.endpoints = new Endpoints(this);
     this.tasks = new Tasks(this);
     this.evals = new Evals(this);
+    this.auto = new Auto(this);
   }
   /** Build from PARETA_API_KEY (+ optional PARETA_BASE_URL); explicit opts win. */
   static fromEnv(options = {}) {
@@ -991,6 +880,9 @@ var Pareta = class _Pareta {
     const url = this.buildURL(path, params);
     const isMultipart = files != null || data != null;
     const headers = this.headers({ jsonBody: body != null && !isMultipart });
+    if (method.toUpperCase() === "POST" && !headers["Idempotency-Key"]) {
+      headers["Idempotency-Key"] = `pareta-js-${randomKey()}`;
+    }
     let payload;
     if (isMultipart) {
       payload = this.buildFormData(files, data);
@@ -1032,12 +924,15 @@ var Pareta = class _Pareta {
   /**
    * Yield parsed SSE objects. Retries ONLY the initial connect/handshake — once
    * bytes are flowing a mid-stream drop raises (re-issuing would re-run a
-   * metered generation / re-trigger a deploy). `events:true` → `{event,data}`.
+   * metered generation). `events:true` → `{event,data}`.
    */
   async *stream(method, path, opts = {}) {
     const { body, params, cast, events } = opts;
     const url = this.buildURL(path, params);
     const headers = this.headers({ stream: true, jsonBody: body != null });
+    if (method.toUpperCase() === "POST" && !headers["Idempotency-Key"]) {
+      headers["Idempotency-Key"] = `pareta-js-${randomKey()}`;
+    }
     const payload = body != null ? JSON.stringify(body) : void 0;
     for (let attempt = 0; attempt <= this.maxRetries; attempt++) {
       let started = false;
@@ -1089,9 +984,8 @@ exports.ChatCompletion = ChatCompletion;
 exports.ChatCompletionChunk = ChatCompletionChunk;
 exports.Choice = Choice;
 exports.ConflictError = ConflictError;
-exports.Endpoint = Endpoint;
-exports.EndpointMetrics = EndpointMetrics;
 exports.EndpointNotReadyError = EndpointNotReadyError;
+exports.EvalItemResult = EvalItemResult;
 exports.EvalResult = EvalResult;
 exports.EvalRun = EvalRun;
 exports.EvalRuns = EvalRuns;
@@ -1099,8 +993,6 @@ exports.EvalSet = EvalSet;
 exports.EvalSets = EvalSets;
 exports.FrontierModel = FrontierModel;
 exports.InsufficientCreditsError = InsufficientCreditsError;
-exports.Leaderboard = Leaderboard;
-exports.LeaderboardEntry = LeaderboardEntry;
 exports.Message = Message;
 exports.Model = Model;
 exports.ModelList = ModelList;