notdiamond 2.0.0-rc2 → 2.0.0-rc21

package/README.md

@@ -1,12 +1,26 @@
-# Not Diamond TypeScript API Library
+# Notdiamond TypeScript API Library
 
 [![NPM version](<https://img.shields.io/npm/v/notdiamond.svg?label=npm%20(stable)>)](https://npmjs.org/package/notdiamond) ![npm bundle size](https://img.shields.io/bundlephobia/minzip/notdiamond)
 
-This library provides convenient access to the Not Diamond REST API from server-side TypeScript or JavaScript.
+This library provides convenient access to the Notdiamond REST API from server-side TypeScript or JavaScript.
 
-The REST API documentation can be found on [docs.notdiamond.ai](https://docs.notdiamond.ai). The full API of this library can be found in [api.md](api.md).
+The library includes type definitions for all request params and response fields.
+
+## What is Prompt Adaptation?
+
+Not Diamond specializes in **Prompt Adaptation** - automatically optimizing your prompts to work optimally across different LLMs. Each language model has unique characteristics, instruction-following patterns, and preferred prompt formats. A prompt that works perfectly for GPT-5 might perform poorly on Claude or Gemini.
+Manually rewriting prompts for each model is time-consuming and requires deep expertise in each model's quirks.
 
-It is generated with [Stainless](https://www.stainless.com/).
+**The Solution**: Not Diamond automatically adapts your prompts with:
+
+- Automatic optimization of both system and user prompts
+- Built-in evaluation metrics
+- Minimum 25 training examples recommended
+- Processing time: typically 10–30 minutes
+
+## Documentation
+
+The REST API documentation can be found on [docs.notdiamond.ai](https://docs.notdiamond.ai). The full API of this library can be found in [api.md](api.md).
 
 ## Installation
 
@@ -16,130 +30,217 @@ npm install notdiamond
 
 ## Usage
 
-The full API of this library can be found in [api.md](api.md).
+#### Quick Start
 
 <!-- prettier-ignore -->
-```js
+
+```ts
 import NotDiamond from 'notdiamond';
 
-const client = new NotDiamond({
-  apiKey: process.env['NOT_DIAMOND_API_KEY'], // This is the default and can be omitted
-  environment: 'staging', // defaults to 'production'
+const client = new Notdiamond({
+  apiKey: process.env['NOTDIAMOND_API_KEY'], // This is the default and can be omitted
 });
 
-const response = await client.routing.selectModel({
-  llm_providers: [
-    { model: 'gpt-4o', provider: 'openai' },
-    { model: 'claude-sonnet-4-5-20250929', provider: 'anthropic' },
-    { model: 'gemini-1.5-pro', provider: 'google' },
+// Step 1: Start a prompt adaptation job with prototype mode
+const adaptation = await client.promptAdaptation.adapt({
+  fields: ['question'],
+  system_prompt: 'You are a mathematical assistant that counts digits accurately.',
+  target_models: [
+    {
+      model: 'claude-sonnet-4-5-20250929',
+      provider: 'anthropic',
+    },
+    {
+      model: 'gemini-2.5-flash',
+      provider: 'google',
+    },
   ],
-  messages: [
-    { role: 'system', content: 'You are a helpful assistant.' },
-    { role: 'user', content: 'Explain quantum computing in simple terms' },
+  template: 'Question: {question}\nAnswer:',
+  train_goldens: [
+    {
+      fields: { question: 'How many digits are in (23874045494*2789392485)?' },
+      answer: '20',
+    },
+    {
+      fields: { question: 'How many odd digits are in (999*777*555*333*111)?' },
+      answer: '10',
+    },
+    {
+      fields: { question: "How often does the number '17' appear in the digits of (287558*17)?" },
+      answer: '0',
+    },
+    {
+      fields: { question: 'How many even digits are in (222*444*666*888)?' },
+      answer: '16',
+    },
+    {
+      fields: { question: 'How many 0s are in (1234567890*1357908642)?' },
+      answer: '2',
+    },
   ],
+  test_goldens: [
+    {
+      fields: { question: 'How many digits are in (9876543210*123456)?' },
+      answer: '15',
+    },
+    {
+      fields: { question: 'How many odd digits are in (135*579*246)?' },
+      answer: '8',
+    },
+    {
+      fields: { question: "How often does the number '42' appear in the digits of (123456789*42)?" },
+      answer: '1',
+    },
+    {
+      fields: { question: 'How many even digits are in (1111*2222*3333)?' },
+      answer: '10',
+    },
+    {
+      fields: { question: 'How many 9s are in (999999*888888)?' },
+      answer: '11',
+    },
+  ],
+  evaluation_metric: 'LLMaaJ:Sem_Sim_1', // Or use custom evaluation
+  prototype_mode: true, // Enable faster prototype mode for quick experimentation
 });
 
-console.log(response.providers);
+console.log(`Adaptation started: ${adaptation.adaptation_run_id}`);
+
+// Step 2: Poll for completion (typically takes 10-30 minutes)
+let status;
+while (true) {
+  status = await client.promptAdaptation.getAdaptStatus(adaptation.adaptation_run_id);
+  console.log(`Status: ${status.status}`);
+  
+  if (status.status === 'queued') {
+    console.log(`Queue position: ${status.queue_position}`);
+  }
+  
+  if (status.status === 'completed' || status.status === 'failed') {
+    break;
+  }
+  
+  await new Promise(resolve => setTimeout(resolve, 30000)); // Poll every 30 seconds
+}
+
+// Step 3: Get the optimized prompts
+if (status.status === 'completed') {
+  const results = await client.promptAdaptation.getAdaptResults(adaptation.adaptation_run_id);
+  
+  console.log(`\nOrigin model baseline: ${results.origin_model.score.toFixed(2)}`);
+  
+  for (const target of results.target_models) {
+    console.log('\n' + '='.repeat(50));
+    console.log(`Model: ${target.model.model} (${target.model.provider})`);
+    console.log(`Optimized System Prompt:\n${target.system_prompt}`);
+    console.log(`Optimized Template:\n${target.user_message_template}`);
+    console.log(`Pre-optimization score: ${target.pre_optimization_score.toFixed(2)}`);
+    console.log(`Post-optimization score: ${target.post_optimization_score.toFixed(2)}`);
+    console.log(`Improvement: ${((target.post_optimization_score / target.pre_optimization_score - 1) * 100).toFixed(1)}%`);
+    console.log(`Cost: $${target.cost.toFixed(4)}`);
+  }
+}
 ```
 
-### Request & Response types
+For more details, see the [Prompt Adaptation documentation](https://docs.notdiamond.ai/docs/adapting-prompts-to-new-models).
 
-This library includes TypeScript definitions for all request params and response fields. You may import and use them like so:
+### Model Routing
+
+Select the best model automatically:
 
 <!-- prettier-ignore -->
 ```ts
 import NotDiamond from 'notdiamond';
 
 const client = new NotDiamond({
-  apiKey: process.env['NOT_DIAMOND_API_KEY'], // This is the default and can be omitted
-  environment: 'staging', // defaults to 'production'
+  apiKey: process.env['NOTDIAMOND_API_KEY'], // This is the default and can be omitted
 });
 
-const params: NotDiamond.RoutingSelectModelParams = {
+const response = await client.modelRouter.selectModel({
   llm_providers: [
     { model: 'gpt-4o', provider: 'openai' },
     { model: 'claude-sonnet-4-5-20250929', provider: 'anthropic' },
-    { model: 'gemini-1.5-pro', provider: 'google' },
+    { model: 'gemini-2.5-flash', provider: 'google' },
   ],
   messages: [
     { role: 'system', content: 'You are a helpful assistant.' },
     { role: 'user', content: 'Explain quantum computing in simple terms' },
   ],
-};
-const response: NotDiamond.RoutingSelectModelResponse = await client.routing.selectModel(params);
-```
-
-Documentation for each method, request param, and response field are available in docstrings and will appear on hover in most modern editors.
+});
 
-## File uploads
+console.log(response.providers);
+```
 
-Request parameters that correspond to file uploads can be passed in many different forms:
+### Train Custom Router
 
-- `File` (or an object with the same structure)
-- a `fetch` `Response` (or an object with the same structure)
-- an `fs.ReadStream`
-- the return value of our `toFile` helper
+For even better performance, you can train a custom router on your own dataset. This allows the router to learn the specific patterns and preferences of your use case:
 
 ```ts
 import fs from 'fs';
-import NotDiamond, { toFile } from 'notdiamond';
+import NotDiamond from 'notdiamond';
 
-const client = new NotDiamond();
+const client = new NotDiamond({
+  apiKey: process.env['NOTDIAMOND_API_KEY'], // This is the default and can be omitted
+});
 
-// If you have access to Node `fs` we recommend using `fs.createReadStream()`:
-await client.routing.createSurveyResponse({
-  constraint_priorities: 'constraint_priorities',
-  email: 'email',
-  llm_providers: 'llm_providers',
-  use_case_desc: 'use_case_desc',
-  user_id: 'user_id',
-  'x-token': 'x-token',
+await client.customRouter.trainCustomRouter({
   dataset_file: fs.createReadStream('/path/to/file'),
-});
+  language: 'english',
+  llm_providers:
+    '[{"provider": "openai", "model": "gpt-4o"}, {"provider": "anthropic", "model": "claude-sonnet-4-5-20250929"}]',
+  maximize: true,
+  prompt_column: 'prompt',
+});cust
+```
 
-// Or if you have the web `File` API you can pass a `File` instance:
-await client.routing.createSurveyResponse({
-  constraint_priorities: 'constraint_priorities',
-  email: 'email',
-  llm_providers: 'llm_providers',
-  use_case_desc: 'use_case_desc',
-  user_id: 'user_id',
-  'x-token': 'x-token',
-  dataset_file: new File(['my bytes'], 'file'),
-});
+### Request & Response types
 
-// You can also pass a `fetch` `Response`:
-await client.routing.createSurveyResponse({
-  constraint_priorities: 'constraint_priorities',
-  email: 'email',
-  llm_providers: 'llm_providers',
-  use_case_desc: 'use_case_desc',
-  user_id: 'user_id',
-  'x-token': 'x-token',
-  dataset_file: await fetch('https://somesite/file'),
-});
+This library includes TypeScript definitions for all request params and response fields. You may import and use them like so:
 
-// Finally, if none of the above are convenient, you can use our `toFile` helper:
-await client.routing.createSurveyResponse({
-  constraint_priorities: 'constraint_priorities',
-  email: 'email',
-  llm_providers: 'llm_providers',
-  use_case_desc: 'use_case_desc',
-  user_id: 'user_id',
-  'x-token': 'x-token',
-  dataset_file: await toFile(Buffer.from('my bytes'), 'file'),
-});
-await client.routing.createSurveyResponse({
-  constraint_priorities: 'constraint_priorities',
-  email: 'email',
-  llm_providers: 'llm_providers',
-  use_case_desc: 'use_case_desc',
-  user_id: 'user_id',
-  'x-token': 'x-token',
-  dataset_file: await toFile(new Uint8Array([0, 1, 2]), 'file'),
+<!-- prettier-ignore -->
+```ts
+import Notdiamond from 'notdiamond';
+
+const client = new Notdiamond({
+  apiKey: process.env['NOTDIAMOND_API_KEY'], // This is the default and can be omitted
 });
+
+const params: NotDiamond.PromptAdaptCreateParams = {
+  fields: ['question', 'context'],
+  system_prompt: 'You are a helpful assistant.',
+  target_models: [
+    {
+      model: 'claude-sonnet-4-5-20250929',
+      provider: 'anthropic',
+    },
+  ],
+  template: 'Context: {context}\nQuestion: {question}\nAnswer:',
+  train_goldens: [
+    {
+      fields: {
+        question: 'What is 2+2?',
+        context: 'Basic arithmetic',
+      },
+      answer: '4',
+    },
+    // Add at least 25 examples for best results
+  ],
+  test_goldens: [
+    {
+      fields: {
+        question: 'What is 3*3?',
+        context: 'Basic arithmetic',
+      },
+      answer: '9',
+    },
+  ],
+};
+const response: NotDiamond.PromptAdaptCreateResponse = await client.promptAdaptation.adapt(params);
+console.log(response.adaptation_run_id);
 ```
 
+Documentation for each method, request param, and response field are available in docstrings and will appear on hover in most modern editors.
+
 ## Handling errors
 
 When the library is unable to connect to the API,
@@ -148,27 +249,47 @@ a subclass of `APIError` will be thrown:
 
 <!-- prettier-ignore -->
 ```ts
-const response = await client.routing
-  .selectModel({
-    llm_providers: [
-      { model: 'gpt-4o', provider: 'openai' },
-      { model: 'claude-sonnet-4-5-20250929', provider: 'anthropic' },
-      { model: 'gemini-1.5-pro', provider: 'google' },
+import NotDiamond from 'notdiamond';
+
+const client = new NotDiamond();
+
+try {
+  await client.promptAdaptation.adapt({
+    fields: ['question'],
+    system_prompt: 'You are a helpful assistant.',
+    target_models: [
+      {
+        model: 'claude-sonnet-4-5-20250929',
+        provider: 'anthropic',
+      },
+      {
+        model: 'gemini-2.5-flash',
+        provider: 'google',
+      },
     ],
-    messages: [
-      { role: 'system', content: 'You are a helpful assistant.' },
-      { role: 'user', content: 'Explain quantum computing in simple terms' },
+    template: 'Question: {question}\nAnswer:',
+    train_goldens: [
+      { fields: { question: 'What is 2+2?' }, answer: '4' },
+      // Add at least 25 examples...
+    ],
+    test_goldens: [
+      { fields: { question: 'What is 3*3?' }, answer: '9' },
     ],
-  })
-  .catch(async (err) => {
-    if (err instanceof NotDiamond.APIError) {
-      console.log(err.status); // 400
-      console.log(err.name); // BadRequestError
-      console.log(err.headers); // {server: 'nginx', ...}
-    } else {
-      throw err;
-    }
   });
+} catch (err) {
+  if (err instanceof NotDiamond.APIConnectionError) {
+    console.log('The server could not be reached');
+    console.log(err.cause); // an underlying Error, likely from fetch()
+  } else if (err instanceof NotDiamond.RateLimitError) {
+    console.log('A 429 status code was received; we should back off a bit.');
+  } else if (err instanceof NotDiamond.APIError) {
+    console.log(err.status); // 400
+    console.log(err.name); // BadRequestError
+    console.log(err.headers); // {server: 'nginx', ...}
+  } else {
+    throw err;
+  }
+}
 ```
 
 Error codes are as follows:
@@ -184,27 +305,6 @@ Error codes are as follows:
 | >=500       | `InternalServerError`      |
 | N/A         | `APIConnectionError`       |
 
-### Retries
-
-Certain errors will be automatically retried 2 times by default, with a short exponential backoff.
-Connection errors (for example, due to a network connectivity problem), 408 Request Timeout, 409 Conflict,
-429 Rate Limit, and >=500 Internal errors will all be retried by default.
-
-You can use the `maxRetries` option to configure or disable this:
-
-<!-- prettier-ignore -->
-```js
-// Configure the default for all requests:
-const client = new NotDiamond({
-  maxRetries: 0, // default is 2
-});
-
-// Or, configure per-request:
-await client.routing.selectModel({ llm_providers: [{ model: 'gpt-4o', provider: 'openai' }, { model: 'claude-sonnet-4-5-20250929', provider: 'anthropic' }, { model: 'gemini-1.5-pro', provider: 'google' }], messages: [{ role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'Explain quantum computing in simple terms' }] }, {
-  maxRetries: 5,
-});
-```
-
 ### Timeouts
 
 Requests time out after 1 minute by default. You can configure this with a `timeout` option:
@@ -212,13 +312,13 @@ Requests time out after 1 minute by default. You can configure this with a `time
 <!-- prettier-ignore -->
 ```ts
 // Configure the default for all requests:
-const client = new NotDiamond({
+const client = new Notdiamond({
   timeout: 20 * 1000, // 20 seconds (default is 1 minute)
 });
 
-// Override per-request:
-await client.routing.selectModel({ llm_providers: [{ model: 'gpt-4o', provider: 'openai' }, { model: 'claude-sonnet-4-5-20250929', provider: 'anthropic' }, { model: 'gemini-1.5-pro', provider: 'google' }], messages: [{ role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: 'Explain quantum computing in simple terms' }] }, {
-  timeout: 5 * 1000,
+// Override per-request (note: prompt adaptation may take 10-30 minutes, so increase timeout accordingly):
+await client.prompt.getAdaptStatus('your-adaptation-run-id', {
+  timeout: 120 * 1000, // 2 minutes
 });
 ```
 
@@ -238,39 +338,61 @@ Unlike `.asResponse()` this method consumes the body, returning once it is parse
 
 <!-- prettier-ignore -->
 ```ts
-const client = new NotDiamond();
-
-const response = await client.routing
-  .selectModel({
-    llm_providers: [
-      { model: 'gpt-4o', provider: 'openai' },
-      { model: 'claude-sonnet-4-5-20250929', provider: 'anthropic' },
-      { model: 'gemini-1.5-pro', provider: 'google' },
+const client = new Notdiamond();
+
+const response = await client.prompt.adapt
+  .create({
+    fields: ['question'],
+    system_prompt: 'You are a helpful assistant.',
+    target_models: [
+      {
+        model: 'claude-sonnet-4-5-20250929',
+        provider: 'anthropic',
+      },
+      {
+        model: 'gemini-2.5-flash',
+        provider: 'google',
+      },
     ],
-    messages: [
-      { role: 'system', content: 'You are a helpful assistant.' },
-      { role: 'user', content: 'Explain quantum computing in simple terms' },
+    template: 'Question: {question}\nAnswer:',
+    train_goldens: [
+      { fields: { question: 'What is 2+2?' }, answer: '4' },
+      //Add at least 25 examples...
+    ],
+    test_goldens: [
+      { fields: { question: 'What is 3*3?' }, answer: '9' },
     ],
   })
   .asResponse();
 console.log(response.headers.get('X-My-Header'));
 console.log(response.statusText); // access the underlying Response object
 
-const { data: response, response: raw } = await client.routing
-  .selectModel({
-    llm_providers: [
-      { model: 'gpt-4o', provider: 'openai' },
-      { model: 'claude-sonnet-4-5-20250929', provider: 'anthropic' },
-      { model: 'gemini-1.5-pro', provider: 'google' },
+const { data: adaptResponse, response: raw } = await client.prompt.adapt
+  .create({
+    fields: ['question'],
+    system_prompt: 'You are a helpful assistant.',
+    target_models: [
+      {
+        model: 'claude-sonnet-4-5-20250929',
+        provider: 'anthropic',
+      },
+      {
+        model: 'gemini-2.5-flash',
+        provider: 'google',
+      },
+    ],
+    template: 'Question: {question}\nAnswer:',
+    train_goldens: [
+      { fields: { question: 'What is 2+2?' }, answer: '4' },
+      // Add at least 25 examples...
     ],
-    messages: [
-      { role: 'system', content: 'You are a helpful assistant.' },
-      { role: 'user', content: 'Explain quantum computing in simple terms' },
+    test_goldens: [
+      { fields: { question: 'What is 3*3?' }, answer: '9' },
     ],
   })
   .withResponse();
 console.log(raw.headers.get('X-My-Header'));
-console.log(response.providers);
+console.log(adaptResponse.adaptation_run_id);
 ```
 
 ### Logging
@@ -283,13 +405,13 @@ console.log(response.providers);
 
 The log level can be configured in two ways:
 
-1. Via the `NOT_DIAMOND_LOG` environment variable
+1. Via the `NOTDIAMOND_LOG` environment variable
 2. Using the `logLevel` client option (overrides the environment variable if set)
 
 ```ts
-import NotDiamond from 'notdiamond';
+import Notdiamond from 'notdiamond';
 
-const client = new NotDiamond({
+const client = new Notdiamond({
   logLevel: 'debug', // Show all log messages
 });
 ```
@@ -315,13 +437,13 @@ When providing a custom logger, the `logLevel` option still controls which messa
 below the configured level will not be sent to your logger.
 
 ```ts
-import NotDiamond from 'notdiamond';
+import Notdiamond from 'notdiamond';
 import pino from 'pino';
 
 const logger = pino();
 
-const client = new NotDiamond({
-  logger: logger.child({ name: 'NotDiamond' }),
+const client = new Notdiamond({
+  logger: logger.child({ name: 'Notdiamond' }),
   logLevel: 'debug', // Send all messages to pino, allowing it to filter
 });
 ```
@@ -350,10 +472,14 @@ parameter. This library doesn't validate at runtime that the request matches the
 send will be sent as-is.
 
 ```ts
-client.routing.selectModel({
-  // ...
-  // @ts-expect-error baz is not yet public
-  baz: 'undocumented option',
+client.promptAdaptation.adapt({
+  fields: ['question'],
+  system_prompt: 'You are a helpful assistant.',
+  target_models: [{ model: 'claude-sonnet-4-5-20250929', provider: 'anthropic' }],
+  template: 'Question: {question}\nAnswer:',
+  train_goldens: [{ fields: { question: 'What is 2+2?' }, answer: '4' }],
+  // @ts-expect-error experimental_feature is not yet public
+  experimental_feature: true,
 });
 ```
 
@@ -384,10 +510,10 @@ globalThis.fetch = fetch;
 Or pass it to the client:
 
 ```ts
-import NotDiamond from 'notdiamond';
+import Notdiamond from 'notdiamond';
 import fetch from 'my-fetch';
 
-const client = new NotDiamond({ fetch });
+const client = new Notdiamond({ fetch });
 ```
 
 ### Fetch options
@@ -395,9 +521,9 @@ const client = new NotDiamond({ fetch });
 If you want to set custom `fetch` options without overriding the `fetch` function, you can provide a `fetchOptions` object when instantiating the client or making a request. (Request-specific options override client options.)
 
 ```ts
-import NotDiamond from 'notdiamond';
+import Notdiamond from 'notdiamond';
 
-const client = new NotDiamond({
+const client = new Notdiamond({
   fetchOptions: {
     // `RequestInit` options
   },
@@ -412,11 +538,11 @@ options to requests:
 <img src="https://raw.githubusercontent.com/stainless-api/sdk-assets/refs/heads/main/node.svg" align="top" width="18" height="21"> **Node** <sup>[[docs](https://github.com/nodejs/undici/blob/main/docs/docs/api/ProxyAgent.md#example---proxyagent-with-fetch)]</sup>
 
 ```ts
-import NotDiamond from 'notdiamond';
+import Notdiamond from 'notdiamond';
 import * as undici from 'undici';
 
 const proxyAgent = new undici.ProxyAgent('http://localhost:8888');
-const client = new NotDiamond({
+const client = new Notdiamond({
   fetchOptions: {
     dispatcher: proxyAgent,
   },
@@ -426,9 +552,9 @@ const client = new NotDiamond({
 <img src="https://raw.githubusercontent.com/stainless-api/sdk-assets/refs/heads/main/bun.svg" align="top" width="18" height="21"> **Bun** <sup>[[docs](https://bun.sh/guides/http/proxy)]</sup>
 
 ```ts
-import NotDiamond from 'notdiamond';
+import Notdiamond from 'notdiamond';
 
-const client = new NotDiamond({
+const client = new Notdiamond({
   fetchOptions: {
     proxy: 'http://localhost:8888',
   },
@@ -438,10 +564,10 @@ const client = new NotDiamond({
 <img src="https://raw.githubusercontent.com/stainless-api/sdk-assets/refs/heads/main/deno.svg" align="top" width="18" height="21"> **Deno** <sup>[[docs](https://docs.deno.com/api/deno/~/Deno.createHttpClient)]</sup>
 
 ```ts
-import NotDiamond from 'npm:notdiamond';
+import Notdiamond from 'npm:notdiamond';
 
 const httpClient = Deno.createHttpClient({ proxy: { url: 'http://localhost:8888' } });
-const client = new NotDiamond({
+const client = new Notdiamond({
   fetchOptions: {
     client: httpClient,
   },