@trymirai/uzu 0.1.3 → 0.1.5

package/README.md

@@ -8,6 +8,7 @@
 <a href="https://docsend.com/v/76bpr/mirai2025"><img src="https://img.shields.io/badge/View-Deck-red" alt="View our deck"></a>
 <a href="mailto:alexey@getmirai.co,dima@getmirai.co,aleksei@getmirai.co?subject=Interested%20in%20Mirai"><img src="https://img.shields.io/badge/Send-Email-green" alt="Contact us"></a>
 <a href="https://docs.trymirai.com/components/inference-engine"><img src="https://img.shields.io/badge/Read-Docs-blue" alt="Read docs"></a>
+[![npm (scoped)](https://img.shields.io/npm/v/%40trymirai%2Fuzu)](https://www.npmjs.com/package/@trymirai/uzu)
 [![License](https://img.shields.io/badge/License-MIT-blue)](LICENSE)
 
 # uzu-ts
@@ -19,192 +20,185 @@ Node package for [uzu](https://github.com/trymirai/uzu), a **high-performance**
 - [Broad model support](https://trymirai.com/models)
 - Observable model manager
 
-## Quick Start
+## Setup
 
+Add the `uzu` dependency to your project's `package.json`:
 
-```bash
-# 1 /  Install TS dependencies (installs @napi-rs/cli locally)
-cd bindings/ts
-pnpm install
-
-# 2 /  Build the native addon in release mode
-pnpm run build
-
-# 3 /  Set your API key in `examples/api_key.ts`, then run the examples
-pnpm run chat
-pnpm run summarize
-pnpm run classify
+```json
+"dependencies": {
+    "@trymirai/uzu": "0.1.4"
+}
 ```
 
-### Setup
-
-Add the `uzu-ts` dependency to your project:
-
-Set up your project via [Platform](https://platform.trymirai.com), obtain an `API_KEY`, and initialize engine:
+Create and activate engine:
 
 ```ts
-import { Engine } from './uzu'
-
 const engine = new Engine()
-const licenseStatus = await engine.activate('API_KEY')
+const licenseStatus: LicenseStatus = await engine.activate(apiKey)
 ```
 
-### Refresh models registry:
+### Refresh models registry / list cloud models:
 
 ```ts
-const registry = await engine.updateRegistry()
-const modelIdentifiers = registry.map((m) => m.identifier)
+const localModels: LocalModel[] = await engine.updateRegistry()
+const localModelId = 'Meta-Llama-3.2-1B-Instruct-bfloat16'
 ```
 
 ### Download with progress handle
 
 ```ts
-const modelIdentifier = 'Meta-Llama-3.2-1B-Instruct-bfloat16'
+const donwloadHandle = engine.downloadHandle(localModelId)
+donwloadHandle.start()
 
-const handle = engine.downloadHandle(modelIdentifier)
-handle.start()
-
-for await (const downloadProgress of handle.progress()) {
-  console.log(`Progress: ${Math.round(downloadProgress.progress * 100)}%`)
+for await (const donwloadProgress of donwloadHandle.progress()) {
+    handleProgress(donwloadProgress)
 }
 ```
 
 Alternatively, you may use engine to control and observe model download:
 
 ```ts
-engine.download(modelIdentifier)
-engine.pause(modelIdentifier)
-engine.resume(modelIdentifier)
-engine.delete(modelIdentifier)
+// engine.download(localModelId)
+// engine.pause(localModelId)
+// engine.resume(localModelId)
+// engine.stop(localModelId)
+// engine.delete(localModelId)
 
-// ... later you can query state
-const state = engine.getState(modelIdentifier)
+const modelState: ModelDownloadState = engine.getState(localModelId)
 ```
 
-Possible model state values:
-
-- `.notDownloaded`
-- `.downloading(progress: Double)`
-- `.paused(progress: Double)`
-- `.downloaded`
-- `.error(message: String)`
-
 ### Session
 
 `Session` is the core entity used to communicate with the model:
 
 ```ts
-const session = engine.createSession({ type: 'Local', id: 'Meta-Llama-3.2-1B-Instruct-bfloat16' })
+// Choose one of the two options below by commenting/uncommenting:
+// const modelId: ModelID = { type: 'Cloud', id: cloudRepoId }
+const modelId: ModelID = { type: 'Local', id: localModelId }
+const session: Session = engine.createSession(modelId)
 ```
 
-`Session` offers different configuration presets that can provide significant performance boosts for common use cases like classification and summarization:
+### Chat
 
-```ts
-import { type SessionConfig } from './uzu'
+Load `Session` with a chat-configured config and run it with a specific prompt or a list of messages:
 
+```ts
 const config: SessionConfig = {
-  preset: { type: 'General' },
-  samplingSeed: { type: 'Default' },
-  contextLength: { type: 'Default' },
+    preset: { type: 'General' },
+    samplingSeed: { type: 'Custom', seed: 12345 },
+    contextLength: { type: 'Default' },
 }
 session.load(config)
 ```
 
-Once loaded, the same `Session` can be reused for multiple requests until you drop it. Each model may consume a significant amount of RAM, so it's important to keep only one session loaded at a time.
-
-### Inference
-
-After loading, you can run the `Session` with a specific prompt or a list of messages:
-
 ```ts
-import { SessionMessageRole, type SessionInput } from './uzu'
-
 const input: SessionInput = {
-  type: 'Messages',
-  messages: [
-    { role: SessionMessageRole.System, content: 'You are a helpful assistant' },
-    { role: SessionMessageRole.User, content: 'Tell about London' },
-  ],
+    type: 'Messages',
+    messages: [
+        {
+            role: SessionMessageRole.System,
+            content: 'You are a helpful assistant.'
+        },
+        {
+            role: SessionMessageRole.User,
+            content: 'Tell me a short, funny story about a robot.'
+        },
+    ],
 }
-
-const output = session.run(
-  input,
-  { tokensLimit: 128, samplingConfig: { type: 'Argmax' } },
-  (partialOutput) => {
-    // Access the current text using partialOutput.text
-    return true // Return true to continue generation
-  },
-)
 ```
 
-`SessionOutput` also includes generation metrics such as prefill duration and tokens per second. It’s important to note that you should run a **release** build to obtain accurate metrics.
+```ts
+const runConfig: SessionRunConfig = {
+    tokensLimit: 128,
+    samplingConfig: { type: 'Argmax' },
+}
+```
 
-### Presets
+```ts
+const output = session.run(input, runConfig, (partialOutput) => {
+    return handlePartialOutput(partialOutput)
+})
+```
 
-#### Summarization
+### Summarization
 
 In this example, we will extract a summary of the input text:
 
 ```ts
-import { type SessionConfig, type SessionInput } from './uzu'
-
-const textToSummarize =
-  'A Large Language Model (LLM) is a type of artificial intelligence that processes and generates human-like text. It is trained on vast datasets containing books, articles, and web content, allowing it to understand and predict language patterns. LLMs use deep learning, particularly transformer-based architectures, to analyze text, recognize context, and generate coherent responses. These models have a wide range of applications, including chatbots, content creation, translation, and code generation. One of the key strengths of LLMs is their ability to generate contextually relevant text based on prompts. They utilize self-attention mechanisms to weigh the importance of words within a sentence, improving accuracy and fluency. Examples of popular LLMs include OpenAI's GPT series, Google's BERT, and Meta's LLaMA. As these models grow in size and sophistication, they continue to enhance human-computer interactions, making AI-powered communication more natural and effective.'
-const text = `Text is: "${textToSummarize}". Write only summary itself.`
-
 const config: SessionConfig = {
-  preset: { type: 'Summarization' },
-  samplingSeed: { type: 'Default' },
-  contextLength: { type: 'Default' },
+    preset: { type: 'Summarization' },
+    samplingSeed: { type: 'Default' },
+    contextLength: { type: 'Default' },
 }
 session.load(config)
+```
 
-const input: SessionInput = { type: 'Text', text }
+```ts
+const input: SessionInput = {
+    type: 'Text',
+    text: `Text is: "${textToSummarize}". Write only summary itself.`,
+}
+```
 
-const output = session.run(
-  input,
-  { tokensLimit: 1024, samplingConfig: { type: 'Argmax' } },
-  () => true,
-)
+```ts
+const runConfig: SessionRunConfig = {
+    tokensLimit: 256,
+    samplingConfig: { type: 'Argmax' },
+}
 ```
 
-This will generate 34 output tokens with only 5 model runs during the generation phase, instead of 34 runs.
+```ts
+const output = session.run(input, runConfig, (partialOutput) => {
+    return handlePartialOutput(partialOutput)
+})
+```
 
-#### Classification
+### Classification
 
 Let’s look at a case where you need to classify input text based on a specific feature, such as `sentiment`:
 
 ```ts
-import { type SessionClassificationFeature, type SessionConfig, type SessionInput } from './uzu'
-
 const feature: SessionClassificationFeature = {
-  name: 'sentiment',
-  values: ['Happy', 'Sad', 'Angry', 'Fearful', 'Surprised', 'Disgusted'],
+    name: 'sentiment',
+    values: ['Happy', 'Sad', 'Angry', 'Fearful', 'Surprised', 'Disgusted'],
 }
+```
 
-const textToDetectFeature = "Today's been awesome! Everything just feels right, and I can't stop smiling."
-const text = `Text is: "${textToDetectFeature}". Choose ${feature.name} from the list: ${feature.values.join(
-  ', ',
-)}. Answer with one word. Don't add a dot at the end.`
-
+```ts
 const config: SessionConfig = {
-  preset: { type: 'Classification', feature },
-  samplingSeed: { type: 'Default' },
-  contextLength: { type: 'Default' },
+    preset: { type: 'Classification', feature },
+    samplingSeed: { type: 'Default' },
+    contextLength: { type: 'Default' },
 }
 session.load(config)
+```
 
-const input: SessionInput = { type: 'Text', text }
+```ts
+const textToDetectFeature =
+    "Today's been awesome! Everything just feels right, and I can't stop smiling."
+const classificationPrompt =
+    `Text is: "${textToDetectFeature}". Choose ${feature.name} from the list: ${feature.values.join(', ')}. ` +
+    "Answer with one word. Don't add a dot at the end."
+const input: SessionInput = { type: 'Text', text: classificationPrompt }
+```
 
-const output = session.run(
-  input,
-  { tokensLimit: 32, samplingConfig: { type: 'Argmax' } },
-  () => true,
-)
+```ts
+const runConfig: SessionRunConfig = {
+    tokensLimit: 32,
+    samplingConfig: { type: 'Argmax' },
+}
+```
+
+```ts
+const output = session.run(input, runConfig, (partialOutput) => {
+    return handlePartialOutput(partialOutput)
+})
 ```
 
 In this example, you will get the answer `Happy` immediately after the prefill step, and the actual generation won't even start.
 
 ## License
 
-This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
+
+

package/README.orig.md

@@ -0,0 +1,215 @@
+<p align="center">
+  <picture>
+    <img alt="Mirai" src="https://artifacts.trymirai.com/social/github/uzu-typescript.jpg" style="max-width: 100%;">
+  </picture>
+</p>
+
+<a href="https://artifacts.trymirai.com/social/about_us.mp3"><img src="https://img.shields.io/badge/Listen-Podcast-red" alt="Listen to our podcast"></a>
+<a href="https://docsend.com/v/76bpr/mirai2025"><img src="https://img.shields.io/badge/View-Deck-red" alt="View our deck"></a>
+<a href="mailto:alexey@getmirai.co,dima@getmirai.co,aleksei@getmirai.co?subject=Interested%20in%20Mirai"><img src="https://img.shields.io/badge/Send-Email-green" alt="Contact us"></a>
+<a href="https://docs.trymirai.com/components/inference-engine"><img src="https://img.shields.io/badge/Read-Docs-blue" alt="Read docs"></a>
+[![npm (scoped)](https://img.shields.io/npm/v/%40trymirai%2Fuzu)](https://www.npmjs.com/package/@trymirai/uzu)
+[![License](https://img.shields.io/badge/License-MIT-blue)](LICENSE)
+
+# uzu-ts
+
+Node package for [uzu](https://github.com/trymirai/uzu), a **high-performance** inference engine for AI models on Apple Silicon. It allows you to deploy AI directly in your app with **zero latency**, **full data privacy**, and **no inference costs**. You don’t need an ML team or weeks of setup - one developer can handle everything in minutes. Key features:
+
+- Simple, high-level API
+- Specialized configurations with significant performance boosts for common use cases like classification and summarization
+- [Broad model support](https://trymirai.com/models)
+- Observable model manager
+
+## Quick Start
+
+Set up your project via [Platform](https://platform.trymirai.com), obtain an `MIRAI_API_KEY`.
+
+```bash
+# Set your API key in `examples/api_key.ts`, then run the examples
+pnpm run chat
+pnpm run summarisation
+pnpm run classification
+```
+
+## Setup
+
+Add the `uzu` dependency to your project's `package.json`:
+
+```json
+"dependencies": {
+    "@trymirai/uzu": "0.1.4"
+}
+```
+
+Create and activate engine:
+
+```ts
+const engine = new Engine()
+const licenseStatus: LicenseStatus = await engine.activate(apiKey)
+```
+
+### Refresh models registry / list cloud models:
+
+```ts
+const localModels: LocalModel[] = await engine.updateRegistry()
+const localModelId = 'Meta-Llama-3.2-1B-Instruct-bfloat16'
+```
+
+### Download with progress handle
+
+```ts
+const donwloadHandle = engine.downloadHandle(localModelId)
+donwloadHandle.start()
+
+for await (const donwloadProgress of donwloadHandle.progress()) {
+    handleProgress(donwloadProgress)
+}
+```
+
+Alternatively, you may use engine to control and observe model download:
+
+```ts
+// engine.download(localModelId)
+// engine.pause(localModelId)
+// engine.resume(localModelId)
+// engine.stop(localModelId)
+// engine.delete(localModelId)
+
+const modelState: ModelDownloadState = engine.getState(localModelId)
+```
+
+### Session
+
+`Session` is the core entity used to communicate with the model:
+
+```ts
+// Choose one of the two options below by commenting/uncommenting:
+// const modelId: ModelID = { type: 'Cloud', id: cloudRepoId }
+const modelId: ModelID = { type: 'Local', id: localModelId }
+const session: Session = engine.createSession(modelId)
+```
+
+### Chat
+
+Load `Session` with a chat-configured config and run it with a specific prompt or a list of messages:
+
+```ts
+const config: SessionConfig = {
+    preset: { type: 'General' },
+    samplingSeed: { type: 'Custom', seed: 12345 },
+    contextLength: { type: 'Default' },
+}
+session.load(config)
+```
+
+```ts
+const input: SessionInput = {
+    type: 'Messages',
+    messages: [
+        {
+            role: SessionMessageRole.System,
+            content: 'You are a helpful assistant.'
+        },
+        {
+            role: SessionMessageRole.User,
+            content: 'Tell me a short, funny story about a robot.'
+        },
+    ],
+}
+```
+
+```ts
+const runConfig: SessionRunConfig = {
+    tokensLimit: 128,
+    samplingConfig: { type: 'Argmax' },
+}
+```
+
+```ts
+const output = session.run(input, runConfig, (partialOutput) => {
+    return handlePartialOutput(partialOutput)
+})
+```
+
+### Summarization
+
+In this example, we will extract a summary of the input text:
+
+```ts
+const config: SessionConfig = {
+    preset: { type: 'Summarization' },
+    samplingSeed: { type: 'Default' },
+    contextLength: { type: 'Default' },
+}
+session.load(config)
+```
+
+```ts
+const input: SessionInput = {
+    type: 'Text',
+    text: `Text is: "${textToSummarize}". Write only summary itself.`,
+}
+```
+
+```ts
+const runConfig: SessionRunConfig = {
+    tokensLimit: 256,
+    samplingConfig: { type: 'Argmax' },
+}
+```
+
+```ts
+const output = session.run(input, runConfig, (partialOutput) => {
+    return handlePartialOutput(partialOutput)
+})
+```
+
+### Classification
+
+Let’s look at a case where you need to classify input text based on a specific feature, such as `sentiment`:
+
+```ts
+const feature: SessionClassificationFeature = {
+    name: 'sentiment',
+    values: ['Happy', 'Sad', 'Angry', 'Fearful', 'Surprised', 'Disgusted'],
+}
+```
+
+```ts
+const config: SessionConfig = {
+    preset: { type: 'Classification', feature },
+    samplingSeed: { type: 'Default' },
+    contextLength: { type: 'Default' },
+}
+session.load(config)
+```
+
+```ts
+const textToDetectFeature =
+    "Today's been awesome! Everything just feels right, and I can't stop smiling."
+const classificationPrompt =
+    `Text is: "${textToDetectFeature}". Choose ${feature.name} from the list: ${feature.values.join(', ')}. ` +
+    "Answer with one word. Don't add a dot at the end."
+const input: SessionInput = { type: 'Text', text: classificationPrompt }
+```
+
+```ts
+const runConfig: SessionRunConfig = {
+    tokensLimit: 32,
+    samplingConfig: { type: 'Argmax' },
+}
+```
+
+```ts
+const output = session.run(input, runConfig, (partialOutput) => {
+    return handlePartialOutput(partialOutput)
+})
+```
+
+In this example, you will get the answer `Happy` immediately after the prefill step, and the actual generation won't even start.
+
+## License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
+
+

package/README.src.md

@@ -0,0 +1,146 @@
+<p align="center">
+  <picture>
+    <img alt="Mirai" src="https://artifacts.trymirai.com/social/github/uzu-typescript.jpg" style="max-width: 100%;">
+  </picture>
+</p>
+
+<a href="https://artifacts.trymirai.com/social/about_us.mp3"><img src="https://img.shields.io/badge/Listen-Podcast-red" alt="Listen to our podcast"></a>
+<a href="https://docsend.com/v/76bpr/mirai2025"><img src="https://img.shields.io/badge/View-Deck-red" alt="View our deck"></a>
+<a href="mailto:alexey@getmirai.co,dima@getmirai.co,aleksei@getmirai.co?subject=Interested%20in%20Mirai"><img src="https://img.shields.io/badge/Send-Email-green" alt="Contact us"></a>
+<a href="https://docs.trymirai.com/components/inference-engine"><img src="https://img.shields.io/badge/Read-Docs-blue" alt="Read docs"></a>
+[![npm (scoped)](https://img.shields.io/npm/v/%40trymirai%2Fuzu)](https://www.npmjs.com/package/@trymirai/uzu)
+[![License](https://img.shields.io/badge/License-MIT-blue)](LICENSE)
+
+# uzu-ts
+
+Node package for [uzu](https://github.com/trymirai/uzu), a **high-performance** inference engine for AI models on Apple Silicon. It allows you to deploy AI directly in your app with **zero latency**, **full data privacy**, and **no inference costs**. You don’t need an ML team or weeks of setup - one developer can handle everything in minutes. Key features:
+
+- Simple, high-level API
+- Specialized configurations with significant performance boosts for common use cases like classification and summarization
+- [Broad model support](https://trymirai.com/models)
+- Observable model manager
+
+## Quick Start
+
+Set up your project via [Platform](https://platform.trymirai.com), obtain an `MIRAI_API_KEY`.
+
+```bash
+# Set your API key in `examples/api_key.ts`, then run the examples
+pnpm run chat
+pnpm run summarisation
+pnpm run classification
+```
+
+## Setup
+
+Add the `uzu` dependency to your project's `package.json`:
+
+```json
+"dependencies": {
+    "@trymirai/uzu": "0.1.4"
+}
+```
+
+Create and activate engine:
+
+```ts
+// include:examples/chat.ts#activation lang=ts
+```
+
+### Refresh models registry / list cloud models:
+
+```ts
+// include:examples/chat.ts#registry lang=ts
+```
+
+### Download with progress handle
+
+```ts
+// include:examples/chat.ts#download lang=ts
+```
+
+Alternatively, you may use engine to control and observe model download:
+
+```ts
+// include:examples/chat.ts#model-state lang=ts
+```
+
+### Session
+
+`Session` is the core entity used to communicate with the model:
+
+```ts
+// include:examples/chat.ts#session-create lang=ts
+```
+
+### Chat
+
+Load `Session` with a chat-configured config and run it with a specific prompt or a list of messages:
+
+```ts
+// include:examples/chat.ts#session-load lang=ts
+```
+
+```ts
+// include:examples/chat.ts#session-input lang=ts
+```
+
+```ts
+// include:examples/chat.ts#session-run-config lang=ts
+```
+
+```ts
+// include:examples/chat.ts#session-run lang=ts
+```
+
+### Summarization
+
+In this example, we will extract a summary of the input text:
+
+```ts
+// include:examples/summarisation.ts#session-load lang=ts
+```
+
+```ts
+// include:examples/summarisation.ts#session-input lang=ts
+```
+
+```ts
+// include:examples/summarisation.ts#session-run-config lang=ts
+```
+
+```ts
+// include:examples/summarisation.ts#session-run lang=ts
+```
+
+### Classification
+
+Let’s look at a case where you need to classify input text based on a specific feature, such as `sentiment`:
+
+```ts
+// include:examples/classification.ts#classification-feature lang=ts
+```
+
+```ts
+// include:examples/classification.ts#session-load lang=ts
+```
+
+```ts
+// include:examples/classification.ts#session-input lang=ts
+```
+
+```ts
+// include:examples/classification.ts#session-run-config lang=ts
+```
+
+```ts
+// include:examples/classification.ts#session-run lang=ts
+```
+
+In this example, you will get the answer `Happy` immediately after the prefill step, and the actual generation won't even start.
+
+## License
+
+This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
+
+

package/package.json

@@ -1,22 +1,25 @@
 {
   "name": "@trymirai/uzu",
-  "version": "0.1.3",
+  "version": "0.1.5",
   "private": false,
   "main": "uzu.node",
   "types": "uzu.d.ts",
   "packageManager": "pnpm@10.14.0",
   "scripts": {
-    "build": "./build.sh",
-    "chat": "ts-node examples/chat_example.ts",
-    "summarize": "ts-node examples/summarization_example.ts",
-    "classify": "ts-node examples/classification_example.ts"
+    "lint": "eslint .",
+    "generate:readme": "cargo run -q --manifest-path ../../Cargo.toml --bin uzu_docgen -- --root $PWD --src $PWD/README.src.md --dest $PWD/README.md",
+    "chat": "ts-node examples/chat.ts",
+    "summarisation": "ts-node examples/summarisation.ts",
+    "classification": "ts-node examples/classification.ts"
   },
   "devDependencies": {
     "@napi-rs/cli": "3.1.2",
     "@types/node": "24.2.0",
+    "@types/progress": "2.0.7",
+    "eslint": "9.33.0",
     "ts-node": "10.9.2",
     "typescript": "5.9.2",
-    "@types/progress": "2.0.7"
+    "typescript-eslint": "8.40.0"
   },
   "dependencies": {
     "progress": "2.0.3"