cactus-react-native 0.2.11 → 1.0.2

package/README.md

@@ -1,104 +1,585 @@
 ![Cactus Logo](assets/logo.png)
 
-Official React-Native plugin for Cactus, a framework for deploying LLM/VLM/TTS models locally in your app. Requires iOS 12.0+, Android API 24+ and Yarn. For iOS apps, ensure you have cocoapods or install with `brew install cocoapods`. For Android apps, you need Java 17 installed. Expo is strongly recommended.
-
 ## Resources
+
 [![cactus](https://img.shields.io/badge/cactus-000000?logo=github&logoColor=white)](https://github.com/cactus-compute/cactus) [![HuggingFace](https://img.shields.io/badge/HuggingFace-FFD21E?logo=huggingface&logoColor=black)](https://huggingface.co/Cactus-Compute/models?sort=downloads) [![Discord](https://img.shields.io/badge/Discord-5865F2?logo=discord&logoColor=white)](https://discord.gg/bNurx3AXTJ) [![Documentation](https://img.shields.io/badge/Documentation-4285F4?logo=googledocs&logoColor=white)](https://cactuscompute.com/docs/react-native)
 
 ## Installation
-Execute the following command in your project terminal:
+
 ```bash
-npm install cactus-react-native 
-# or 
-yarn add cactus-react-native
+npm install cactus-react-native react-native-nitro-modules
 ```
-*N/B*: To build locally or use this repo, see instructions in `example/README.md`
 
-## Text Completion
+## Quick Start
+
+Get started with Cactus in just a few lines of code:
+
 ```typescript
-import { CactusLM } from 'cactus-react-native';
+import { CactusLM, type Message } from 'cactus-react-native';
 
-const { lm, error } = await CactusLM.init({
-    model: '/path/to/model.gguf', // this is a local model file inside the app sandbox
-    n_ctx: 2048,
+// Create a new instance
+const cactusLM = new CactusLM();
+
+// Download the model
+await cactusLM.download({
+  onProgress: (progress) => console.log(`Download: ${Math.round(progress * 100)}%`)
 });
 
-const messages = [{ role: 'user', content: 'Hello!' }];
-const params = { n_predict: 100, temperature: 0.7 };
-const response = await lm.completion(messages, params);
+// Generate a completion
+const messages: Message[] = [
+  { role: 'user', content: 'What is the capital of France?' }
+];
+
+const result = await cactusLM.complete({ messages });
+console.log(result.response); // "The capital of France is Paris."
+
+// Clean up resources
+await cactusLM.destroy();
 ```
-## Embeddings
- ```typescript
-import { CactusLM } from 'cactus-react-native';
 
-const { lm, error } = await CactusLM.init({
-    model: '/path/to/model.gguf', // local model file inside the app sandbox
-    n_ctx: 2048,
-    embedding: true,
-});
+**Using the React Hook:**
+
+```tsx
+import { useCactusLM } from 'cactus-react-native';
 
-const text = 'Your text to embed';
-const params = { normalize: true };
-const result = await lm.embedding(text, params);
+const App = () => {
+  const cactusLM = useCactusLM();
+
+  useEffect(() => {
+    // Download the model if not already available
+    if (!cactusLM.isDownloaded) {
+      cactusLM.download();
+    }
+  }, []);
+
+  const handleGenerate = () => {
+    // Generate a completion
+    cactusLM.complete({
+      messages: [{ role: 'user', content: 'Hello!' }],
+    });
+  };
+
+  if (cactusLM.isDownloading) {
+    return (
+      <Text>
+        Downloading model: {Math.round(cactusLM.downloadProgress * 100)}%
+      </Text>
+    );
+  }
+
+  return (
+    <>
+      <Button onPress={handleGenerate} title="Generate" />
+      <Text>{cactusLM.completion}</Text>
+    </>
+  );
+};
 ```
-## Visual Language Models
+
+## Language Model
+
+### Completion
+
+Generate text responses from the model by providing a conversation history.
+
+#### Class
+
 ```typescript
-import { CactusVLM } from 'cactus-react-native';
+import { CactusLM, type Message } from 'cactus-react-native';
 
-const { vlm, error } = await CactusVLM.init({
-    model: '/path/to/vision-model.gguf', // local model file inside the app sandbox
-    mmproj: '/path/to/mmproj.gguf', // local model file inside the app sandbox
-});
+const cactusLM = new CactusLM();
+
+const messages: Message[] = [{ role: 'user', content: 'Hello, World!' }];
+const onToken = (token: string) => { console.log('Token:', token) };
+
+const result = await cactusLM.complete({ messages, onToken });
+console.log('Completion result:', result);
+```
+
+#### Hook
+
+```tsx
+import { useCactusLM, type Message } from 'cactus-react-native';
 
-const messages = [{ role: 'user', content: 'Describe this image' }];
+const App = () => {
+  const cactusLM = useCactusLM();
 
-const params = {
-    images: ['/absolute/path/to/image.jpg'],
-    n_predict: 200,
-    temperature: 0.3,
+  const handleComplete = async () => {
+    const messages: Message[] = [{ role: 'user', content: 'Hello, World!' }];
+
+    const result = await cactusLM.complete({ messages });
+    console.log('Completion result:', result);
+  };
+
+  return (
+    <>
+      <Button title="Complete" onPress={handleComplete} />
+      <Text>{cactusLM.completion}</Text>
+    </>
+  );
 };
+```
+
+### Vision
+
+Vision allows you to pass images along with text prompts, enabling the model to analyze and understand visual content.
+
+#### Class
+
+```typescript
+import { CactusLM, type Message } from 'cactus-react-native';
+
+// Vision-capable model
+const cactusLM = new CactusLM({ model: 'lfm2-vl-450m' });
+
+const messages: Message[] = [
+  {
+    role: 'user',
+    content: "What's in the image?",
+    images: ['path/to/your/image'],
+  },
+];
+
+const result = await cactusLM.complete({ messages });
+console.log('Response:', result.response);
+```
+
+#### Hook
+
+```tsx
+import { useCactusLM, type Message } from 'cactus-react-native';
+
+const App = () => {
+  // Vision-capable model
+  const cactusLM = useCactusLM({ model: 'lfm2-vl-450m' });
+
+  const handleAnalyze = async () => {
+    const messages: Message[] = [
+      {
+        role: 'user',
+        content: "What's in the image?",
+        images: ['path/to/your/image'],
+      },
+    ];
 
-const response = await vlm.completion(messages, params);
+    await cactusLM.complete({ messages });
+  };
+
+  return (
+    <>
+      <Button title="Analyze Image" onPress={handleAnalyze} />
+      <Text>{cactusLM.completion}</Text>
+    </>
+  );
+};
 ```
-## Cloud Fallback
+
+### Tool Calling
+
+Enable the model to generate function calls by defining available tools and their parameters.
+
+#### Class
+
 ```typescript
-const { lm } = await CactusLM.init({
-  model: '/path/to/model.gguf',
-  n_ctx: 2048,
-}, undefined, 'your_cactus_token');
+import { CactusLM, type Message, type Tool } from 'cactus-react-native';
 
-// Try local first, fallback to cloud if local fails (its blazing fast)
-const embedding = await lm.embedding('text', undefined, 'localfirst');
+const tools: Tool[] = [
+  {
+    type: 'function',
+    name: 'get_weather',
+    description: 'Get current weather for a location',
+    parameters: {
+      type: 'object',
+      properties: {
+        location: {
+          type: 'string',
+          description: 'City name',
+        },
+      },
+      required: ['location'],
+    },
+  },
+];
 
-// local (default): strictly only run on-device
-// localfirst: fallback to cloud if device fails
-// remotefirst: primarily remote, run local if API fails
-// remote: strictly run on cloud
+const cactusLM = new CactusLM();
+
+const messages: Message[] = [
+  { role: 'user', content: "What's the weather in San Francisco?" },
+];
+
+const result = await cactusLM.complete({ messages, tools });
+console.log('Response:', result.response);
+console.log('Function calls:', result.functionCalls);
 ```
 
-## Agents
+#### Hook
+
+```tsx
+import { useCactusLM, type Message, type Tool } from 'cactus-react-native';
+
+const tools: Tool[] = [
+  {
+    type: 'function',
+    name: 'get_weather',
+    description: 'Get current weather for a location',
+    parameters: {
+      type: 'object',
+      properties: {
+        location: {
+          type: 'string',
+          description: 'City name',
+        },
+      },
+      required: ['location'],
+    },
+  },
+];
+
+const App = () => {
+  const cactusLM = useCactusLM();
+
+  const handleComplete = async () => {
+    const messages: Message[] = [
+      { role: 'user', content: "What's the weather in San Francisco?" },
+    ];
+
+    const result = await cactusLM.complete({ messages, tools });
+    console.log('Response:', result.response);
+    console.log('Function calls:', result.functionCalls);
+  };
+
+  return <Button title="Complete" onPress={handleComplete} />;
+};
+```
+
+### RAG (Retrieval Augmented Generation)
+
+RAG allows you to provide a corpus of documents that the model can reference during generation, enabling it to answer questions based on your data.
+
+#### Class
+
 ```typescript
-import { CactusAgent } from 'cactus-react-native';
+import { CactusLM, type Message } from 'cactus-react-native';
 
-// we recommend Qwen 3 family, 0.6B is great
-const { agent, error } = await CactusAgent.init({
-    model: '/path/to/model.gguf',
-    n_ctx: 2048,
+const cactusLM = new CactusLM({
+  corpusDir: 'path/to/your/corpus', // Directory containing .txt files
 });
 
-const weatherTool = agent.addTool(
-    (location: string) => `Weather in ${location}: 72°F, sunny`,
-    'Get current weather for a location',
-    {
-        location: { type: 'string', description: 'City name', required: true }
-    }
-);
+const messages: Message[] = [
+  { role: 'user', content: 'What information is in the documents?' },
+];
 
-const messages = [{ role: 'user', content: 'What\'s the weather in NYC?' }];
-  const result = await agent.completionWithTools(messages, {
-  n_predict: 200,
-  temperature: 0.7,
-});
+const result = await cactusLM.complete({ messages });
+console.log(result.response);
+```
+
+#### Hook
+
+```tsx
+import { useCactusLM, type Message } from 'cactus-react-native';
+
+const App = () => {
+  const cactusLM = useCactusLM({
+    corpusDir: 'path/to/your/corpus', // Directory containing .txt files
+  });
+
+  const handleAsk = async () => {
+    const messages: Message[] = [
+      { role: 'user', content: 'What information is in the documents?' },
+    ];
+
+    await cactusLM.complete({ messages });
+  };
+
+  return (
+    <>
+      <Button title="Ask Question" onPress={handleAsk} />
+      <Text>{cactusLM.completion}</Text>
+    </>
+  );
+};
+```
+
+### Embedding
+
+Convert text into numerical vector representations that capture semantic meaning, useful for similarity search and semantic understanding.
+
+#### Class
+
+```typescript
+import { CactusLM } from 'cactus-react-native';
+
+const cactusLM = new CactusLM();
+
+const result = await cactusLM.embed({ text: 'Hello, World!' });
+console.log('Embedding vector:', result.embedding);
+console.log('Embedding vector length:', result.embedding.length);
+```
+
+#### Hook
+
+```tsx
+import { useCactusLM } from 'cactus-react-native';
+
+const App = () => {
+  const cactusLM = useCactusLM();
+
+  const handleEmbed = async () => {
+    const result = await cactusLM.embed({ text: 'Hello, World!' });
+    console.log('Embedding vector:', result.embedding);
+    console.log('Embedding vector length:', result.embedding.length);
+  };
+
+  return <Button title="Embed" onPress={handleEmbed} />;
+};
+```
+
+## API Reference
+
+### CactusLM Class
+
+#### Constructor
+
+**`new CactusLM(params?: CactusLMParams)`**
+
+**Parameters:**
+- `model` - Model slug (default: `'qwen3-0.6'`).
+- `contextSize` - Context window size (default: `2048`).
+- `corpusDir` - Directory containing text files for RAG (default: `undefined`).
+
+#### Methods
+
+**`download(params?: CactusLMDownloadParams): Promise<void>`**
+
+Downloads the model. If the model is already downloaded, returns immediately with progress at 100%. Throws an error if a download is already in progress. Automatically refreshes the models list after successful download.
+
+**Parameters:**
+- `onProgress` - Callback for download progress (0-1).
+
+**`init(): Promise<void>`**
+
+Initializes the model and prepares it for inference. Safe to call multiple times (idempotent). Throws an error if the model is not downloaded yet. Automatically initializes telemetry if not already done.
+
+**`complete(params: CactusLMCompleteParams): Promise<CactusLMCompleteResult>`**
+
+Performs text completion with optional streaming and tool support. Automatically calls `init()` if not already initialized. Throws an error if a generation (completion or embedding) is already in progress.
+
+**Parameters:**
+- `messages` - Array of `Message` objects.
+- `options` - Generation options:
+  - `temperature` - Sampling temperature (default: model-optimized).
+  - `topP` - Nucleus sampling threshold (default: model-optimized).
+  - `topK` - Top-K sampling limit (default: model-optimized).
+  - `maxTokens` - Maximum number of tokens to generate (default: `512`).
+  - `stopSequences` - Array of strings to stop generation (default: `undefined`).
+- `tools` - Array of `Tool` objects for function calling (default: `undefined`).
+- `onToken` - Callback for streaming tokens.
+
+**`embed(params: CactusLMEmbedParams): Promise<CactusLMEmbedResult>`**
+
+Generates embeddings for the given text. Automatically calls `init()` if not already initialized. Throws an error if a generation (completion or embedding) is already in progress.
+
+**Parameters:**
+- `text` - Text to embed.
+
+**`stop(): Promise<void>`**
+
+Stops ongoing generation.
+
+**`reset(): Promise<void>`**
+
+Resets the model's internal state, clearing any cached context. Automatically calls `stop()` first.
+
+**`destroy(): Promise<void>`**
+
+Releases all resources associated with the model. Automatically calls `stop()` first. Safe to call even if the model is not initialized.
+
+**`getModels(params?: CactusLMGetModelsParams): Promise<CactusModel[]>`**
+
+Fetches available models and persists the results locally for caching. Returns cached results if available, unless `forceRefresh` is `true`. Checks the download status for each model and includes it in the results.
+
+**Parameters:**
+- `forceRefresh` - If `true`, fetches from the server and updates the local cache (default: `false`).
+
+### useCactusLM Hook
+
+The `useCactusLM` hook manages a `CactusLM` instance with reactive state. When model parameters (`model`, `contextSize`, or `corpusDir`) change, the hook creates a new instance and resets all state. The hook automatically cleans up resources when the component unmounts.
+
+#### State
+
+- `completion: string` - Current generated text. Automatically accumulated during streaming. Cleared before each new completion and when calling `reset()` or `destroy()`.
+- `isGenerating: boolean` - Whether the model is currently generating (completion or embedding). Both operations share this flag.
+- `isInitializing: boolean` - Whether the model is initializing.
+- `isDownloaded: boolean` - Whether the model is downloaded locally. Automatically checked when the hook mounts or model changes.
+- `isDownloading: boolean` - Whether the model is being downloaded.
+- `downloadProgress: number` - Download progress (0-1). Reset to `0` after download completes.
+- `error: string | null` - Last error message from any operation, or `null` if there is no error. Cleared before starting new operations.
+
+#### Methods
+
+- `download(params?: CactusLMDownloadParams): Promise<void>` - Downloads the model. Updates `isDownloading` and `downloadProgress` state during download. Sets `isDownloaded` to `true` on success.
+- `init(): Promise<void>` - Initializes the model for inference. Sets `isInitializing` to `true` during initialization.
+- `complete(params: CactusLMCompleteParams): Promise<CactusLMCompleteResult>` - Generates text completions. Automatically accumulates tokens in the `completion` state during streaming. Sets `isGenerating` to `true` while generating. Clears `completion` before starting.
+- `embed(params: CactusLMEmbedParams): Promise<CactusLMEmbedResult>` - Generates embeddings for the given text. Sets `isGenerating` to `true` during operation.
+- `stop(): Promise<void>` - Stops ongoing generation. Clears any errors.
+- `reset(): Promise<void>` - Resets the model's internal state, clearing cached context. Also clears the `completion` state.
+- `destroy(): Promise<void>` - Releases all resources associated with the model. Clears the `completion` state. Automatically called when the component unmounts.
+- `getModels(params?: CactusLMGetModelsParams): Promise<CactusModel[]>` - Fetches available models and returns them. Results are cached locally.
+
+## Type Definitions
+
+### CactusLMParams
+
+```typescript
+interface CactusLMParams {
+  model?: string;
+  contextSize?: number;
+  corpusDir?: string;
+}
+```
+
+### CactusLMDownloadParams
+
+```typescript
+interface CactusLMDownloadParams {
+  onProgress?: (progress: number) => void;
+}
+```
+
+### Message
+
+```typescript
+interface Message {
+  role: 'user' | 'assistant' | 'system';
+  content?: string;
+  images?: string[];
+}
+```
+
+### Options
+
+```typescript
+interface Options {
+  temperature?: number;
+  topP?: number;
+  topK?: number;
+  maxTokens?: number;
+  stopSequences?: string[];
+}
+```
+
+### Tool
+
+```typescript
+interface Tool {
+  type: 'function';
+  name: string;
+  description: string;
+  parameters: {
+    type: 'object';
+    properties: {
+      [key: string]: {
+        type: string;
+        description: string;
+      };
+    };
+    required: string[];
+  };
+}
+```
+
+### CactusLMCompleteParams
+
+```typescript
+interface CactusLMCompleteParams {
+  messages: Message[];
+  options?: Options;
+  tools?: Tool[];
+  onToken?: (token: string) => void;
+}
+```
+
+### CactusLMCompleteResult
+
+```typescript
+interface CactusLMCompleteResult {
+  success: boolean;
+  response: string;
+  functionCalls?: {
+    name: string;
+    arguments: { [key: string]: any };
+  }[];
+  timeToFirstTokenMs: number;
+  totalTimeMs: number;
+  tokensPerSecond: number;
+  prefillTokens: number;
+  decodeTokens: number;
+  totalTokens: number;
+}
+```
+
+### CactusLMEmbedParams
+
+```typescript
+interface CactusLMEmbedParams {
+  text: string;
+}
+```
+
+### CactusLMEmbedResult
+
+```typescript
+interface CactusLMEmbedResult {
+  embedding: number[];
+}
+```
+
+### CactusLMGetModelsParams
+
+```typescript
+interface CactusLMGetModelsParams {
+  forceRefresh?: boolean;
+}
+```
+
+### CactusModel
+
+```typescript
+interface CactusModel {
+  name: string;
+  slug: string;
+  quantization: number;
+  sizeMb: number;
+  downloadUrl: string;
+  supportsToolCalling: boolean;
+  supportsVision: boolean;
+  createdAt: Date;
+  isDownloaded: boolean;
+}
+```
+
+## Configuration
+
+### Telemetry
+
+Cactus offers powerful telemetry for all your projects. Create a token on the [Cactus dashboard](https://www.cactuscompute.com/dashboard).
+
+```typescript
+import { CactusConfig } from 'cactus-react-native';
+
+// Enable Telemetry for your project
+CactusConfig.telemetryToken = 'your-token-here';
+
+// Disable telemetry
+CactusConfig.isTelemetryEnabled = false;
+```
+
+## Performance Tips
+
+- **Model Selection** - Choose smaller models for faster inference on mobile devices.
+- **Context Size** - Reduce the context size to lower memory usage.
+- **Memory Management** - Always call `destroy()` when you're done with models to free up resources.
+
+## Example App
 
-await agent.release();
-```
+Check out [our example app](/example) for a complete React Native implementation.

package/android/CMakeLists.txt

@@ -0,0 +1,40 @@
+project(cactus)
+cmake_minimum_required(VERSION 3.9.0)
+
+set(PACKAGE_NAME cactus)
+set(CMAKE_VERBOSE_MAKEFILE ON)
+set(CMAKE_CXX_STANDARD 20)
+
+# Define C++ library and add all sources
+add_library(${PACKAGE_NAME} SHARED 
+    src/main/cpp/cpp-adapter.cpp
+    ../cpp/HybridCactus.cpp
+    ../cpp/HybridCactusUtil.cpp
+)
+
+add_library(libcactus STATIC IMPORTED)
+set_target_properties(libcactus PROPERTIES
+    IMPORTED_LOCATION "${CMAKE_CURRENT_LIST_DIR}/src/main/jniLibs/${ANDROID_ABI}/libcactus.a"
+)
+
+add_library(libcactus_util SHARED IMPORTED)
+set_target_properties(libcactus_util PROPERTIES
+    IMPORTED_LOCATION "${CMAKE_CURRENT_LIST_DIR}/src/main/jniLibs/${ANDROID_ABI}/libcactus_util.so"
+)
+
+# Add Nitrogen specs :)
+include(${CMAKE_SOURCE_DIR}/../nitrogen/generated/android/cactus+autolinking.cmake)
+
+# Set up local includes
+include_directories("src/main/cpp" "../cpp")
+
+find_library(LOG_LIB log)
+
+# Link all libraries together
+target_link_libraries(
+        ${PACKAGE_NAME}
+        ${LOG_LIB}
+        android # <-- Android core
+        libcactus
+        libcactus_util
+)