react-native-litert-lm 0.3.6 → 0.4.0

package/README.md

@@ -4,16 +4,21 @@ High-performance on-device LLM inference for React Native, powered by [LiteRT-LM
 
 ## Features
 
-- 🚀 **Native Performance** — Kotlin (Android) / C++ (iOS) via Nitro Modules JSI bindings
-- 🧠 **Gemma 4 Ready** — First-class support for Gemma 4 E2B/E4B multimodal models (text + vision + audio)
-- ⚡ **GPU Acceleration** — GPU delegate (Android), Metal/MPS (iOS)
-- 🔄 **Streaming Support** — Token-by-token generation callbacks
-- 📱 **Cross-Platform** — Android API 26+ / iOS 15.0+
-- 🖼️ **Multimodal** — Image and audio input support
-- 🧵 **Async API** — Non-blocking inference on dedicated large-stack threads
-- 📊 **Real Memory Tracking** — OS-level memory metrics (RSS, native heap, available memory) via native APIs
-- 🧮 **Zero-Copy Buffers** — Memory snapshots stored in native ArrayBuffers via Nitro Modules
-- 📥 **Automatic Model Download** — Downloads models from URL with progress tracking and local caching
+- 🚀 **Native Swift Bridge (iOS)** — Bypasses Swift actor deadlocks (User Rule #1) via direct C FFI dispatched on a serial `dev.litert.engine` background queue.
+- 🤖 **Stateless Kotlin Bridge (Android)** — Fully conforms to `HybridLiteRTLMSpec` using direct JSI memory access.
+- ⚡ **Zero-Copy Multimodal API** — Native-owned `ArrayBuffer` mapping straight to FFI inputs for image/audio data without copy overhead (complying with User Rule #2).
+- 🧠 **Speculative Decoding** — Active multi-token prediction support with pre-flight model capability validation.
+- 🛠️ **Function / Tool Calling** — Native JSON-encoded schema specification support for structured outputs.
+- 🏎️ **GPU Acceleration** — Metal (iOS), OpenCL GPU delegate (Android, Pixel devices).
+- 🔄 **Streaming Support** — Non-blocking token-by-token callbacks.
+- 📊 **Real Memory Tracking** — OS-level memory metrics (RSS, native heap, available memory) via native APIs (complying with User Rule #3).
+- 📥 **Automatic Model Download** — Downloads models from URL with progress tracking and local caching.
+
+## Demo
+
+> Gemma 4 E2B running on-device on a Samsung Galaxy S22 (Snapdragon 8 Gen 1, 4 GB RAM) — CPU backend, streaming inference.
+
+<video src="https://github.com/user-attachments/assets/1da527ce-0432-4f8b-8899-474f81b2feea" width="300" controls></video>
 
 ## Installation
 
@@ -65,7 +70,8 @@ The `example/` directory contains a fully functional test app with a dark-themed
 - Multi-turn conversation with context retention
 - Performance benchmarking (tokens/sec, latency)
 - Real-time memory tracking
-- Quick chat interface
+- Speculative decoding & tool calling settings toggles
+- Zero-copy multimodal inference loading images/audio directly into ArrayBuffers
 
 ### Running the Example
 
@@ -87,42 +93,45 @@ The `example/` directory contains a fully functional test app with a dark-themed
    ```bash
    npx expo prebuild --clean
    npx expo run:android  # Android
-   npx expo run:ios      # iOS (requires XCFramework — see "Building the iOS Engine" below)
+   npx expo run:ios      # iOS (pre-linked with CLiteRTLM.xcframework)
    ```
 
-> **Note:** If you change native code (C++/Kotlin/Obj-C++), you must run `npx expo prebuild --clean` again before rebuilding.
+> **Note:** If you change native code (Swift/Kotlin), you must run `npx expo prebuild --clean` again before rebuilding.
 
 ## Model Management
 
-LiteRT-LM models (like Gemma 4) are large files (2–4 GB) and cannot be bundled into your app binary. They are downloaded at runtime.
+LiteRT-LM models (like Gemma 4) are large files (1–4 GB) and cannot be bundled into your app binary. They are downloaded at runtime.
 
 ### Automatic Downloading
 
-The library handles downloading automatically when you pass a URL to `loadModel` or `useModel`. Downloads include:
+Pass an HTTPS URL to `useModel()` or `loadModel()` — the library handles the rest:
 
 - **Progress tracking** — real-time download percentage via callbacks
 - **Local caching** — downloaded models are cached and reused across app launches
-  - **Android**: app-local temp directory
+  - **Android**: `files/models/` (app-private)
   - **iOS**: `Library/Caches/litert_models/` (survives app relaunch; reclaimable by iOS under storage pressure)
 - **HTTPS enforcement** — only secure URLs are accepted
 
-### Manual Downloading (Optional)
+### Manual Downloading
 
-If you prefer to manage downloads yourself (e.g., using `expo-file-system`), download the `.litertlm` file to a local path and pass that path to the library:
+If you need custom control over downloads (e.g., authentication headers for private model hosting, resumable downloads, or custom caching), use your preferred HTTP client and pass the local file path:
 
 ```typescript
-import * as FileSystem from "expo-file-system";
-import { GEMMA_4_E2B_IT } from "react-native-litert-lm";
+import { fetch } from "expo/fetch";
+import { File, Paths } from "expo-file-system";
+import { useModel } from "react-native-litert-lm";
 
-const localPath = `${FileSystem.documentDirectory}gemma-4-E2B-it.litertlm`;
+const MODEL_URL = "https://example.com/private-model.litertlm";
 
-async function downloadModel() {
-  const info = await FileSystem.getInfoAsync(localPath);
-  if (info.exists) return localPath;
+// Download with custom headers using expo/fetch
+const response = await fetch(MODEL_URL, {
+  headers: { Authorization: `Bearer ${token}` },
+});
+const modelFile = new File(Paths.cache, "my-model.litertlm");
+modelFile.write(await response.bytes());
 
-  await FileSystem.downloadAsync(GEMMA_4_E2B_IT, localPath);
-  return localPath;
-}
+// Pass the local path — no download occurs
+const { model, isReady } = useModel(modelFile.uri, { backend: "cpu" });
 ```
 
 ## Usage
@@ -194,32 +203,80 @@ llm.sendMessageAsync("Tell me a story", (token, done) => {
 });
 ```
 
-### Multimodal (Image / Audio)
+### Multimodal (Image / Audio) & Zero-Copy Buffers
+
+Multimodal features are fully supported via standard file paths or high-performance zero-copy `ArrayBuffer` objects:
 
-> **Note**: Multimodal is fully supported on Android. iOS has the code paths implemented but vision/audio executors may not be available in the current XCFramework build — use `checkMultimodalSupport()` to verify at runtime.
+#### 1. Zero-Copy Multimodal Messages (Recommended)
+This API uses Nitro Modules' native-backed `ArrayBuffer` directly mapped to native memory buffers, avoiding any base64 heap copying overhead (User Rule #2):
 
 ```typescript
 import { checkMultimodalSupport } from "react-native-litert-lm";
 
 const warning = checkMultimodalSupport();
 if (warning) {
-  console.warn(warning); // Experimental on iOS
+  console.warn(warning); // Experimental or unsupported on current platform (e.g. iOS simulator)
 } else {
-  // Image input (for vision models like Gemma 4)
-  // Images >1024px are automatically resized to prevent OOM
-  const response = await llm.sendMessageWithImage(
-    "What's in this image?",
-    "/path/to/image.jpg",
-  );
-
-  // Audio input
-  const transcription = await llm.sendMessageWithAudio(
-    "Transcribe this audio",
-    "/path/to/audio.wav",
-  );
+  // Read local assets or files straight into ArrayBuffers using fetch
+  const response = await fetch(Image.resolveAssetSource(require("./test.jpeg")).uri);
+  const imageBuffer = await response.arrayBuffer();
+
+  const reply = await llm.sendMultimodalMessage([
+    { type: "image", imageBuffer },
+    { type: "text", text: "Describe what is in this image." }
+  ]);
+  console.log(reply);
 }
 ```
 
+#### 2. Path-Based Multimodal Messages
+```typescript
+// Image input
+const response = await llm.sendMessageWithImage(
+  "What's in this image?",
+  "/path/to/image.jpg",
+);
+
+// Audio input
+const transcription = await llm.sendMessageWithAudio(
+  "Transcribe this audio",
+  "/path/to/audio.wav",
+);
+```
+
+### Speculative Decoding & Tools
+
+#### 1. Speculative Decoding (MTP)
+Enable speculative decoding in `LLMConfig` to accelerate inference using multi-token prediction when supported by your model:
+
+```typescript
+const { model } = useModel(GEMMA_4_E2B_IT, {
+  enableSpeculativeDecoding: true,
+});
+```
+
+#### 2. Function / Tool Calling
+Inject tools as an array of definitions, specifying parameter validation using standard JSON schema format:
+
+```typescript
+const { model } = useModel(GEMMA_4_E2B_IT, {
+  tools: [
+    {
+      name: "get_current_weather",
+      description: "Get the current weather for a location",
+      parametersJson: JSON.stringify({
+        type: "object",
+        properties: {
+          location: { type: "string", description: "The city and state, e.g. San Francisco, CA" },
+          unit: { type: "string", enum: ["celsius", "fahrenheit"] }
+        },
+        required: ["location"]
+      })
+    }
+  ]
+});
+```
+
 ### Performance Stats
 
 ```typescript
@@ -229,6 +286,8 @@ console.log(`Speed: ${stats.tokensPerSecond.toFixed(1)} tokens/sec`);
 console.log(`Time to first token: ${stats.timeToFirstToken.toFixed(0)} ms`);
 ```
 
+> **Note**: Stats are available for both sync (`sendMessage`) and streaming (`sendMessageAsync`) on both platforms. iOS uses real benchmark data from the C API; Android uses heuristic token counts with precise timing.
+
 ### Memory Tracking
 
 The library provides real OS-level memory data — no estimation. It reads directly from `mach_task_basic_info` (iOS) and `Debug.getNativeHeapAllocatedSize()` + `/proc/self/status` (Android).
@@ -307,19 +366,19 @@ const buffer = tracker.getNativeBuffer();
 
 ## Supported Models
 
-Download `.litertlm` models automatically using the exported URL constants, or manually from [HuggingFace](https://huggingface.co/litert-community):
+All exported model URLs are **public — no authentication required**. Pass them directly to `useModel()` or `loadModel()` for automatic downloading with progress tracking and local caching.
 
-| Constant               | Model                           | Size    | Min RAM | Auth Required  |
-| :--------------------- | :------------------------------ | :------ | :------ | :------------- |
-| `GEMMA_4_E2B_IT`       | Gemma 4 E2B (Multimodal, IT)    | 2.58 GB | 4 GB+   | ❌ No          |
-| `GEMMA_4_E4B_IT`       | Gemma 4 E4B (Higher Quality)    | 3.65 GB | 6 GB+   | ❌ No          |
-| `GEMMA_3N_E2B_IT_INT4` | Gemma 3n E2B (Int4, Multimodal) | ~1.3 GB | 4 GB+   | ✅ HuggingFace |
+| Constant               | Model                           | Size    | Min RAM | Source      |
+| :--------------------- | :------------------------------ | :------ | :------ | :---------- |
+| `GEMMA_4_E2B_IT`       | Gemma 4 E2B (Multimodal, IT)    | 2.58 GB | 4 GB+   | HuggingFace |
+| `GEMMA_4_E4B_IT`       | Gemma 4 E4B (Higher Quality)    | 3.65 GB | 6 GB+   | HuggingFace |
+| `GEMMA_3N_E2B_IT_INT4` | Gemma 3n E2B (Int4, Multimodal) | ~1.3 GB | 4 GB+   | litert.dev  |
 
-> **Recommended:** Use `GEMMA_4_E2B_IT` for most use cases. It's multimodal (text + vision + audio) and downloads directly from HuggingFace without requiring an account.
+> **Recommended:** Use `GEMMA_4_E2B_IT` for most use cases — multimodal (text + vision + audio) and the best quality-to-size ratio.
 >
-> **iOS Note:** Models larger than ~2 GB (like Gemma 4) require the `com.apple.developer.kernel.extended-virtual-addressing` entitlement. See [iOS Entitlements](#ios-entitlements) below.
+> **iOS Note:** Models larger than ~2 GB require the `com.apple.developer.kernel.extended-virtual-addressing` entitlement. See [iOS Entitlements](#ios-entitlements) below. Gemma 3n E2B (~1.3 GB) works without it.
 
-**Other compatible models** (download manually from HuggingFace):
+**Other compatible models** (download `.litertlm` files manually from [HuggingFace](https://huggingface.co/litert-community)):
 
 | Model         | Size    | Min RAM | Notes                 |
 | ------------- | ------- | ------- | --------------------- |
@@ -352,13 +411,15 @@ Loads a model from a local path or HTTPS URL.
 
 #### Backend Options
 
-| Backend | Engine              | Speed   | Notes                                          |
-| ------- | ------------------- | ------- | ---------------------------------------------- |
-| `'cpu'` | CPU inference       | Slowest | Always available, lower RAM requirement        |
-| `'gpu'` | GPU / Metal         | Fast    | Recommended default                            |
-| `'npu'` | NPU / Neural Engine | Fastest | Requires supported hardware; falls back to GPU |
+| Backend | Engine                         | Speed   | Notes                                                                              |
+| ------- | ------------------------------ | ------- | ---------------------------------------------------------------------------------- |
+| `'cpu'` | CPU inference                  | Slowest | Always available on all devices                                                    |
+| `'gpu'` | Metal (iOS) / OpenCL (Android) | Fast    | iOS: always available. Android: requires OpenCL (Pixel only, not Samsung/Qualcomm) |
+| `'npu'` | NPU / Neural Engine            | Fastest | Requires supported hardware; experimental                                          |
 
-> **iOS**: `'cpu'` is the recommended default backend. `'gpu'` (Metal/MPS) is also supported. The engine automatically tries multiple backend combinations if the primary one fails.
+> **iOS**: Both `'cpu'` and `'gpu'` (Metal) are supported. The engine automatically tries fallback backend combinations if the primary one fails.
+>
+> **Android GPU**: The GPU backend requires OpenCL, which is **not available on most Samsung and Qualcomm devices**. Use `checkBackendSupport('gpu')` to check before loading. The engine will throw a clear error if GPU is unsupported.
 
 ### `sendMessage(message): Promise<string>`
 
@@ -380,30 +441,10 @@ Send a message with audio (for audio-capable models like Gemma 4 E2B).
 
 Returns performance metrics from the last inference call.
 
-```typescript
-interface GenerationStats {
-  tokensPerSecond: number;
-  totalTime: number; // seconds
-  timeToFirstToken: number; // seconds
-  promptTokens: number;
-  completionTokens: number;
-  prefillSpeed: number; // tokens/sec
-}
-```
-
 ### `getMemoryUsage(): MemoryUsage`
 
 Returns real OS-level memory usage.
 
-```typescript
-interface MemoryUsage {
-  nativeHeapBytes: number;
-  residentBytes: number;
-  availableMemoryBytes: number;
-  isLowMemory: boolean;
-}
-```
-
 ### `getHistory(): Message[]`
 
 Returns the conversation history.
@@ -427,21 +468,19 @@ import {
   checkBackendSupport,
   checkMultimodalSupport,
   getRecommendedBackend,
-  applyGemmaTemplate,
-  applyPhiTemplate,
-  applyLlamaTemplate,
 } from "react-native-litert-lm";
 
-// Check if a backend is supported
-const warning = checkBackendSupport("npu"); // string | undefined
+// Check if GPU is supported on this device
+const gpuWarning = checkBackendSupport("gpu");
+
+// Check NPU support
+const npuWarning = checkBackendSupport("npu"); // string | undefined
+
+// Check multimodal support
 const mmError = checkMultimodalSupport(); // string | undefined
-const backend = getRecommendedBackend(); // 'gpu' | 'cpu'
 
-// Manual prompt formatting (advanced)
-const prompt = applyGemmaTemplate(
-  [{ role: "user", content: "Hello!" }],
-  "You are helpful.",
-);
+// Get recommended backend
+const backend = getRecommendedBackend(); // 'cpu'
 ```
 
 ## Requirements
@@ -452,29 +491,29 @@ const prompt = applyGemmaTemplate(
 | react-native-nitro-modules | 0.35.0+       |
 | Android API                | 26+ (ARM64)   |
 | iOS                        | 15.0+ (ARM64) |
-| LiteRT-LM Engine           | 0.10.2        |
+| LiteRT-LM Engine           | 0.12.0        |
 
 ## Platform Support
 
-| Platform | Status   | Architecture | Backends         |
-| -------- | -------- | ------------ | ---------------- |
-| Android  | ✅ Ready | arm64-v8a    | CPU, GPU, NPU    |
-| iOS      | ✅ Ready | arm64        | CPU, GPU (Metal) |
+| Platform | Status   | Architecture | Backends                                          |
+| -------- | -------- | ------------ | ------------------------------------------------- |
+| Android  | ✅ Ready | arm64-v8a    | CPU (all devices), GPU (OpenCL devices only), NPU |
+| iOS      | ✅ Ready | arm64        | CPU, GPU (Metal — always available)               |
 
 ### iOS Feature Matrix
 
-| Feature                      | Status | Notes                                                 |
-| ---------------------------- | ------ | ----------------------------------------------------- |
-| Text inference (blocking)    | ✅     | Via LiteRT-LM C API                                   |
-| Text inference (streaming)   | ✅     | Token-by-token callbacks                              |
-| CPU inference                | ✅     | Recommended default backend                           |
-| GPU inference (Metal/MPS)    | ✅     | Supported via `backend: 'gpu'`                        |
-| Model download with progress | ✅     | NSURLSession, cached in `Caches/`                     |
-| Memory tracking              | ✅     | `mach_task_basic_info`                                |
-| Multi-turn conversation      | ✅     | Context retained across turns                         |
-| Multimodal (image/audio)     | 🧪     | Code paths exist; vision/audio executors experimental |
-| Constrained decoding         | ❌     | Requires llguidance Rust runtime                      |
-| Function calling             | ❌     | Requires Rust CXX bridge runtime                      |
+| Feature                      | Status | Notes                                                  |
+| ---------------------------- | ------ | ------------------------------------------------------ |
+| Text inference (blocking)    | ✅     | Direct FFI using `dev.litert.engine` background queue  |
+| Text inference (streaming)   | ✅     | Token-by-token callbacks                               |
+| CPU inference                | ✅     | Safe fallback default                                  |
+| GPU inference (Metal/MPS)    | ✅     | Supported via `backend: 'gpu'`                         |
+| Model download with progress | ✅     | URLSession-based, cached in `Caches/`                  |
+| Memory tracking              | ✅     | Real-time Resident Set Size (RSS) tracking             |
+| Multi-turn conversation      | ✅     | Context retained across turns                          |
+| Multimodal (image/audio)     | ✅     | Zero-copy `ArrayBuffer` mapping to FFI input buffers   |
+| Speculative Decoding         | ✅     | Dynamic capabilities check during model pre-load       |
+| Function / Tool Calling      | ✅     | Supported via JSON-encoded schema specification        |
 
 ### iOS Entitlements
 
@@ -489,78 +528,88 @@ Add to your app's `.entitlements` file:
 
 > **Note:** This entitlement requires a **paid Apple Developer account** ($99/year). Gemma 3n E2B (~1.3 GB) works without it.
 
-## Building the iOS Engine
+## iOS FFI Architecture & Integration
 
-The iOS build uses a **Bazel-to-XCFramework pipeline** that compiles the LiteRT-LM C engine and all transitive dependencies into a static library (~82–84 MB).
+The library uses a highly optimized Swift Direct-FFI bridge that links directly with the pre-compiled C library `CLiteRTLM.xcframework`.
 
-### Prerequisites
+### Key Design Commitments
 
-- **Bazel 7.6.1+** (via [Bazelisk](https://github.com/bazelbuild/bazelisk) recommended)
-- **Xcode command line tools** (`xcode-select --install`)
+1. **JSI Thread Safety (User Rule #1)**:
+   - The JSI/JS thread must never be blocked by native synchronous lock-waiting operations.
+   - We dispatch all FFI calls to a serial background `dev.litert.engine` queue, executing callbacks asynchronously to prevent deadlocking JSI execution.
 
-### Build
+2. **Zero-Copy Memory Pipelines (User Rule #2)**:
+   - Enforce the use of Nitro Modules' `ArrayBuffer` directly referencing native memory pointers (`ArrayBuffer.data`) when processing heavy media assets like images or audio.
 
-```bash
-./scripts/build-ios-engine.sh
+3. **Manual FFI Resource Management (User Rule #3)**:
+   - Raw pointers (`LiteRtLmEngine*`, `LiteRtLmConversation*`) are manually allocated and strictly deallocated inside Swift `deinit` and `close()` destructors to guarantee 0% memory leaks during prolonged inference sessions.
+
+### Architecture Topology
+
+```
+┌──────────────────────────────────────────────────────────┐
+│  React Native (TypeScript / JavaScript)                  │
+├──────────────────────────────────────────────────────────┤
+│  Nitro Modules JSI Bindings (`HybridLiteRTLMSpec`)       │
+├─────────────────────────────┬────────────────────────────┤
+│  Android (Kotlin)           │  iOS (Swift Direct FFI)    │
+│  `HybridLiteRTLM.kt`        │  `HybridLiteRTLM.swift`    │
+│  `litertlm-android` AAR     │  `CLiteRTLM.xcframework`   │
+└─────────────────────────────┴────────────────────────────┘
 ```
 
-This will:
+#### Android Bridging
+- Conforms fully to `HybridLiteRTLMSpec` using Kotlin.
+- Incorporates Proguard keep rules to prevent dynamic JSI/JNI code stripping.
+- Declares `<uses-native-library android:name="libOpenCL.so" android:required="false" />` to load dynamic OpenCL for GPU delegate acceleration on Android 12+ without throwing platform installer exceptions.
 
-1. Clone/checkout LiteRT-LM `v0.10.2` source into `.litert-lm-build/`
-2. Apply `scripts/patches/ios-engine-fixes.patch` (PromptTemplate simplification, linker fixes)
-3. Build `//c:engine` for `ios_arm64` and `ios_sim_arm64` via Bazel
-4. Collect all transitive `.o` files (engine, protobuf, re2, sentencepiece, etc.)
-5. Compile C/C++ stubs for unavailable Rust dependencies
-6. Merge ~1,909 object files into a static library via `libtool`
-7. Package into `ios/Frameworks/LiteRTLM.xcframework`
+#### iOS Bridging
+- Entirely written in native Swift (`HybridLiteRTLM.swift`) calling direct FFI.
+- Avoids the upstream Swift SDK `actor` lock-blocking deadlocks by utilizing low-level C functions directly.
+- Implements custom `getMemoryUsage` that queries the OS directly via `mach_task_basic_info` to get precise real-time Resident Set Size (RSS) metrics.
 
-### Output
+## Testing
 
-```
-ios/Frameworks/LiteRTLM.xcframework/
-├── Info.plist
-├── ios-arm64/LiteRTLM.framework/              # Device
-│   ├── LiteRTLM                                # ~82 MB static library
-│   └── Headers/litert_lm_engine.h
-└── ios-arm64-simulator/LiteRTLM.framework/    # Simulator
-    ├── LiteRTLM                                # ~84 MB static library
-    └── Headers/litert_lm_engine.h
-```
+The library includes a comprehensive multi-tier unit testing suite designed to run quickly on host machines (CI runners or local development environments) without requiring a physical test device.
 
-### FFI Stubs
+### 1. JavaScript / TypeScript Layer (Jest)
 
-Certain LiteRT-LM features depend on Rust libraries (llguidance, CXX bridge, MinijinjaTemplate) that are not available in the iOS Bazel build. These are replaced with stubs:
+The JS/TS layer uses Jest to validate the `useModel` hook, download progress callbacks, URL query scrubbing, file storage helpers, and the zero-copy native memory tracker buffer allocations.
 
-| Stub File                            | Location         | Purpose                                  |
-| ------------------------------------ | ---------------- | ---------------------------------------- |
-| `cxx_bridge_stubs.cc`                | `scripts/stubs/` | CXX bridge runtime + Rust FFI type stubs |
-| `llguidance_stubs.c`                 | `scripts/stubs/` | llguidance constrained decoding C API    |
-| `gemma_model_constraint_provider.cc` | `scripts/stubs/` | Gemma constraint provider factory        |
+* **Setup & Mocking**: Includes an active stub (`src/__mocks__/react-native-nitro-modules.ts`) that mocks the Nitro Modules `HybridObject` architecture.
+* **How to run**:
+  ```bash
+  npm run test
+  ```
 
-Additionally, `PromptTemplate` is patched at build time to use a simplified C++ template formatter instead of the Rust MinijinjaTemplate, which avoids all Rust FFI calls during conversation setup.
+### 2. Android Kotlin Layer (Robolectric)
 
-> **Text inference works fully without these Rust components.** Only constrained decoding, function calling parsers, and advanced Jinja2 template features are affected.
+The Android layer uses local JUnit Robolectric tests to run Android code on the JVM, sandboxing OS dependencies. It validates HTTPS schema constraints, path traversal mitigations, and initial telemetry states.
 
-## Architecture
+* **Setup & Mocking**: Uses a local shadow `Promise` implementation to test thread-asynchronous errors.
+* **How to run**:
+  ```bash
+  cd example/android
+  ./gradlew :react-native-litert-lm:testDebugUnitTest
+  ```
 
-```
-┌─────────────────────────────────────────────────┐
-│  React Native (TypeScript)                      │
-│  useModel() / createLLM() / sendMessage()       │
-├─────────────────────────────────────────────────┤
-│  Nitro Modules JSI Bridge                       │
-├──────────────────────┬──────────────────────────┤
-│  Android (Kotlin)    │  iOS (C++)               │
-│  HybridLiteRTLM.kt   │  HybridLiteRTLM.cpp      │
-│  litertlm-android    │  LiteRTLM C API          │
-│  AAR (GPU delegate)  │  XCFramework (Metal)     │
-└──────────────────────┴──────────────────────────┘
-```
+### 3. iOS Swift Layer (XCTest)
+
+The iOS layer leverages native XCTests integrated directly into CocoaPods via standard development test specs. It verifies FFI path traversal blocking, non-HTTPS download blocks, automatic `deinit` cleanup, and Mach-based telemetry bounds.
 
-- **Android**: Kotlin (`HybridLiteRTLM.kt`) interfacing with the `litertlm-android` AAR.
-- **iOS**: C++ (`HybridLiteRTLM.cpp`) interfacing with the LiteRT-LM C API via a prebuilt `LiteRTLM.xcframework`. All engine operations (load, inference, streaming) run on dedicated `pthread` threads with 8 MB stack to accommodate XNNPack's stack requirements. Platform-specific code (model downloading, file management) is in Objective-C++ (`ios/IOSDownloadHelper.mm`).
+* **How to run**:
+  1. Boot your preferred iOS simulator (e.g., iPhone 16 running iOS 18.6).
+  2. Run the tests using `xcodebuild`:
+     ```bash
+     cd example/ios
+     xcodebuild test -workspace LLMTest.xcworkspace -scheme react-native-litert-lm-Unit-Tests -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 16'
+     ```
 
-> **For contributors**: Changes to `cpp/HybridLiteRTLM.cpp` do not affect Android. Feature changes must be applied to both the Kotlin and C++ implementations.
+### Security & Sanitization Protections Checked
+Every test run automatically asserts:
+- **Defense in depth for download boundaries**: Blocks non-HTTPS schemes at both JS model factory and low-level native layers.
+- **Path Traversal protections**: Prevents directory traversal attacks (`..`, `/`, `\`) in download and deletion APIs.
+- **Telemetry sanity**: Ensures zero-leak memory usage telemetry boundaries stay strictly linear.
 
 ## License
 

package/android/build.gradle

@@ -19,6 +19,7 @@ android {
     
     defaultConfig {
         minSdk 26  // LiteRT-LM requires API 26+
+        consumerProguardFiles 'consumer-rules.pro'
         
         externalNativeBuild {
             cmake {
@@ -66,6 +67,12 @@ android {
             keepDebugSymbols.add("**/*.so")
         }
     }
+    
+    testOptions {
+        unitTests {
+            includeAndroidResources = true
+        }
+    }
 }
 
 repositories {
@@ -89,4 +96,9 @@ dependencies {
     
     // LiteRT-LM Kotlin API
     implementation "com.google.ai.edge.litertlm:litertlm-android:${litertLmVersion}"
+    
+    // Testing Dependencies
+    testImplementation 'junit:junit:4.13.2'
+    testImplementation 'org.robolectric:robolectric:4.11.1'
+    testImplementation 'org.jetbrains.kotlinx:kotlinx-coroutines-test:1.7.3'
 }

package/android/src/main/AndroidManifest.xml

@@ -7,5 +7,10 @@
             android:authorities="${applicationId}.litertlm.init"
             android:exported="false"
             android:initOrder="100" />
+            
+        <!-- Allow dynamic loading of OpenCL for GPU delegate acceleration on Android 12+ -->
+        <uses-native-library
+            android:name="libOpenCL.so"
+            android:required="false" />
     </application>
 </manifest>