@coherentglobal/spark-execute-sdk 0.8.3 → 0.8.4

package/README.md

@@ -1,10 +1,51 @@
 # Spark Execute SDK
 
-# About the Project
+Execute [Spark](https://coherent.global/spark/) models both online and offline with a unified API. This SDK seamlessly switches between local WebAssembly execution and remote API calls, giving you the flexibility to run models wherever makes sense for your application.
 
-A Software Development Kit (SDK) that can run either [Spark](https://coherent.global/spark/) model web-assembly or Calling an HTTPS endpoint.
+## Features
 
-# How to's
+- Run models locally using WebAssembly (offline)
+- Call Spark API endpoints (online)
+- Automatic fallback from offline to online on errors
+- Cancel long-running executions mid-flight
+- Automatic memory management with configurable limits
+- Full support for cross-service calls (XCall)
+- Works in Node.js and browser environments
+
+## Quick Start
+
+Install the package:
+
+```bash
+npm install @coherentglobal/spark-execute-sdk
+```
+
+Basic usage:
+
+```js
+// CommonJS
+const Spark = require('@coherentglobal/spark-execute-sdk');
+
+// ES Modules / TypeScript
+import Spark from '@coherentglobal/spark-execute-sdk';
+
+const spark = new Spark({
+  sparkEndpoint: {
+    url: "https://excel.uat.us.coherent.global",
+    tenant: "your-tenant",
+    authType: "public"
+  }
+});
+
+const result = await spark.execute({
+  request_data: { inputs: { Input: 1 } },
+  request_meta: { version_id: "model-uuid" }
+});
+
+await spark.destroy();
+```
+
+# How-to Guides
 
 ## Installation
 
@@ -22,159 +63,443 @@ yarn add @coherentglobal/spark-execute-sdk
 
 ### Configuration
 
-#### Both Online and Offline Config is enabled
+**Execution Modes:**
+- Include both `sparkEndpoint` and `nodegenModels` for **automatic fallback** (offline → online)
+- Include only `sparkEndpoint` for **online-only** execution
+- Include only `nodegenModels` for **offline-only** execution
 
 ```js
 const config = {
-  parallel: 1, // parallel (threads) to be used to run the model in parallel
+  // Online execution (optional - omit for offline-only)
   sparkEndpoint: {
-    url: "https://excel.dev.coherent.global", // Spark Endpoint to be used to run model using Spark SaaS
-    tenant: "tenant", // Spark tenant name
-    authType: public | syntheticKey | bearerToken, // public endpoint don't require token / synthetic key
-    // If syntheticKey
-    syntheticKey: "apiKey" or generateSyntheticKey(), // value or a func - func call every time making API call, user will do cache
-    // If bearerToken
-    bearerToken: generateToken(), //a func - func call every time making API call, user will do cache
+    url: "https://excel.uat.us.coherent.global",
+    tenant: "tenant",
+    authType: "public" | "syntheticKey" | "bearerToken",
+
+    // Provide value OR function (called per request - you handle caching)
+    syntheticKey: "apiKey" || generateSyntheticKey(),
+    bearerToken: "token" || generateToken(),
   },
+
+  // Offline execution (optional - omit or use [] for online-only)
   nodegenModels: [
     {
       versionId: "uuid",
-      type: "base64",
-      binary: Blob | Base64 | BinaryUnit | Func(versionId), // a binary zipped file or a function the will get the zipped file
-      preventCleanup: true | false, // prevent model from being unload (default: false)
-      instance: int, // number of model instance (default: 1)
-      metadata: {
-        // Spark Model Metadata
+      type: "base64", // Always "base64"
+      binary: string | Blob | Buffer | ArrayBuffer | Function, // See below
+      preventCleanup: false, // Keep in memory even when limit reached
+      replica: 1, // Number of concurrent worker instances
+      metaData: {
+        EngineInformation: {
+          ProductName: "folder-name",
+          ServiceName: "service-name",
+          VersionId: "uuid"
+        }
       }
     }
-  ]
-}
+  ],
+
+  // Advanced options
+  offlineMaxService: 20, // Max models in memory before LRU cleanup
+  timeout: 60000 // Request timeout (ms)
+};
 ```
 
-#### Online Only Configuration
+### Understanding the `binary` Configuration
+
+The `binary` field accepts different formats depending on your environment.
+
+| Environment | Accepted Formats |
+|-------------|------------------|
+| **Browser** | Base64 string, `Blob`, `ArrayBuffer`, Function |
+| **Node.js** | Base64 string, `Buffer`, `ArrayBuffer`, Function |
+
+> **Important:** The `type` field should always be `"base64"` regardless of actual binary format.
 
+#### Common Patterns
+
+**Static values:**
 ```js
-const config = {
-  sparkEndpoint: {
-    url: "https://excel.dev.coherent.global", // Spark Endpoint to be used to run model using Spark SaaS
-    tenant: "tenant", // Spark tenant name
-    authType: public | syntheticKey | bearerToken, // public endpoint don't require token / synthetic key
-    // If syntheticKey
-    syntheticKey: "apiKey" || generateSyntheticKey(), // value or a func - func call every time making API call, user will do cache
-    // If bearerToken
-    bearerToken: "token" || generateToken(), //a func - func call every time making API call, user will do cache
-  },
-  nodegenModels: [],
-};
-```
+// Base64 string (most common)
+const fs = require('fs');
+binary: fs.readFileSync('./model.zip', 'base64')
+
+// Buffer (Node.js)
+binary: fs.readFileSync('./model.zip')
+
+// ArrayBuffer (fetch in browser/Node.js)
+const response = await fetch('/models/model.zip');
+binary: await response.arrayBuffer()
 
-#### Offline Only Configuration
+// Blob (browser file upload)
+binary: new Blob([zipData], { type: 'application/zip' })
+```
 
+**Dynamic loading (function):**
 ```js
-const config = {
-  nodegenModels: [
-    {
-      versionId: "uuid",
-      type: "base64",
-      binary: Blob | Base64 | BinaryUnit | Func(versionId), // a binary zipped file or a function the will get the zipped file
-      preventCleanup: true | false, // prevent model from being unload (default: false)
-      replica: int, // number of model instance (default: 1)
-      metadata: {
-        // Spark Model Metadata
-      },
-    },
-  ],
-};
+// Lazy load from filesystem
+binary: (versionId) => require('fs').readFileSync(`./models/${versionId}.zip`, 'base64')
+
+// Fetch from remote storage
+binary: async (versionId) => {
+  const response = await fetch(`https://storage.example.com/models/${versionId}.zip`);
+  return await response.arrayBuffer();
+}
+
+// With caching
+const modelCache = new Map();
+binary: async (versionId) => {
+  if (!modelCache.has(versionId)) {
+    modelCache.set(versionId, await fetchModelFromS3(versionId));
+  }
+  return modelCache.get(versionId);
+}
 ```
 
-## Methods
+## API Reference
 
-### Initialize Spark Execute SDK
+### Initialization
 
-Instantiate the Model Class.
+Create a Spark instance with your configuration:
 
 ```js
 const spark = new Spark(config);
 ```
 
-### Execute
+### execute(input, version_id?)
 
-### (async) execute(input)
-
-Perform the calculation. The sdk will use the request_meta.version_uuid of input argument to locate the model
+Execute a model using the provided input payload.
 
 ```js
-const response = await spark.execute(input).catch((err) => {
-  // Do something here
-});
+// Basic usage
+const response = await spark.execute(input);
+
+// With explicit version ID
+const response = await spark.execute(input, 'model-uuid-here');
 ```
 
-or
+> **Tip:** The `version_id` parameter is optional if already in `request_meta`. See [Error Handling](#error-handling) below.
+
+### executeWithCancellationToken(input, version_id?)
+
+Execute a model with the ability to cancel the operation mid-execution. Useful for long-running calculations that users may want to cancel.
 
 ```js
-const response = await spark.execute(input, version_id).catch((err) => {
-  // Do something here
-});
+const execution = spark.executeWithCancellationToken(input);
+
+// Cancel anytime before completion
+execution.cancellationToken.cancel();
+
+execution.response
+  .then((result) => {
+    console.log('Completed:', result);
+  })
+  .catch((err) => {
+    if (err instanceof Spark.WasmRunnerErrors.ExecuteCancelled) {
+      console.log('User cancelled the execution');
+    } else {
+      console.error('Execution failed:', err);
+    }
+  });
 ```
 
-### executeWithCancellationToken(input)
+> **Important:** Cancellation is best-effort. The underlying WebAssembly execution may continue running in its worker thread, but the result will be discarded and an `ExecuteCancelled` error will be thrown. The worker is cleaned up after completion.
+
+### destroy()
 
-Perform the calculation with cancellationToken. The sdk will use the request_meta.version_uuid of input argument to locate the model
+Clean up all resources, unload models, and terminate worker threads. Call this when you're done with a Spark instance to prevent memory leaks.
 
 ```js
-const execute = spark.executeWithCancellationToken(input)
+await spark.destroy();
+```
 
-// Cancel the execution (best effort)
-execute.cancellationToken.cancel()
+> **Critical:** Always call `destroy()` when finished, especially in browsers or long-running Node.js apps. Forgetting this can lead to memory leaks as WebAssembly modules and workers won't be garbage collected.
 
-execute.response.then((result) => {
-  // Do something here
-})
-.catch((err) => {
-  // Do something here
-  // If error is instance of Spark.WasmRunnerErrors.ExecuteCancelled, meaning the request has been cancelled
-});
-```
+### Error Handling
 
-or
+Specific error types accessible via `Spark.WasmRunnerErrors`:
 
 ```js
-const execute = spark.executeWithCancellationToken(input)
+try {
+  const response = await spark.execute(input);
+} catch (err) {
+  if (err instanceof Spark.WasmRunnerErrors.MissingModelError) {
+    console.error('Model not found:', err.message);
+  } else if (err instanceof Spark.WasmRunnerErrors.ExecuteCancelled) {
+    console.error('Execution was cancelled');
+  } else if (err instanceof Spark.WasmRunnerErrors.UnauthorizedError) {
+    console.error('Authentication failed');
+  } else if (err instanceof Spark.WasmRunnerErrors.BadRequestError) {
+    console.error('Bad request:', err.message);
+  } else {
+    console.error('Unknown error:', err);
+  }
+}
+```
 
-// Cancel the execution (best effort)
-execute.cancellationToken.cancel()
+**Available Error Types:**
+- `MissingModelError` - Model not found
+- `ExecuteCancelled` - Execution was cancelled via cancellation token
+- `UnauthorizedError` - Authentication failed (invalid token/key)
+- `BadRequestError` - Invalid request or SDK not initialized properly
 
-execute.response.then((result) => {
-  // Do something here
-})
-.catch((err) => {
-  // Do something here
-  // If error is instance of Spark.WasmRunnerErrors.ExecuteCancelled, meaning the request has been cancelled
-});
+### Error Response Structure
+
+When an online API request fails, the Spark API returns structured error responses:
+
+```json
+{
+  "error": {
+    "status": 401,
+    "message": "Unauthorized: Invalid API key",
+    "code": "UNAUTHORIZED"
+  }
+}
 ```
 
-Note: The second parameter (version_id) is optional if version_id is already included on request_meta
+Common HTTP status codes:
+- **400 Bad Request** - Invalid input data or malformed request
+- **401 Unauthorized** - Missing or invalid authentication credentials
+- **404 Not Found** - Model or endpoint not found
+- **500 Internal Server Error** - Server-side error during execution
+- **503 Service Unavailable** - Service temporarily unavailable
 
-### Sample Input
+### Input Structure
 
 ```json
 {
-  "request_data": {
-    "inputs": {
-      "Input": 1
-    }
-  },
+  "request_data": { "inputs": { "Input": 1 } },
   "request_meta": {
     "version_id": "<model id>",
     "transaction_date": "2022-09-19T04:17:17.142Z",
-    "call_purpose": "Spark - API Tester",
-    "source_system": "",
-    "correlation_id": "",
-    "requested_output": ""
+    "call_purpose": "Spark - API Tester"
+    // correlation_id, source_system, requested_output (optional)
   }
 }
 ```
 
-### DEMO
+## Platform-Specific Usage
+
+### Node.js
+
+```js
+// CommonJS
+const Spark = require('@coherentglobal/spark-execute-sdk');
+
+// ES Modules / TypeScript
+import Spark from '@coherentglobal/spark-execute-sdk';
+
+const fs = require('fs');
+const spark = new Spark({
+  nodegenModels: [{
+    versionId: "model-uuid",
+    binary: fs.readFileSync('./model.zip', 'base64'),
+    metaData: { /* ... */ }
+  }]
+});
+
+const response = await spark.execute(input);
+await spark.destroy();
+```
+
+### Browser
+
+```html
+<script src="node_modules/@coherentglobal/spark-execute-sdk/dist/browser.js"></script>
+<script>
+  const config = {
+    nodegenModels: [{
+      versionId: "model-uuid",
+      binary: base64EncodedModel,
+      metaData: { /* ... */ }
+    }]
+  };
+
+  const spark = new Spark(config);
+
+  spark.execute(input)
+    .then(response => console.log('Result:', response))
+    .catch(err => console.error('Error:', err))
+    .finally(() => spark.destroy()); // Always cleanup!
+</script>
+```
+
+## Troubleshooting
+
+### Common Issues
+
+#### "Model not found" Error
+
+**Problem:** SDK throws `MissingModelError` when trying to execute.
+
+**Solutions:**
+- Verify that `versionId` in your input matches a configured model in `nodegenModels`
+- Check that `request_meta.version_id` is correctly specified in your input
+- Check if the model binary is loaded (check browser console for loading errors)
+
+#### "Binary corrupted" or Initialization Errors
+
+**Problem:** Model fails to initialize or throws corruption errors.
+
+**Solutions:**
+- Verify the model binary is valid and not truncated during download
+- Check that the base64 encoding is correct (no line breaks or extra characters)
+- Try re-downloading the model from Spark
+- Ensure the binary isn't corrupted (use `type: "base64"` for base64-encoded zips)
+
+#### "Worker terminated unexpectedly"
+
+**Problem:** WebAssembly worker crashes during execution.
+
+**Solutions:**
+- Check browser console for memory errors
+- Reduce the number of concurrent replicas
+- Ensure the model isn't too large for the available memory
+- Try increasing timeout value if the model is computationally intensive
+
+#### ExecuteCancelled Thrown Instantly
+
+**Problem:** Cancellation error occurs immediately even without calling cancel.
+
+**Solutions:**
+- Check if cancellation token is being reused from a previous execution
+- Create a new execution instance for each call to `executeWithCancellationToken`
+- Verify that cancel() isn't being called earlier in your code flow
+
+#### Online Fallback Not Triggered
+
+**Problem:** SDK doesn't fall back to online execution when offline fails.
+
+**Solutions:**
+- Ensure both `sparkEndpoint` and `nodegenModels` are configured
+- Check that the offline error is not being caught before fallback occurs
+- Verify network connectivity for online execution
+- Check authentication credentials for the online endpoint
+
+#### Memory Leaks in Long-Running Applications
+
+**Problem:** Application memory usage keeps increasing over time.
+
+**Solutions:**
+- Always call `spark.destroy()` when finished with an instance
+- Don't create multiple Spark instances unnecessarily
+- Monitor the number of loaded models (check against `offlineMaxService` limit)
+- Use `preventCleanup: false` for models that don't need to persist
+
+## Memory Usage & Cleanup
+
+### How Model Cleanup Works
+
+The SDK automatically manages memory using LRU (Least Recently Used) eviction:
+
+- **offlineMaxService** (default: 20): Maximum models kept in memory
+- **preventCleanup**: Set to `true` to prevent a model from being unloaded
+- When the limit is reached, the least recently used model (without `preventCleanup`) is unloaded
+
+### Worker Thread Management
+
+Each model uses dedicated worker threads for parallel execution.
+
+#### Concurrency Model
+
+- Each model creates a pool of isolated worker threads based on the `replica` count (default: 1)
+- Incoming execution requests are distributed across available workers in round-robin fashion
+- Each worker runs in its own thread with its own WebAssembly instance—no shared state
+- Model are lazy-loaded on first run
+- Higher `replica` counts increase initial execution time (more workers to load) but improve throughput for concurrent requests
+
+#### Configuration Guide
+
+| Use Case | `replica` | `preventCleanup` |
+|----------|-----------|------------------|
+| High-frequency model** | 4-8 | `true` |
+| Occasional use | 1 | `false` |
+| Critical path | 2-4 | `true` |
+| Background jobs | 1-2 | `false` |
+
+#### Memory Considerations
+
+Each worker instance loads the full WebAssembly module into memory:
+
+- **Memory per worker** = Model required memory (~150-300 MB)
+- **Total memory** = (Number of models) × (Replicas per model) × (Memory per worker)
+- **Example**: 10 models × 3 replicas × 150 MB = ~4500 MB minimum
+
+**NOTE**: We recommend keeping `replica` at 2 or fewer, unless you have a very specific performance requirement that justifies higher concurrency.
+
+### Cleanup Best Practices
+
+**Always call destroy():**
+```js
+const spark = new Spark(config);
+try {
+  await spark.execute(input);
+} finally {
+  await spark.destroy();
+}
+```
+
+**For long-running apps:**
+- Set `offlineMaxService` based on available memory
+- Use `preventCleanup: true` sparingly
+- Call `destroy()` before page unload (browsers)
+
+## Known Limitations
+
+### Browser Memory Constraints
+
+WebAssembly in browsers has strict memory limits:
+
+| Constraint | Limit | Impact |
+|------------|-------|--------|
+| **WebAssembly memory** | ~1-2 GB (browser dependent, varied between platform) | Large models may fail to load |
+| **Total tab memory** | ~2-4 GB (browser dependent) | Multiple models + high replicas can hit browser limits |
+
+**Recommendations for Browser:**
+- Limit `replica` count to 2-4 for large / highly complex models
+- Monitor memory usage with browser DevTools
+- Consider online-only mode for very large models
+
+### Node.js Worker Thread Cost
+
+Each worker thread in Node.js has overhead:
+
+- **Startup time**: 50-200ms per worker
+- **Memory overhead**: ~150-300 MB per thread (V8 isolate + runtime)
+- **Efficient thread limit**: Typically 50-200 threads (OS dependent)
+
+**For Node.js applications:**
+- Don't create hundreds of workers (replica count × model count should be reasonable)
+- Workers are lazy-loaded but all replicas for a model are initialized on first execution
+
+### Cross-Service Call (XCall) Depth
+
+**Recursion Limit:**
+- Maximum XCall depth: ~10-20 levels (implementation dependent)
+- Deeply nested XCalls increase memory usage and execution time
+- Stack overflow possible with excessive recursion
+
+**Best Practice:**
+- Design models to minimize XCall depth
+- Avoid circular dependencies between models
+- Consider flattening deeply nested call chains
+
+### Model Loading Failures
+
+**Common offline mode failures:**
+
+| Scenario | Browser | Node.js |
+|----------|---------|---------|
+| 10+ concurrent replicas | May hit memory limits | Works but slow startup |
+| Complex XCall graphs | Stack overflow possible | More headroom |
+
+**Mitigation:**
+- Use online mode for exceptionally large models
+- Reduce `replica` count if initialization fails
+- Monitor memory consumption during development
+
+## Live Demo
+
+See the SDK in action with this interactive CodeSandbox example:
 
-DEMO LINK- https://codesandbox.io/s/autumn-hill-1342ln?file=/index.html
+**[View Demo →](https://codesandbox.io/s/autumn-hill-1342ln?file=/index.html)**