@ruvector/cnn 0.1.0

package/README.md

@@ -0,0 +1,301 @@
+# @ruvector/cnn
+
+[![npm version](https://img.shields.io/npm/v/@ruvector/cnn.svg)](https://www.npmjs.com/package/@ruvector/cnn)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+**Turn images into searchable vectors** — runs in browsers, no backend needed.
+
+## What Does This Do?
+
+This package converts images into numbers (called "embeddings") that describe what's in the picture. Similar images produce similar numbers.
+
+**Use it to:**
+- Build "find similar images" features
+- Group photos by what they show
+- Create visual search for products
+- Compare images without AI APIs
+
+```javascript
+import { init, CnnEmbedder } from '@ruvector/cnn';
+
+await init();
+const embedder = new CnnEmbedder();
+
+// Turn an image into numbers
+const numbers = embedder.extract(imagePixels, 224, 224);
+
+// Compare two images (1.0 = identical, 0 = unrelated)
+const similarity = embedder.cosineSimilarity(numbers1, numbers2);
+```
+
+## Why Use This?
+
+| What You Get | Why It Matters |
+|--------------|----------------|
+| Runs in the browser | No server costs, instant results |
+| ~5ms per image | Fast enough for real-time |
+| ~900KB download | Small enough for any website |
+| No API calls | Works offline, no per-image fees |
+| Training included | Teach it your own categories |
+
+## Installation
+
+```bash
+npm install @ruvector/cnn
+```
+
+## How to Use It
+
+### 1. Extract Image Features
+
+```javascript
+import { init, CnnEmbedder } from '@ruvector/cnn';
+
+// Start the engine (do this once)
+await init();
+
+// Create the feature extractor
+const embedder = new CnnEmbedder({
+  embeddingDim: 512,  // How many numbers per image
+  normalize: true      // Make comparisons easier
+});
+
+// Get pixels from an image (RGB, no transparency)
+// Each pixel has 3 values: red, green, blue (0-255)
+const pixels = new Uint8Array(224 * 224 * 3);
+
+// Turn pixels into 512 numbers that describe the image
+const features = embedder.extract(pixels, 224, 224);
+console.log('Got', features.length, 'numbers'); // 512
+```
+
+### 2. Compare Two Images
+
+```javascript
+const features1 = embedder.extract(image1Pixels, 224, 224);
+const features2 = embedder.extract(image2Pixels, 224, 224);
+
+// How similar are they? (1.0 = same, 0 = different, -1 = opposite)
+const score = embedder.cosineSimilarity(features1, features2);
+
+if (score > 0.8) {
+  console.log('These images are very similar!');
+} else if (score > 0.5) {
+  console.log('These images have some things in common');
+} else {
+  console.log('These images are different');
+}
+```
+
+### 3. Find the Most Similar Image
+
+```javascript
+// Your collection of images (already converted to features)
+const catalog = [
+  { name: 'red-shoe.jpg', features: embedder.extract(redShoePixels, 224, 224) },
+  { name: 'blue-bag.jpg', features: embedder.extract(blueBagPixels, 224, 224) },
+  { name: 'red-dress.jpg', features: embedder.extract(redDressPixels, 224, 224) },
+];
+
+// User uploads a photo
+const userPhoto = embedder.extract(uploadedPixels, 224, 224);
+
+// Find the best match
+let bestMatch = null;
+let bestScore = -1;
+
+for (const item of catalog) {
+  const score = embedder.cosineSimilarity(userPhoto, item.features);
+  if (score > bestScore) {
+    bestScore = score;
+    bestMatch = item.name;
+  }
+}
+
+console.log('Best match:', bestMatch, 'Score:', bestScore);
+```
+
+### 4. Get Pixels from a Canvas
+
+```javascript
+// If you have an image in a canvas element
+const canvas = document.getElementById('myCanvas');
+const ctx = canvas.getContext('2d');
+
+// Get the pixel data
+const imageData = ctx.getImageData(0, 0, 224, 224);
+
+// Canvas gives RGBA (4 values per pixel), we need RGB (3 values)
+const rgb = new Uint8Array(224 * 224 * 3);
+for (let i = 0, j = 0; i < imageData.data.length; i += 4, j += 3) {
+  rgb[j] = imageData.data[i];       // Red
+  rgb[j + 1] = imageData.data[i + 1]; // Green
+  rgb[j + 2] = imageData.data[i + 2]; // Blue
+  // Skip alpha (imageData.data[i + 3])
+}
+
+const features = embedder.extract(rgb, 224, 224);
+```
+
+## Training (Teaching It Your Categories)
+
+You can train the model to be better at recognizing your specific images.
+
+### Contrastive Training (SimCLR style)
+
+Show it pairs of images that should match:
+
+```javascript
+import { init, InfoNCELoss, CnnEmbedder } from '@ruvector/cnn';
+
+await init();
+
+const embedder = new CnnEmbedder();
+const trainer = new InfoNCELoss(0.1); // Lower = stricter matching
+
+// Get features for your training pairs
+// Pairs: image1 and image1_different_angle should match
+const image1 = embedder.extract(photo1, 224, 224);
+const image1_alt = embedder.extract(photo1_rotated, 224, 224);
+
+// Pack into one array: [view1s..., view2s...]
+const batch = new Float32Array(2 * 512);
+batch.set(image1, 0);
+batch.set(image1_alt, 512);
+
+const loss = trainer.forward(batch, 1, 512);
+console.log('Loss:', loss); // Lower is better
+```
+
+### Triplet Training
+
+Show it: "A is similar to B, but different from C"
+
+```javascript
+import { init, TripletLoss, CnnEmbedder } from '@ruvector/cnn';
+
+await init();
+
+const embedder = new CnnEmbedder();
+const trainer = new TripletLoss(1.0); // margin
+
+// Anchor: the reference image
+// Positive: should be similar to anchor
+// Negative: should be different from anchor
+const anchor = embedder.extract(redShoePhoto, 224, 224);
+const positive = embedder.extract(redShoePhoto2, 224, 224); // Same shoe
+const negative = embedder.extract(blueBagPhoto, 224, 224);   // Different item
+
+const loss = trainer.forward(
+  new Float32Array(anchor),
+  new Float32Array(positive),
+  new Float32Array(negative),
+  512
+);
+console.log('Loss:', loss);
+```
+
+## Fast Math Operations
+
+If you're building custom features, these are optimized:
+
+```javascript
+import { init, SimdOps, LayerOps } from '@ruvector/cnn';
+
+await init();
+
+// Dot product (sum of element-wise multiplication)
+const a = new Float32Array([1, 2, 3, 4]);
+const b = new Float32Array([5, 6, 7, 8]);
+const result = SimdOps.dotProduct(a, b); // 70
+
+// ReLU: set negative values to 0
+const data = new Float32Array([-1, 0, 1, 7]);
+SimdOps.relu(data); // [0, 0, 1, 7]
+
+// ReLU6: clamp between 0 and 6
+SimdOps.relu6(data); // [0, 0, 1, 6]
+
+// L2 normalize (make length = 1)
+SimdOps.l2Normalize(data);
+```
+
+## Performance
+
+| What | How Long | Notes |
+|------|----------|-------|
+| Extract features (224×224 image) | ~5ms | With SIMD |
+| Compare two images | ~0.01ms | Just math |
+| Training step | ~1ms | Per batch |
+| First load | ~100ms | Downloads WASM |
+
+## Browser Support
+
+Works in all modern browsers with WebAssembly:
+- Chrome 57+
+- Firefox 52+
+- Safari 11+
+- Edge 16+
+
+For best speed, use browsers with SIMD128 support:
+- Chrome 91+
+- Firefox 89+
+- Safari 16.4+
+
+## Troubleshooting
+
+**"init() takes too long"**
+- Normal: First call downloads ~900KB WASM file
+- Fix: Call init() early, before user needs results
+
+**"Images look wrong"**
+- Check: Images must be 224×224 pixels
+- Check: Pixel format is RGB (3 values per pixel, not RGBA)
+- Check: Values are 0-255, not 0-1
+
+**"Similarity scores are all low"**
+- Try: Set `normalize: true` in CnnEmbedder options
+- Check: Are your images actually similar?
+
+## API Reference
+
+### CnnEmbedder
+
+```typescript
+new CnnEmbedder(options?: {
+  embeddingDim?: number;  // Default: 512
+  normalize?: boolean;    // Default: true
+})
+
+.extract(pixels: Uint8Array, width: number, height: number): Float32Array
+.cosineSimilarity(a: Float32Array, b: Float32Array): number
+.embeddingDim: number
+```
+
+### InfoNCELoss
+
+```typescript
+new InfoNCELoss(temperature?: number)  // Default: 0.1
+
+.forward(embeddings: Float32Array, batchSize: number, dim: number): number
+.temperature: number
+```
+
+### TripletLoss
+
+```typescript
+new TripletLoss(margin?: number)  // Default: 1.0
+
+.forward(anchors, positives, negatives: Float32Array, dim: number): number
+.margin: number
+```
+
+## Related Packages
+
+- [`ruvector`](https://www.npmjs.com/package/ruvector) — Core vector database
+- [`@ruvector/attention`](https://www.npmjs.com/package/@ruvector/attention) — AI attention layers
+- [`@ruvector/gnn`](https://www.npmjs.com/package/@ruvector/gnn) — Graph neural networks
+
+## License
+
+MIT OR Apache-2.0

package/index.d.ts

@@ -0,0 +1,164 @@
+/**
+ * @ruvector/cnn - CNN feature extraction for image embeddings
+ */
+
+/**
+ * Initialize the WASM module
+ */
+export function init(): Promise<void>;
+
+/**
+ * Configuration for CNN embedder
+ */
+export interface EmbedderConfig {
+  /** Input image size (square), default: 224 */
+  inputSize?: number;
+  /** Output embedding dimension, default: 512 */
+  embeddingDim?: number;
+  /** L2 normalize embeddings, default: true */
+  normalize?: boolean;
+}
+
+/**
+ * CNN Embedder for extracting image features
+ */
+export class CnnEmbedder {
+  /**
+   * Create a new CNN embedder
+   * @param config - Configuration options
+   */
+  constructor(config?: EmbedderConfig);
+
+  /**
+   * Extract embedding from image data
+   * @param imageData - RGB image data (row-major, no alpha)
+   * @param width - Image width
+   * @param height - Image height
+   * @returns Embedding vector
+   */
+  extract(imageData: Uint8Array, width: number, height: number): Float32Array;
+
+  /**
+   * Compute cosine similarity between two embeddings
+   * @param a - First embedding
+   * @param b - Second embedding
+   * @returns Similarity in [-1, 1]
+   */
+  cosineSimilarity(a: Float32Array, b: Float32Array): number;
+
+  /**
+   * Get the embedding dimension
+   */
+  readonly embeddingDim: number;
+}
+
+/**
+ * InfoNCE loss for contrastive learning (SimCLR style)
+ */
+export class InfoNCELoss {
+  /**
+   * Create InfoNCE loss
+   * @param temperature - Temperature parameter (default: 0.1)
+   */
+  constructor(temperature?: number);
+
+  /**
+   * Compute loss for embedding pairs
+   * @param embeddings - Flattened [2N, D] array where (i, i+N) are positive pairs
+   * @param batchSize - N (number of pairs)
+   * @param dim - D (embedding dimension)
+   * @returns Loss value
+   */
+  forward(embeddings: Float32Array, batchSize: number, dim: number): number;
+
+  /**
+   * Get temperature parameter
+   */
+  readonly temperature: number;
+}
+
+/**
+ * Triplet loss for metric learning
+ */
+export class TripletLoss {
+  /**
+   * Create triplet loss
+   * @param margin - Margin parameter (default: 1.0)
+   */
+  constructor(margin?: number);
+
+  /**
+   * Compute triplet loss
+   * @param anchors - Anchor embeddings [N, D]
+   * @param positives - Positive embeddings [N, D]
+   * @param negatives - Negative embeddings [N, D]
+   * @param dim - Embedding dimension D
+   * @returns Loss value
+   */
+  forward(
+    anchors: Float32Array,
+    positives: Float32Array,
+    negatives: Float32Array,
+    dim: number
+  ): number;
+
+  /**
+   * Get margin parameter
+   */
+  readonly margin: number;
+}
+
+/**
+ * SIMD-optimized operations
+ */
+export namespace SimdOps {
+  /**
+   * Compute dot product of two vectors
+   */
+  function dotProduct(a: Float32Array, b: Float32Array): number;
+
+  /**
+   * Apply ReLU activation in-place
+   */
+  function relu(data: Float32Array): void;
+
+  /**
+   * Apply ReLU6 activation in-place
+   */
+  function relu6(data: Float32Array): void;
+
+  /**
+   * L2 normalize a vector in-place
+   */
+  function l2Normalize(data: Float32Array): void;
+}
+
+/**
+ * Layer operations for building custom networks
+ */
+export namespace LayerOps {
+  /**
+   * Apply batch normalization
+   */
+  function batchNorm(
+    input: Float32Array,
+    gamma: Float32Array,
+    beta: Float32Array,
+    mean: Float32Array,
+    variance: Float32Array,
+    epsilon?: number
+  ): void;
+
+  /**
+   * Apply global average pooling
+   * @param input - Input tensor [C, H, W]
+   * @param channels - Number of channels
+   * @param spatialSize - H * W
+   * @returns Output [C]
+   */
+  function globalAvgPool(
+    input: Float32Array,
+    channels: number,
+    spatialSize: number
+  ): Float32Array;
+}