mini-jstorch 1.3.2 → 1.4.2

package/README.md

@@ -1,39 +1,42 @@
 # Mini-JSTorch
 
-A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices Inspired by PyTorch.
+A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices, inspired by PyTorch.
 
 ## Overview
 
-Mini-JSTorch is a lightweight, high-performance JavaScript library for building neural networks that runs efficiently in both frontend and backend environments, including low-end devices. The library enables experimentation and learning in AI without compromising stability, accuracy, or training reliability.
+Mini-JSTorch is a high-performance, minimalist JavaScript library for building neural networks. It runs efficiently in both frontend and backend environments, including low-end devices. The library enables quick experimentation and learning in AI without compromising stability, accuracy, or training reliability.
 
-This release, **version 1.2.3**, is a just some fix a typo and delete not used files.
+This release, **version 1.4.3**, we introduces **learning rate schedulers**, improved testing/demo templates, and other minor enhancements.
 
 ---
 
-## Features Overview
-
-- Full **Conv2D support** with forward and backward operations.
-- **Tensor operations** now support broadcasting and reshaping.
-- Added new activations:  `LeakyReLU`, `GELU`.
-- Optimizers: `Adam` and `SGD` fully integrated with gradient updates.
-- Dropout layer added for regularization.
-- Advanced utilities for tensor manipulation: `flatten`, `stack`, `concat`, `eye`, `reshape`.
-- End-to-end training and prediction workflow now fully tested.
-- Save and load model functionality included for seamless persistence.
-- Optimized performance for both frontend and backend usage.
-- Maintained backward compatibility for previous core layers and activations.
-- folder tests files template that you can use it before make your models.
+## Feature Highlights
+
+- **Learning Rate Schedulers:** New `StepLR` and `LambdaLR` for dynamic optimizer learning rate adjustment.
+- **Full Conv2D support:** Forward and backward operations for convolutional layers.
+- **Tensor operations:** Broadcasting, reshaping, and reduction utilities.
+- **Advanced Activations:** Includes `LeakyReLU`, `GELU`, `Mish`, `SiLU`, `ELU`, and more.
+- **Optimizers:** `Adam` and `SGD` with gradient updates.
+- **Dropout Layer:** For regularization during training.
+- **BatchNorm2D:** For stable training in convolutional models.
+- **Tensor Manipulation:** Utilities like `flatten`, `stack`, `concat`, `eye`, `reshape`.
+- **Model Save & Load:** Easy persistence and restore of models.
+- **Test/Demo Templates:** The `tests/` folder provides ready-to-run examples for model building and feature usage.
+- **Performance Optimized:** Suitable for both frontend and backend usage.
+- **Backward Compatibility:** Maintained for core layers and activations.
 
 ---
 
-## Features
+## Core Features
 
-- **Core Layers**: Linear, Conv2D  
-- **Activations**: ReLU, Sigmoid, Tanh, LeakyReLU, GELU  
-- **Loss Functions**: MSELoss, CrossEntropyLoss  
-- **Optimizers**: Adam, SGD  
-- **Utilities**: zeros, randomMatrix, softmax, crossEntropy, dot, addMatrices, reshape, stack, flatten, eye, concat  
-- **Model Container**: Sequential (for stacking layers with forward/backward passes)  
+- **Layers:** Linear, Conv2D  
+- **Activations:** ReLU, Sigmoid, Tanh, LeakyReLU, GELU, Mish, SiLU, ELU  
+- **Loss Functions:** MSELoss, CrossEntropyLoss
+- **Optimizers:** Adam, SGD  
+- **Schedulers:** StepLR, LambdaLR  
+- **Regularization:** Dropout, BatchNorm2D  
+- **Utilities:** zeros, randomMatrix, softmax, crossEntropy, dot, addMatrices, reshape, stack, flatten, eye, concat  
+- **Model Container:** Sequential (for stacking layers with forward/backward passes)  
 
 ---
 
@@ -49,7 +52,7 @@ npm install mini-jstorch
 ## Quick Start Example
 
 ```javascript
-import { Sequential, Linear, ReLU, Sigmoid, CrossEntropyLoss, Adam } from './src/MainEngine.js';
+import { Sequential, Linear, ReLU, Sigmoid, CrossEntropyLoss, Adam, StepLR } from 'mini-jstorch';
 
 // Build model
 const model = new Sequential([
@@ -70,6 +73,7 @@ const Y = [
 // Loss & optimizer
 const lossFn = new CrossEntropyLoss();
 const optimizer = new Adam(model.parameters(), 0.1);
+const scheduler = new StepLR(optimizer, 20, 0.5); // Halve LR every 20 epochs
 
 // Training loop
 for (let epoch = 1; epoch <= 100; epoch++) {
@@ -78,7 +82,8 @@ for (let epoch = 1; epoch <= 100; epoch++) {
   const gradLoss = lossFn.backward();
   model.backward(gradLoss);
   optimizer.step();
-  if (epoch % 20 === 0) console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}`);
+  scheduler.step();
+  if (epoch % 20 === 0) console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}, LR: ${optimizer.lr.toFixed(4)}`);
 }
 
 // Prediction
@@ -94,7 +99,7 @@ predTest.forEach((p,i) => {
 ## Save & Load Models
 
 ```javascript
-import { saveModel, loadModel } from 'mini-jstorch';
+import { saveModel, loadModel, Sequential } from './src/MainEngine.js';
 
 const json = saveModel(model);
 const model2 = new Sequential([...]); // same architecture
@@ -103,7 +108,22 @@ loadModel(model2, json);
 
 ---
 
+## Demos & Testing
+
+Check the `tests/` directory for ready-to-run demos:
+- **tests/MakeModel.js:** Build and run a simple neural network.
+- **tests/scheduler.js:** Experiment with learning rate schedulers.
+- Add your own scripts for quick prototyping!
+
+```bash
+node tests/MakeModel.js
+node tests/scheduler.js
+```
+
+---
+
 ## Intended Use Cases
+
  - Rapid prototyping of neural networks in frontend and backend.
  - Learning and teaching foundational neural network concepts.
  - Experimentation on low-end devices or mobile browsers.
@@ -114,13 +134,3 @@ loadModel(model2, json);
 # License
 
 **MIT © 2025 Rizal**
-
----
-
-## Facts
-
-- **This module is implemented entirely in pure JavaScript.**
-- **The `Dummy` folder contains modules used for development, testing, and debugging before integration into the main engine.**
-- **files startup.cpu is actually an some random files lol.**
-- **This module was created by a `single` developer.**
-- **You can join to the `mjs-group` Organization on my profile!**

package/index.js

@@ -1,90 +1,2 @@
-// =====================================================
-// FILE: index.js - Main entry point
-// =====================================================
-
-// Import all modules
-import { Tensor } from './src/tensor.js';
-import { 
-    Linear, Dense, Conv2d, MaxPool2d, AvgPool2d, Flatten, Dropout, 
-    BatchNorm2d, ReLU, Sigmoid, Tanh, LeakyReLU, ELU, Softmax, Sequential 
-} from './src/layers.js';
-import { MSELoss, CrossEntropyLoss, BCELoss } from './src/loss.js';
-import { SGD, Adam, RMSprop } from './src/optimizers.js';
-import { Model, Trainer, models, optimizers, losses, layers, tensors, utils } from './src/model.js';
-
-// Export everything
-export {
-    // Core classes
-    Tensor,
-    Model,
-    Trainer,
-    
-    // Layer classes
-    Linear,
-    Dense,
-    Conv2d,
-    MaxPool2d,
-    AvgPool2d,
-    Flatten,
-    Dropout,
-    BatchNorm2d,
-    ReLU,
-    Sigmoid,
-    Tanh,
-    LeakyReLU,
-    ELU,
-    Softmax,
-    Sequential,
-    
-    // Loss classes
-    MSELoss,
-    CrossEntropyLoss,
-    BCELoss,
-    
-    // Optimizer classes
-    SGD,
-    Adam,
-    RMSprop,
-    
-    // Factory functions
-    models,
-    optimizers,
-    losses,
-    layers,
-    tensors,
-    utils
-};
-
-// Default export
-export default {
-    Tensor,
-    Model,
-    Trainer,
-    Linear,
-    Dense,
-    Conv2d,
-    MaxPool2d,
-    AvgPool2d,
-    Flatten,
-    Dropout,
-    BatchNorm2d,
-    ReLU,
-    Sigmoid,
-    Tanh,
-    LeakyReLU,
-    ELU,
-    Softmax,
-    Sequential,
-    MSELoss,
-    CrossEntropyLoss,
-    BCELoss,
-    SGD,
-    Adam,
-    RMSprop,
-    models,
-    optimizers,
-    losses,
-    layers,
-    tensors,
-    utils
-};
+// package root
+export * from "./src/MainEngine.js";

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mini-jstorch",
-  "version": "1.3.2",
+  "version": "1.4.2",
   "type": "module", 
   "description": "A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices Inspired by PyTorch.",
   "main": "index.js",

package/src/MainEngine.js

@@ -1,8 +1,7 @@
 
-// MINI JSTORCH ENGINE
-// LICENSE: MIT (C) Rizal 2025
-// 1.2.3 ENGINE VERSIONS
-// IMPORTANT: CORE ENGINE DO NOT EDIT THIS FILES VERY SENSITIVE IT WILL CRASHING YOUR ENGINE SYSTEMS!
+// OFFICIAL MINI-JSTORCH ENGINE
+// LICENSED UNDER MIT LICENSE
+// MIT (C) Rizal 2025
 
 // ---------------------- Utilities ----------------------
 function zeros(rows, cols) { return Array.from({length:rows},()=>Array(cols).fill(0)); }
@@ -225,6 +224,46 @@ export class Adam{
     }
 }
 
+// ---------------------- Learning Rate Schedulers ----------------------
+export class StepLR {
+    constructor(optimizer, step_size, gamma=1.0) {
+        this.optimizer = optimizer;
+        this.step_size = step_size;
+        this.gamma = gamma;
+        this.last_epoch = 0;
+        this.base_lr = optimizer.lr;
+    }
+
+    step() {
+        this.last_epoch += 1;
+        if (this.last_epoch % this.step_size === 0) {
+            this.optimizer.lr *= this.gamma;
+        }
+    }
+
+    get_lr() {
+        return this.optimizer.lr;
+    }
+}
+
+export class LambdaLR {
+    constructor(optimizer, lr_lambda) {
+        this.optimizer = optimizer;
+        this.lr_lambda = lr_lambda;
+        this.last_epoch = 0;
+        this.base_lr = optimizer.lr;
+    }
+
+    step() {
+        this.last_epoch += 1;
+        this.optimizer.lr = this.base_lr * this.lr_lambda(this.last_epoch);
+    }
+
+    get_lr() {
+        return this.optimizer.lr;
+    }
+}
+
 // ---------------------- ELU Activation ----------------------
 export class ELU {
     constructor(alpha=1.0) {
@@ -551,7 +590,7 @@ export function stack(tensors){ return tensors.map(t=>t.data); }
 export function eye(n){ return Array.from({length:n},(_,i)=>Array.from({length:n},(_,j)=>i===j?1:0)); }
 export function concat(a,b,axis=0){ /* concat along axis */ if(axis===0) return [...a,...b]; if(axis===1) return a.map((row,i)=>[...row,...b[i]]); }
 export function reshape(tensor, rows, cols) {
-    let flat = tensor.data.flat(); // flatten dulu
+    let flat = tensor.data.flat(); // flatten
     if(flat.length < rows*cols) throw new Error("reshape size mismatch");
     const out = Array.from({length: rows}, (_, i) =>
         flat.slice(i*cols, i*cols + cols)

package/tests/MakeModel.js

@@ -1,570 +1,38 @@
-// mini-jstorch-test.js - Fully Synchronous Test Suite
-import MiniJSTorch, { Tensor } from '../testdummy.js';
+// Example: Build and run a simple neural network model using mini-jstorch
 
-// Initialize test framework
-const torch = new MiniJSTorch();
-let testsPassed = 0;
-let testsFailed = 0;
+import { Sequential, Linear, ReLU, MSELoss, SGD, Tensor } from "../src/MainEngine.js";
 
-function assert(condition, message) {
-  if (!condition) {
-    console.error(`❌ FAIL: ${message}`);
-    testsFailed++;
-  } else {
-    console.log(`✅ PASS: ${message}`);
-    testsPassed++;
-  }
-}
+// Create dummy input and target data
+const input = new Tensor([[0.5, -1.0], [1.5, 2.0]]); // shape: [2,2]
+const target = new Tensor([[1.0, 0.0], [0.0, 1.0]]); // shape: [2,2]
 
-function testSuite(name) {
-  console.log(`\n🧪 Testing ${name}...`);
-}
+// Build a simple model: Linear -> ReLU -> Linear
+const model = new Sequential([
+    new Linear(2, 4),
+    new ReLU(),
+    new Linear(4, 2)
+]);
 
-function summarize() {
-  console.log(`\n📊 Test Summary:`);
-  console.log(`✅ Passed: ${testsPassed}`);
-  console.log(`❌ Failed: ${testsFailed}`);
-  console.log(`📈 Success Rate: ${((testsPassed / (testsPassed + testsFailed)) * 100).toFixed(1)}%`);
-}
+const criterion = new MSELoss();
+const optimizer = new SGD(model.parameters(), 0.01);
 
-// ====================== TENSOR OPERATIONS TESTS ======================
-testSuite("Tensor Operations");
+// Forward pass
+const output = model.forward(input.data);
+console.log("Model output:", output);
 
-function testTensorOperations() {
-  // Basic tensor creation
-  const t1 = new Tensor([1, 2, 3, 4], [2, 2]);
-  assert(t1.shape[0] === 2 && t1.shape[1] === 2, "Tensor shape creation");
-  assert(t1.data[0] === 1 && t1.data[3] === 4, "Tensor data initialization");
+// Compute loss
+const loss = criterion.forward(output, target.data);
+console.log("Loss:", loss);
 
-  // Addition
-  const t2 = new Tensor([5, 6, 7, 8], [2, 2]);
-  const sum = t1.add(t2);
-  assert(sum.data[0] === 6 && sum.data[3] === 12, "Tensor addition");
+// Backward pass
+const grad = criterion.backward();
+model.backward(grad);
 
-  // Subtraction
-  const sub = t1.sub(t2);
-  assert(sub.data[0] === -4 && sub.data[3] === -4, "Tensor subtraction");
+// Optimizer step
+optimizer.step();
+console.log("Parameters updated!");
 
-  // Multiplication
-  const mul = t1.mul(t2);
-  assert(mul.data[0] === 5 && mul.data[3] === 32, "Tensor multiplication");
-
-  // Division
-  const div = t2.div(t1);
-  assert(Math.abs(div.data[0] - 5) < 0.001, "Tensor division");
-
-  // Matrix multiplication
-  const matmul = t1.matmul(t2);
-  assert(Math.abs(matmul.data[0] - 19) < 0.001, "Matrix multiplication");
-
-  // Activation functions
-  const relu = t1.relu();
-  assert(relu.data[0] === 1 && relu.data[1] === 2, "ReLU activation");
-  
-  const sigmoid = new Tensor([-1, 0, 1]).sigmoid();
-  assert(sigmoid.data[0] < 0.5 && sigmoid.data[1] === 0.5 && sigmoid.data[2] > 0.5, "Sigmoid activation");
-
-  const tanh = new Tensor([-1, 0, 1]).tanh();
-  assert(tanh.data[0] < 0 && tanh.data[1] === 0 && tanh.data[2] > 0, "Tanh activation");
-
-  // Reshape
-  const reshaped = t1.reshape([4]);
-  assert(reshaped.shape[0] === 4 && reshaped.data[2] === 3, "Tensor reshape");
-
-  // Transpose
-  const transposed = t1.transpose();
-  assert(transposed.shape[0] === 2 && transposed.shape[1] === 2, "Tensor transpose");
-  assert(transposed.data[1] === 3 && transposed.data[2] === 2, "Tensor transpose values");
-
-  // Sum and mean
-  const sumAll = t1.sum();
-  assert(sumAll.data[0] === 10, "Tensor sum all");
-
-  const meanAll = t1.mean();
-  assert(Math.abs(meanAll.data[0] - 2.5) < 0.1, "Tensor mean all");
-
-  // Memory management
-  const memoryStats = Tensor.memoryTracker.getStats();
-  assert(memoryStats.tensorCount > 0, "Memory tracking");
-}
-
-// ====================== LAYERS TESTS ======================
-testSuite("Layers");
-
-function testLayers() {
-  // Linear layer
-  const linear = torch.layers.linear(3, 2);
-  const input = new Tensor([1, 2, 3], [1, 3]);
-  const output = linear.forward(input);
-  assert(output.shape[0] === 1 && output.shape[1] === 2, "Linear layer forward pass");
-
-  // Conv2D layer
-  const conv = torch.layers.conv2d(3, 16, 3);
-  const convInput = new Tensor(new Float32Array(1 * 3 * 32 * 32), [1, 3, 32, 32]);
-  const convOutput = conv.forward(convInput);
-  assert(convOutput.shape[0] === 1 && convOutput.shape[1] === 16, "Conv2D layer forward pass");
-
-  // Activation layers
-  const reluLayer = torch.layers.relu();
-  const reluOutput = reluLayer.forward(new Tensor([-1, 0, 1]));
-  assert(reluOutput.data[0] === 0 && reluOutput.data[2] === 1, "ReLU layer");
-
-  const sigmoidLayer = torch.layers.sigmoid();
-  const sigmoidOutput = sigmoidLayer.forward(new Tensor([-1, 0, 1]));
-  assert(sigmoidOutput.data[0] < 0.5 && sigmoidOutput.data[1] === 0.5, "Sigmoid layer");
-
-  const tanhLayer = torch.layers.tanh();
-  const tanhOutput = tanhLayer.forward(new Tensor([-1, 0, 1]));
-  assert(tanhOutput.data[0] < 0 && tanhOutput.data[1] === 0, "Tanh layer");
-
-  // Dropout layer
-  const dropout = torch.layers.dropout(0.5);
-  dropout.train();
-  const dropoutInput = new Tensor([1, 2, 3, 4]);
-  const dropoutOutput = dropout.forward(dropoutInput);
-  assert(dropoutOutput.data.some(val => val === 0), "Dropout layer in train mode");
-
-  dropout.eval();
-  const dropoutOutputEval = dropout.forward(dropoutInput);
-  assert(dropoutOutputEval.data[0] === 1 && dropoutOutputEval.data[3] === 4, "Dropout layer in eval mode");
-
-  // BatchNorm2D
-  const bn = torch.layers.batchNorm2d(16);
-  bn.train();
-  const bnInput = new Tensor(new Float32Array(2 * 16 * 8 * 8), [2, 16, 8, 8]);
-  const bnOutput = bn.forward(bnInput);
-  assert(bnOutput.shape[0] === 2 && bnOutput.shape[1] === 16, "BatchNorm2D forward pass");
-
-  // LSTM
-  const lstm = torch.layers.lstm(10, 20, 2);
-  const lstmInput = new Tensor(new Float32Array(5 * 3 * 10), [5, 3, 10]);
-  const lstmOutput = lstm.forward(lstmInput);
-  assert(lstmOutput.output.shape[0] === 5 && lstmOutput.output.shape[2] === 20, "LSTM forward pass");
-
-  // MultiHeadAttention
-  const attn = torch.layers.attention(64, 8);
-  const attnInput = new Tensor(new Float32Array(2 * 10 * 64), [2, 10, 64]);
-  const attnOutput = attn.forward(attnInput, attnInput, attnInput);
-  assert(attnOutput.shape[0] === 2 && attnOutput.shape[2] === 64, "MultiHeadAttention forward pass");
-
-  // Transformer
-  const transformer = torch.layers.transformer(64, 8, 2);
-  const transformerOutput = transformer.forward(attnInput);
-  assert(transformerOutput.shape[0] === 2 && transformerOutput.shape[2] === 64, "Transformer forward pass");
-
-  // LayerNorm
-  const layerNorm = torch.layers.layerNorm(64);
-  const normInput = new Tensor(new Float32Array(2 * 10 * 64), [2, 10, 64]);
-  const normOutput = layerNorm.forward(normInput);
-  assert(normOutput.shape[0] === 2 && normOutput.shape[2] === 64, "LayerNorm forward pass");
-}
-
-// ====================== LOSS FUNCTIONS TESTS ======================
-testSuite("Loss Functions");
-
-function testLossFunctions() {
-  // MSE Loss
-  const mse = torch.loss.mse();
-  const pred = new Tensor([1, 2, 3, 4]);
-  const target = new Tensor([0, 2, 3, 5]);
-  const mseLoss = mse.forward(pred, target);
-  assert(Math.abs(mseLoss.data[0] - 0.5) < 0.001, "MSE Loss calculation");
-
-  // CrossEntropy Loss
-  const ce = torch.loss.crossEntropy();
-  const cePred = new Tensor([0.1, 0.9, 0.1, 0.9], [2, 2]);
-  const ceTarget = new Tensor([0, 1, 1, 0], [2, 2]);
-  const ceLoss = ce.forward(cePred, ceTarget);
-  assert(ceLoss.data[0] > 0, "CrossEntropy Loss calculation");
-
-  // Huber Loss
-  const huber = torch.loss.huber(1.0);
-  const huberLoss = huber.forward(pred, target);
-  assert(huberLoss.data[0] > 0, "Huber Loss calculation");
-
-  // Triplet Loss
-  const triplet = torch.loss.tripletLoss(1.0);
-  const anchor = new Tensor([1, 2]);
-  const positive = new Tensor([1.1, 2.1]);
-  const negative = new Tensor([3, 4]);
-  const tripletLoss = triplet.forward(anchor, positive, negative);
-  assert(tripletLoss.data[0] > 0, "Triplet Loss calculation");
-}
-
-// ====================== OPTIMIZERS TESTS ======================
-testSuite("Optimizers");
-
-function testOptimizers() {
-  // Create a simple model
-  const model = torch.nn.sequential([
-    torch.layers.linear(2, 3),
-    torch.layers.relu(),
-    torch.layers.linear(3, 1)
-  ]);
-
-  // SGD
-  const sgd = torch.optim.sgd(0.01);
-  const params = model.getParameters();
-  const initialWeights = params[0].data[0];
-  
-  sgd.step(params);
-  assert(params[0].data[0] !== initialWeights, "SGD parameter update");
-
-  // Adam
-  const adam = torch.optim.adam(0.001);
-  const adamInitialWeights = params[0].data[0];
-  
-  adam.step(params);
-  assert(params[0].data[0] !== adamInitialWeights, "Adam parameter update");
-
-  // AdamW
-  const adamw = torch.optim.adamw(0.001, 0.9, 0.999, 0.01);
-  const adamwInitialWeights = params[0].data[0];
-  
-  adamw.step(params);
-  assert(params[0].data[0] !== adamwInitialWeights, "AdamW parameter update");
-
-  // LAMB
-  const lamb = torch.optim.lamb(0.001);
-  const lambInitialWeights = params[0].data[0];
-  
-  lamb.step(params);
-  assert(params[0].data[0] !== lambInitialWeights, "LAMB parameter update");
-
-  // RMSprop
-  const rmsprop = torch.optim.rmsprop(0.01);
-  const rmspropInitialWeights = params[0].data[0];
-  
-  rmsprop.step(params);
-  assert(params[0].data[0] !== rmspropInitialWeights, "RMSprop parameter update");
-}
-
-// ====================== NEURAL NETWORK TESTS ======================
-testSuite("Neural Network");
-
-function testNeuralNetwork() {
-  // Sequential model
-  const model = torch.nn.sequential();
-  model.add(torch.layers.linear(4, 8));
-  model.add(torch.layers.relu());
-  model.add(torch.layers.dropout(0.2));
-  model.add(torch.layers.linear(8, 2));
-
-  const input = new Tensor([1, 2, 3, 4], [1, 4]);
-  const output = model.forward(input);
-  assert(output.shape[0] === 1 && output.shape[1] === 2, "Sequential model forward pass");
-
-  // Backward pass
-  const gradOutput = new Tensor([0.1, 0.2], [1, 2]);
-  const gradInput = model.backward(gradOutput);
-  assert(gradInput.shape[0] === 1 && gradInput.shape[1] === 4, "Sequential model backward pass");
-
-  // Get parameters
-  const params = model.getParameters();
-  assert(params.length > 0, "Sequential model parameter extraction");
-
-  // Test individual layer parameters
-  const linearLayer = torch.layers.linear(3, 2);
-  const linearParams = linearLayer.getParameters();
-  assert(linearParams.length === 2, "Linear layer parameter count");
-}
-
-// ====================== UTILS TESTS ======================
-testSuite("Utils");
-
-function testUtils() {
-  // DataLoader
-  const data = {
-    inputs: new Float32Array([1, 2, 3, 4, 5, 6, 7, 8]),
-    targets: new Float32Array([0, 1, 0, 1]),
-    inputShape: [2],
-    targetShape: [1]
-  };
-
-  const loader = torch.utils.dataLoader(data, 2, false);
-  const batches = [...loader];
-  assert(batches.length === 2, "DataLoader batch creation");
-  assert(batches[0].inputs.shape[0] === 2, "DataLoader batch size");
-
-  // One-hot encoding
-  const labels = [0, 2, 1];
-  const oneHot = torch.utils.oneHot(labels, 3);
-  assert(oneHot.shape[0] === 3 && oneHot.shape[1] === 3, "One-hot encoding shape");
-  assert(oneHot.data[0] === 1 && oneHot.data[4] === 1, "One-hot encoding values");
-
-  // Accuracy calculation
-  const pred = new Tensor([0.9, 0.1, 0.7, 0.3], [2, 2]);
-  const target = new Tensor([1, 0, 1, 0], [2, 2]);
-  const accuracy = torch.utils.accuracy(pred, target);
-  assert(accuracy === 1.0, "Accuracy calculation");
-
-  // Benchmark
-  const simpleModel = torch.nn.sequential([
-    torch.layers.linear(10, 5),
-    torch.layers.relu()
-  ]);
-  const benchmarkInput = new Tensor(new Float32Array(10), [1, 10]);
-  const benchmark = torch.utils.benchmark(simpleModel, benchmarkInput, 10);
-  assert(benchmark.avgTime > 0, "Benchmark execution");
-  assert(benchmark.minTime > 0, "Benchmark min time");
-  assert(benchmark.maxTime > 0, "Benchmark max time");
-
-  // Profile function
-  const profileResult = torch.utils.profile(() => {
-    const x = new Tensor([1, 2, 3]);
-    return x.relu();
-  });
-  assert(profileResult !== undefined, "Profile function execution");
-}
-
-// ====================== ADVANCED FEATURES TESTS ======================
-testSuite("Advanced Features");
-
-function testAdvancedFeatures() {
-  // Quantization
-  const quant = torch.quant;
-  const tensor = new Tensor([0.1, 0.5, 0.9]);
-  const quantized = quant.quantize(tensor, 'int8');
-  assert(quantized.dtype === 'int8', "Tensor quantization");
-  
-  const dequantized = quant.dequantize(quantized);
-  assert(dequantized.dtype === 'float32', "Tensor dequantization");
-
-  // Automatic Mixed Precision
-  const amp = torch.amp;
-  const ampModel = torch.nn.sequential([
-    torch.layers.linear(4, 2)
-  ]);
-  const ampInput = new Tensor([1, 2, 3, 4], [1, 4]);
-  const ampOutput = amp.forward(ampModel, ampInput);
-  assert(ampOutput.dtype === 'float32', "AMP forward pass");
-
-  // Visualization
-  const viz = torch.viz;
-  viz.init();
-  viz.logLoss(new Tensor([0.5]));
-  viz.logAccuracy(0.8);
-  viz.logLearningRate(0.001);
-  const metrics = viz.getMetrics();
-  assert(metrics.loss.length > 0, "Visualization metrics logging");
-
-  // Learning Rate Schedulers
-  const optimizer = torch.optim.sgd(0.1);
-  const scheduler = new StepLR(optimizer, stepSize=2, gamma=0.1);
-  
-  let lr = scheduler.step();
-  assert(lr === 0.1, "StepLR initial learning rate");
-  
-  lr = scheduler.step();
-  assert(lr === 0.1, "StepLR before step");
-  
-  lr = scheduler.step();
-  assert(lr === 0.01, "StepLR after step");
-
-  // ExponentialLR
-  const expScheduler = new ExponentialLR(optimizer, gamma=0.5);
-  lr = expScheduler.step();
-  assert(lr === 0.005, "ExponentialLR learning rate");
-
-  // CosineAnnealingLR
-  const cosScheduler = new CosineAnnealingLR(optimizer, T_max=10);
-  lr = cosScheduler.step();
-  assert(lr > 0 && lr < 0.1, "CosineAnnealingLR learning rate");
-
-  // Gradient Clipping
-  const params = [new Tensor([1, 2, 3], [3], true)];
-  params[0].grad = new Tensor([0.5, 1.5, 0.8]);
-  
-  const norm = clipGradNorm_(params, 1.0);
-  assert(norm > 0, "Gradient norm calculation");
-  
-  clipGradValue_(params, 1.0);
-  assert(params[0].grad.data[1] <= 1.0, "Gradient value clipping");
-
-  // Tensor map function (synchronous)
-  const mapTensor = new Tensor([1, 2, 3, 4]);
-  const mapped = mapTensor.map(x => x * 2);
-  assert(mapped.data[0] === 2 && mapped.data[3] === 8, "Tensor map function");
-}
-
-// ====================== INTEGRATION TESTS ======================
-testSuite("Integration");
-
-function testIntegration() {
-  // Complete training simulation
-  const model = torch.nn.sequential([
-    torch.layers.linear(4, 8),
-    torch.layers.relu(),
-    torch.layers.dropout(0.2),
-    torch.layers.linear(8, 2)
-  ]);
-
-  const optimizer = torch.optim.adam(0.01);
-  const lossFn = torch.loss.mse();
-  const scheduler = new StepLR(optimizer, stepSize=10, gamma=0.1);
-
-  // Create dummy data
-  const data = {
-    inputs: new Float32Array(100 * 4),
-    targets: new Float32Array(100 * 2),
-    inputShape: [4],
-    targetShape: [2]
-  };
-
-  // Fill with random data
-  for (let i = 0; i < data.inputs.length; i++) {
-    data.inputs[i] = Math.random();
-    data.targets[i] = Math.random();
-  }
-
-  const loader = torch.utils.dataLoader(data, 10);
-
-  // Training loop
-  let initialLoss = Infinity;
-  let finalLoss = 0;
-
-  for (let epoch = 0; epoch < 3; epoch++) {
-    for (const batch of loader) {
-      // Forward
-      const output = model.forward(batch.inputs);
-      const loss = lossFn.forward(output, batch.targets);
-
-      if (epoch === 0) {
-        initialLoss = loss.data[0];
-      }
-
-      // Backward
-      lossFn.backward();
-      model.backward(loss.grad);
-
-      // Update
-      optimizer.step(model.getParameters());
-      model.getParameters().forEach(p => p.zeroGrad());
-    }
-
-    scheduler.step();
-  }
-
-  // Check final loss
-  const finalBatch = [...loader][0];
-  const finalOutput = model.forward(finalBatch.inputs);
-  const finalLossTensor = lossFn.forward(finalOutput, finalBatch.targets);
-  finalLoss = finalLossTensor.data[0];
-
-  assert(finalLoss < initialLoss, "Training reduces loss");
-  console.log(`Initial loss: ${initialLoss.toFixed(4)}, Final loss: ${finalLoss.toFixed(4)}`);
-
-  // Model saving/loading simulation
-  const modelData = {
-    layers: model.layers.map(layer => ({
-      type: layer.constructor.name,
-      params: layer.getParameters().map(param => ({
-        data: Array.from(param.data),
-        shape: param.shape,
-        requiresGrad: param.requiresGrad
-      }))
-    }))
-  };
-
-  assert(modelData.layers.length > 0, "Model serialization");
-
-  // Test distributed initialization (sync version)
-  const distributed = torch.distributed;
-  distributed.init(1, 0); // Single process
-  assert(distributed.worldSize === 1, "Distributed initialization");
-}
-
-// ====================== PERFORMANCE TESTS ======================
-testSuite("Performance");
-
-function testPerformance() {
-  // Large matrix multiplication performance
-  console.log("Testing large matrix multiplication...");
-  const size = 256;
-  const a = new Tensor(new Float32Array(size * size), [size, size]);
-  const b = new Tensor(new Float32Array(size * size), [size, size]);
-
-  const start = performance.now();
-  const c = a.matmul(b);
-  const end = performance.now();
-
-  const timeMs = end - start;
-  console.log(`Matrix multiplication (${size}x${size}): ${timeMs.toFixed(2)}ms`);
-  assert(timeMs < 1000, "Large matrix multiplication performance");
-
-  // Memory usage
-  const memoryStats = Tensor.memoryTracker.getStats();
-  console.log(`Memory usage: ${(memoryStats.totalMemory / 1024 / 1024).toFixed(2)}MB`);
-  assert(memoryStats.totalMemory < 100 * 1024 * 1024, "Memory usage within limits");
-
-  // Synchronous map test
-  console.log("Testing synchronous map...");
-  const largeTensor = new Tensor(new Float32Array(10000), [10000]);
-  
-  const mapStart = performance.now();
-  const result = largeTensor.map(x => x * 2);
-  const mapEnd = performance.now();
-  
-  const mapTime = mapEnd - mapStart;
-  console.log(`Synchronous map (10000 elements): ${mapTime.toFixed(2)}ms`);
-  assert(mapTime < 100, "Synchronous map performance");
-
-  // Convolution performance
-  console.log("Testing convolution performance...");
-  const conv = torch.layers.conv2d(3, 16, 3);
-  const convInput = new Tensor(new Float32Array(1 * 3 * 64 * 64), [1, 3, 64, 64]);
-  
-  const convStart = performance.now();
-  const convOutput = conv.forward(convInput);
-  const convEnd = performance.now();
-  
-  const convTime = convEnd - convStart;
-  console.log(`Convolution (1x3x64x64): ${convTime.toFixed(2)}ms`);
-  assert(convTime < 500, "Convolution performance");
-
-  // LSTM performance
-  console.log("Testing LSTM performance...");
-  const lstm = torch.layers.lstm(64, 128, 2);
-  const lstmInput = new Tensor(new Float32Array(10 * 4 * 64), [10, 4, 64]);
-  
-  const lstmStart = performance.now();
-  const lstmOutput = lstm.forward(lstmInput);
-  const lstmEnd = performance.now();
-  
-  const lstmTime = lstmEnd - lstmStart;
-  console.log(`LSTM (10x4x64): ${lstmTime.toFixed(2)}ms`);
-  assert(lstmTime < 1000, "LSTM performance");
-}
-
-// ====================== RUN ALL TESTS ======================
-function runAllTests() {
-  console.log("🚀 Starting mini-jstorch test suite...\n");
-
-  try {
-    testTensorOperations();
-    testLayers();
-    testLossFunctions();
-    testOptimizers();
-    testNeuralNetwork();
-    testUtils();
-    testAdvancedFeatures();
-    testIntegration();
-    testPerformance();
-
-    summarize();
-
-    if (testsFailed === 0) {
-      console.log("\n🎉 All tests passed! mini-jstorch is working correctly!");
-    } else {
-      console.log(`\n⚠️  ${testsFailed} test(s) failed. Please check the implementation.`);
-    }
-
-  } catch (error) {
-    console.error("\n💥 Test suite crashed:", error);
-    testsFailed++;
-    summarize();
-  }
-}
-
-// Run the tests
-runAllTests();
+// Run again to show change
+const output2 = model.forward(input.data);
+const loss2 = criterion.forward(output2, target.data);
+console.log("New Loss:", loss2);

package/tests/scheduler.js

@@ -0,0 +1,23 @@
+// Example: Test learning rate schedulers (StepLR and LambdaLR) with mini-jstorch optimizers
+
+import { SGD, StepLR, LambdaLR, Tensor } from "../src/MainEngine.js";
+
+const param = { param: [[1, 2], [3, 4]], grad: [[0, 0], [0, 0]] };
+const optimizer = new SGD([param], 0.1);
+
+// --- Test StepLR ---
+console.log("Testing StepLR...");
+const stepScheduler = new StepLR(optimizer, 3, 0.5);
+for (let epoch = 1; epoch <= 10; epoch++) {
+    stepScheduler.step();
+    console.log(`Epoch ${epoch}: LR = ${optimizer.lr.toFixed(4)}`);
+}
+
+// --- Test LambdaLR ---
+console.log("\nTesting LambdaLR...");
+optimizer.lr = 0.1; // Reset LR
+const lambdaScheduler = new LambdaLR(optimizer, epoch => 1.0 / (1 + epoch));
+for (let epoch = 1; epoch <= 5; epoch++) {
+    lambdaScheduler.step();
+    console.log(`Epoch ${epoch}: LR = ${optimizer.lr.toFixed(4)}`);
+}

package/MODULE.md

@@ -1,41 +0,0 @@
-## MODULE STATS ##
-
-New Files that automatically will All notable changes status state to *Mini-JSTorch* will be documented in this file.
-
-# MSG
-
-btw. This files is actually would be modified and note all changes system state automatically without manual i type with myself.
-
----
-
-**OFFICIAL RELEASE:** 2025-Monday-August-23 time: 2:22 AM (estimated time release)
-**VERSION:** 1.3.1
-**LICENSE:** MIT © 2025
-**AUTHOR:** Rizal
-**MODULE NAME:** mini-jstorch
-**MODULE DESC:** A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices Inspired by PyTorch.
-**MODULE TYPE:** module
-**ENGINE VERSIONS:** 1.2.1
-**UPDATE TITLE:** `PATCH` update.
-**ADDED FILES/FOLDER:** {
-    "N/A" //N/A [N/A]
-}
-
----
-
-**MODIFIED FILES:** {
-    "src" //folder
-    "src/startup.cpu" //files
-    "src/MainEngine.js" //files
-    "tests/tests.js" //files [npmignore detected]
-    "tests" //folder
-    "src/Dummy/exp.js" //files [npmignore detected]
-    "package.json" //files
-    "src/EngState.json" //files [npmignore detected]
-    "src/state.txt" //files [npmignore detected]
-    "README.md" //files
-    ".npmignore" //files [npmignore detected]
-    "N/A" //N/A [N/A]
-}
-
----

package/hh.js

@@ -1,38 +0,0 @@
-import { Sequential, Linear, ReLU, Sigmoid, CrossEntropyLoss, Adam } from './src/MainEngine.js';
-
-// Build model
-const model = new Sequential([
-  new Linear(2,4),
-  new ReLU(),
-  new Linear(4,2),
-  new Sigmoid()
-]);
-
-// Sample XOR dataset
-const X = [
-  [0,0], [0,1], [1,0], [1,1]
-];
-const Y = [
-  [1,0], [0,1], [0,1], [1,0]
-];
-
-// Loss & optimizer
-const lossFn = new CrossEntropyLoss();
-const optimizer = new Adam(model.parameters(), 0.1);
-
-// Training loop
-for (let epoch = 1; epoch <= 500; epoch++) {
-  const pred = model.forward(X);
-  const loss = lossFn.forward(pred, Y);
-  const gradLoss = lossFn.backward();
-  model.backward(gradLoss);
-  optimizer.step();
-  if (epoch % 20 === 0) console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}`);
-}
-
-// Prediction
-const predTest = model.forward(X);
-predTest.forEach((p,i) => {
-  const predictedClass = p.indexOf(Math.max(...p));
-  console.log(`Input: ${X[i]}, Predicted class: ${predictedClass}, Raw output: ${p.map(v => v.toFixed(3))}`);
-});

package/tests/DebugModel.js

@@ -1,127 +0,0 @@
-// src/Dummy/debug_train.js
-import { 
-    Tensor,
-    Linear,
-    Sequential,
-    ReLU,
-    Sigmoid,
-    CrossEntropyLoss,
-    Adam,
-    MathOps
-} from '../src/Dummy/exp.js';
-
-// ---------------------- Simple Debug Data ----------------------
-function generateSimpleData() {
-    // VERY simple data: 4 points in 2D
-    const X = [
-        [0, 0],
-        [0, 1], 
-        [1, 0],
-        [1, 1]
-    ];
-    
-    const y = [
-        [0], // AND operation
-        [0],
-        [0], 
-        [1]
-    ];
-    
-    return { X, y };
-}
-
-// ---------------------- Debug Model ----------------------
-function createDebugModel() {
-    return new Sequential([
-        new Linear(2, 2),  // Small layer
-        new ReLU(),
-        new Linear(2, 1),  // Output layer
-        new Sigmoid()
-    ]);
-}
-
-// ---------------------- Debug Training ----------------------
-function debugTraining() {
-    console.log("🔍 DEBUG TRAINING STARTED");
-    console.log("=========================");
-    
-    const data = generateSimpleData();
-    const model = createDebugModel();
-    const lossFunction = new CrossEntropyLoss();
-    const parameters = model.parameters();
-    
-    console.log("Model parameters:", parameters.length);
-    console.log("Data samples:", data.X.length);
-    
-    // Single step debug
-    for (let step = 0; step < 10; step++) {
-        console.log(`\n--- Step ${step} ---`);
-        
-        // Forward pass
-        const predictions = model.forward(data.X);
-        console.log("Predictions:", predictions.map(p => p[0].toFixed(3)));
-        
-        const loss = lossFunction.forward(predictions, data.y);
-        console.log("Loss:", loss);
-        
-        if (isNaN(loss)) {
-            console.log("❌ NaN LOSS DETECTED!");
-            console.log("Predictions:", predictions);
-            console.log("Targets:", data.y);
-            break;
-        }
-        
-        // Backward pass
-        const grad = lossFunction.backward();
-        console.log("Gradient:", grad.map(g => g[0].toFixed(3)));
-        
-        model.backward(grad);
-        
-        // Check gradients
-        console.log("Parameter gradients:");
-        parameters.forEach((param, idx) => {
-            if (Array.isArray(param.grad[0])) {
-                console.log(`  Param ${idx} grad:`, param.grad.map(row => 
-                    row.map(v => v.toFixed(3))
-                ));
-            } else {
-                console.log(`  Param ${idx} grad:`, param.grad.map(v => v.toFixed(3)));
-            }
-        });
-        
-        // Update weights
-        const optimizer = new Adam(parameters, 0.1);
-        optimizer.step();
-        
-        // Reset gradients manually
-        parameters.forEach(param => {
-            if (Array.isArray(param.grad[0])) {
-                for (let i = 0; i < param.grad.length; i++) {
-                    for (let j = 0; j < param.grad[0].length; j++) {
-                        param.grad[i][j] = 0;
-                    }
-                }
-            } else {
-                for (let i = 0; i < param.grad.length; i++) {
-                    param.grad[i] = 0;
-                }
-            }
-        });
-        
-        // Calculate accuracy
-        const accuracy = calculateAccuracy(predictions, data.y);
-        console.log("Accuracy:", (accuracy * 100).toFixed(1) + "%");
-    }
-}
-
-function calculateAccuracy(predictions, targets) {
-    let correct = 0;
-    for (let i = 0; i < predictions.length; i++) {
-        const predLabel = predictions[i][0] > 0.5 ? 1 : 0;
-        if (predLabel === targets[i][0]) correct++;
-    }
-    return correct / predictions.length;
-}
-
-// Run debug
-debugTraining();

package/tests/unit/newver.js

@@ -1,103 +0,0 @@
-// tests/engine.test.js
-import { 
-    MatrixOps,
-    Tensor,
-    Linear,
-    CrossEntropyLoss,  // Ganti pake classnya
-    Adam,
-    Sequential,
-    ReLU,
-    MathOps            // Import MathOps yang ada softmax di dalamnya
-} from '../../src/Dummy/exp.js';
-
-// Alias untuk softmax dari MathOps
-const softmax = MathOps.softmax;
-
-// Test 1: MatrixOps.matmul vs old Tensor.matmul (dot)
-console.log("=== TEST 1: Matrix Multiplication ===");
-const A = [[1, 2], [3, 4]];
-const B = [[5, 6], [7, 8]];
-
-// Correct matrix multiplication
-const correctResult = MatrixOps.matmul(A, B);
-console.log("Correct matmul result:", correctResult);
-
-// Test new Tensor matmul
-const tensorA = new Tensor(A);
-const tensorB = new Tensor(B);
-const tensorResult = tensorA.matmul(tensorB);
-console.log("New Tensor matmul result:", tensorResult.data);
-
-// Test 2: Tensor operations consistency
-console.log("\n=== TEST 2: Tensor Operations ===");
-const testTensor = new Tensor([[1, 2], [3, 4]]);
-console.log("Original shape:", testTensor.shape());
-console.log("Transpose shape:", testTensor.transpose().shape());
-console.log("Flatten shape:", testTensor.flatten().shape());
-
-// Test 3: CrossEntropyLoss backward pass
-console.log("\n=== TEST 3: CrossEntropyLoss Backward ===");
-const pred = [[0.8, 0.1, 0.1], [0.2, 0.7, 0.1]];
-const target = [[1, 0, 0], [0, 1, 0]];
-
-const lossFn = new CrossEntropyLoss();
-const loss = lossFn.forward(pred, target);
-console.log("Loss value:", loss);
-
-const grad = lossFn.backward();
-console.log("Gradient shape per sample:", grad.map(g => g.length));
-console.log("Sample gradient:", grad[0]);
-
-// Test 4: Adam optimizer with 1D and 2D parameters
-console.log("\n=== TEST 4: Adam Optimizer ===");
-
-// Mock parameters (like from a Linear layer)
-const mockParams = [
-    {
-        param: [[0.5, 0.3], [0.1, 0.9]], // 2D weights
-        grad: [[0.1, 0.2], [0.3, 0.4]]
-    },
-    {
-        param: [0.1, 0.2], // 1D bias
-        grad: [0.01, 0.02]
-    }
-];
-
-console.log("Before optimization - Weights:", mockParams[0].param);
-console.log("Before optimization - Bias:", mockParams[1].param);
-
-const optimizer = new Adam(mockParams, 0.01);
-optimizer.step();
-
-console.log("After optimization - Weights:", mockParams[0].param);
-console.log("After optimization - Bias:", mockParams[1].param);
-
-// Test 5: Integration test - Simple forward pass
-console.log("\n=== TEST 5: Integration Test ===");
-try {
-    const linearLayer = new Linear(2, 3);
-    const input = new Tensor([[1, 2]]);
-    
-    console.log("Input shape:", input.shape());
-    
-    const output = linearLayer.forward(input.data);
-    console.log("Linear layer output shape:", [output.length, output[0].length]);
-    
-    // Test with ReLU
-    const relu = new ReLU();
-    const activated = relu.forward(output);
-    console.log("ReLU output shape:", [activated.length, activated[0].length]);
-    
-    console.log("✅ All integration tests passed!");
-} catch (error) {
-    console.log("❌ Integration test failed:", error.message);
-}
-
-// Test 6: Softmax consistency
-console.log("\n=== TEST 6: Softmax ===");
-const logits = [2.0, 1.0, 0.1];
-const sm = softmax(logits);
-console.log("Softmax result:", sm);
-console.log("Sum:", sm.reduce((a, b) => a + b, 0));
-
-console.log("\n=== ALL TESTS COMPLETED ===");

package/tests/unit/ogver.js

@@ -1,87 +0,0 @@
-// TEST JSTORCH WHOLE SYSTEMS AT ONCE
-// THIS FILES TESTINGS AT PREVIOUS UPDATE 1.3.0
-// DEPRECATED FILES NOT NEW
-import { Tensor, Linear, Sequential, ReLU, Sigmoid, Tanh, LeakyReLU, GELU, Dropout, Conv2D, MSELoss, CrossEntropyLoss, Adam, SGD, saveModel, loadModel, flattenBatch, reshape, stack, concat, eye } from '../../src/MainEngine.js';
-
-// ---------------------- Linear Test ----------------------
-console.log("=== Linear Test ===");
-const lin = new Linear(3,2);
-const linInput = [[1,2,3],[4,5,6]];
-const linOut = lin.forward(linInput);
-console.log("Linear forward:", linOut);
-const linGrad = [[0.1,0.2],[0.3,0.4]];
-const linBack = lin.backward(linGrad);
-console.log("Linear backward gradInput:", linBack);
-
-// ---------------------- Sequential + Activations Test ----------------------
-console.log("\n=== Sequential + Activations Test ===");
-const model = new Sequential([new Linear(2,2), new ReLU(), new Linear(2,1), new Sigmoid()]);
-const seqInput = [[0.5,1.0],[1.5,2.0]];
-const seqOut = model.forward(seqInput);
-console.log("Sequential forward:", seqOut);
-const seqGrad = [[0.1],[0.2]];
-const seqBack = model.backward(seqGrad);
-console.log("Sequential backward gradInput:", seqBack);
-
-// ---------------------- Conv2D Test ----------------------
-console.log("\n=== Conv2D Test ===");
-const conv = new Conv2D(1,1,3);
-const convInput = [[[ [1,2,3],[4,5,6],[7,8,9] ]]]; // batch=1, inC=1, HxW=3x3
-const convOut = conv.forward(convInput);
-console.log("Conv2D forward:", convOut);
-
-// Conv2D backward test
-const convGrad = [[[ [0.1,0.2,0.1],[0.2,0.3,0.2],[0.1,0.2,0.1] ]]];
-const convBack = conv.backward(convGrad);
-console.log("Conv2D backward gradInput:", convBack);
-
-// ---------------------- Tensor & Broadcast Test ----------------------
-console.log("\n=== Tensor & Broadcast Test ===");
-const a = Tensor.random(2,3);
-const b = Tensor.ones(2,3);
-const sum = a.add(b);
-console.log("Tensor add broadcast:", sum);
-
-// ---------------------- Loss + Optimizer Test ----------------------
-console.log("\n=== Loss + Optimizer Test ===");
-const lossModel = new Sequential([new Linear(2,2)]);
-const pred = lossModel.forward([[1,2]]);
-const target = [[0,1]];
-const ceLoss = new CrossEntropyLoss();
-const lval = ceLoss.forward(pred,target);
-console.log("CrossEntropyLoss value:", lval);
-
-const gradLoss = ceLoss.backward();
-lossModel.backward(gradLoss);
-
-const opt = new Adam(lossModel.parameters());
-opt.step();
-console.log("Updated parameters after Adam:", lossModel.parameters());
-
-// ---------------------- Dropout Test ----------------------
-console.log("\n=== Dropout Test ===");
-const drop = new Dropout(0.5);
-const dropInput = [[1,2],[3,4]];
-const dropOut = drop.forward(dropInput);
-console.log("Dropout forward:", dropOut);
-const dropBack = drop.backward([[0.1,0.2],[0.3,0.4]]);
-console.log("Dropout backward:", dropBack);
-
-// ---------------------- Save / Load Model Test ----------------------
-console.log("\n=== Save / Load Model Test ===");
-const modelSave = new Sequential([new Linear(2,2)]);
-const json = saveModel(modelSave);
-console.log("Saved model JSON:", json);
-const modelLoad = new Sequential([new Linear(2,2)]);
-loadModel(modelLoad,json);
-console.log("Loaded model parameters:", modelLoad.parameters());
-
-// ---------------------- Advanced Utils Test ----------------------
-console.log("\n=== Advanced Utils Test ===");
-const batch = [[[1,2],[3,4]],[[5,6],[7,8]]];
-console.log("Flatten batch:", flattenBatch(batch));
-console.log("Eye 3:", eye(3));
-console.log("Reshape:", reshape({data:[[1,2,3,4]]},2,2));
-console.log("Stack:", stack([Tensor.ones(2,2), Tensor.zeros(2,2)]));
-console.log("Concat axis0:", concat([[1,2],[3,4]], [[5,6],[7,8]], 0));
-console.log("Concat axis1:", concat([[1,2],[3,4]], [[5,6],[7,8]], 1));