mini-jstorch 1.3.0 → 1.4.2

package/README.md

@@ -1,50 +1,42 @@
-# Mini-JSTorch Major Update
+# Mini-JSTorch
 
-# MAJOR UPDATE IS RELEASED NOW!! [got little late] 
-
----
-
-## 🙏 Big Thanks 🙏
-I genuinely appreciate everyone who has taken the time to install and try out this module. Every download isn’t just a number it’s a real person spending their time to explore, learn, or build something with this module!
-
-Reaching 400+ weekly downloads might sound small compared to giant frameworks, but for me it’s huge. It means this project is actually helping people out there, and that alone makes every late-night coding session worth it.
-
-Thank you for giving this module a chance. Your support and trust are what keep this project moving forward! 🚀
-
----
+A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices, inspired by PyTorch.
 
 ## Overview
 
-Mini-JSTorch is a lightweight, high-performance JavaScript library for building neural networks that runs efficiently in both frontend and backend environments, including low-end devices. The library enables experimentation and learning in AI without compromising stability, accuracy, or training reliability.
+Mini-JSTorch is a high-performance, minimalist JavaScript library for building neural networks. It runs efficiently in both frontend and backend environments, including low-end devices. The library enables quick experimentation and learning in AI without compromising stability, accuracy, or training reliability.
 
-This release, **version 1.2.3**, is a **major update** that introduces full feature coverage, including convolutional layers, advanced activations, dropout, tensor broadcasting, and enhanced utilities. The engine remains pure JavaScript, lightweight, and compatible with both Node.js and browser environments.
+This release, **version 1.4.3**, we introduces **learning rate schedulers**, improved testing/demo templates, and other minor enhancements.
 
 ---
 
-## Major Updates in 1.2.3
-
-- Full **Conv2D support** with forward and backward operations.
-- **Tensor operations** now support broadcasting and reshaping.
-- Added new activations:  `LeakyReLU`, `GELU`.
-- Optimizers: `Adam` and `SGD` fully integrated with gradient updates.
-- Dropout layer added for regularization.
-- Advanced utilities for tensor manipulation: `flatten`, `stack`, `concat`, `eye`, `reshape`.
-- End-to-end training and prediction workflow now fully tested.
-- Save and load model functionality included for seamless persistence.
-- Optimized performance for both frontend and backend usage.
-- Maintained backward compatibility for previous core layers and activations.
-- folder tests files template that you can use it before make your models.
+## Feature Highlights
+
+- **Learning Rate Schedulers:** New `StepLR` and `LambdaLR` for dynamic optimizer learning rate adjustment.
+- **Full Conv2D support:** Forward and backward operations for convolutional layers.
+- **Tensor operations:** Broadcasting, reshaping, and reduction utilities.
+- **Advanced Activations:** Includes `LeakyReLU`, `GELU`, `Mish`, `SiLU`, `ELU`, and more.
+- **Optimizers:** `Adam` and `SGD` with gradient updates.
+- **Dropout Layer:** For regularization during training.
+- **BatchNorm2D:** For stable training in convolutional models.
+- **Tensor Manipulation:** Utilities like `flatten`, `stack`, `concat`, `eye`, `reshape`.
+- **Model Save & Load:** Easy persistence and restore of models.
+- **Test/Demo Templates:** The `tests/` folder provides ready-to-run examples for model building and feature usage.
+- **Performance Optimized:** Suitable for both frontend and backend usage.
+- **Backward Compatibility:** Maintained for core layers and activations.
 
 ---
 
-## Features
+## Core Features
 
-- **Core Layers**: Linear, Conv2D  
-- **Activations**: ReLU, Sigmoid, Tanh, LeakyReLU, GELU  
-- **Loss Functions**: MSELoss, CrossEntropyLoss  
-- **Optimizers**: Adam, SGD  
-- **Utilities**: zeros, randomMatrix, softmax, crossEntropy, dot, addMatrices, reshape, stack, flatten, eye, concat  
-- **Model Container**: Sequential (for stacking layers with forward/backward passes)  
+- **Layers:** Linear, Conv2D  
+- **Activations:** ReLU, Sigmoid, Tanh, LeakyReLU, GELU, Mish, SiLU, ELU  
+- **Loss Functions:** MSELoss, CrossEntropyLoss
+- **Optimizers:** Adam, SGD  
+- **Schedulers:** StepLR, LambdaLR  
+- **Regularization:** Dropout, BatchNorm2D  
+- **Utilities:** zeros, randomMatrix, softmax, crossEntropy, dot, addMatrices, reshape, stack, flatten, eye, concat  
+- **Model Container:** Sequential (for stacking layers with forward/backward passes)  
 
 ---
 
@@ -60,7 +52,7 @@ npm install mini-jstorch
 ## Quick Start Example
 
 ```javascript
-import { Sequential, Linear, ReLU, Sigmoid, CrossEntropyLoss, Adam } from './src/MainEngine.js';
+import { Sequential, Linear, ReLU, Sigmoid, CrossEntropyLoss, Adam, StepLR } from 'mini-jstorch';
 
 // Build model
 const model = new Sequential([
@@ -81,6 +73,7 @@ const Y = [
 // Loss & optimizer
 const lossFn = new CrossEntropyLoss();
 const optimizer = new Adam(model.parameters(), 0.1);
+const scheduler = new StepLR(optimizer, 20, 0.5); // Halve LR every 20 epochs
 
 // Training loop
 for (let epoch = 1; epoch <= 100; epoch++) {
@@ -89,7 +82,8 @@ for (let epoch = 1; epoch <= 100; epoch++) {
   const gradLoss = lossFn.backward();
   model.backward(gradLoss);
   optimizer.step();
-  if (epoch % 20 === 0) console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}`);
+  scheduler.step();
+  if (epoch % 20 === 0) console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}, LR: ${optimizer.lr.toFixed(4)}`);
 }
 
 // Prediction
@@ -105,7 +99,7 @@ predTest.forEach((p,i) => {
 ## Save & Load Models
 
 ```javascript
-import { saveModel, loadModel } from 'mini-jstorch';
+import { saveModel, loadModel, Sequential } from './src/MainEngine.js';
 
 const json = saveModel(model);
 const model2 = new Sequential([...]); // same architecture
@@ -114,7 +108,22 @@ loadModel(model2, json);
 
 ---
 
+## Demos & Testing
+
+Check the `tests/` directory for ready-to-run demos:
+- **tests/MakeModel.js:** Build and run a simple neural network.
+- **tests/scheduler.js:** Experiment with learning rate schedulers.
+- Add your own scripts for quick prototyping!
+
+```bash
+node tests/MakeModel.js
+node tests/scheduler.js
+```
+
+---
+
 ## Intended Use Cases
+
  - Rapid prototyping of neural networks in frontend and backend.
  - Learning and teaching foundational neural network concepts.
  - Experimentation on low-end devices or mobile browsers.
@@ -125,12 +134,3 @@ loadModel(model2, json);
 # License
 
 **MIT © 2025 Rizal**
-
----
-
-## Facts
-
-- **This module is implemented entirely in pure JavaScript.**
-- **The `Dummy` folder contains modules used for development, testing, and debugging before integration into the main engine.**
-- **files startup.cpu is actually an some random files lol.**
-- **This module was created by a `single` developer.**

package/index.js

@@ -1,6 +1,2 @@
-// Entry point of the library, export main classes and functions [DEPRECATED]
-export { Seq } from './models/seq.js';
-export { Dense } from './layers/dense.js';
-export * as act from './act/linear.js';
-export { SGD } from './optim/sgd.js';
-export { train } from './train/loop.js';
+// package root
+export * from "./src/MainEngine.js";

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "mini-jstorch",
-  "version": "1.3.0",
+  "version": "1.4.2",
   "type": "module", 
   "description": "A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices Inspired by PyTorch.",
   "main": "index.js",

package/src/MainEngine.js

@@ -1,5 +1,7 @@
-// MINI JSTORCH ENGINE - MAJOR ULTRA OPTIMIZED 1.2.3
-// LICENSE: MIT (C) Rizal 2025
+
+// OFFICIAL MINI-JSTORCH ENGINE
+// LICENSED UNDER MIT LICENSE
+// MIT (C) Rizal 2025
 
 // ---------------------- Utilities ----------------------
 function zeros(rows, cols) { return Array.from({length:rows},()=>Array(cols).fill(0)); }
@@ -222,8 +224,350 @@ export class Adam{
     }
 }
 
+// ---------------------- Learning Rate Schedulers ----------------------
+export class StepLR {
+    constructor(optimizer, step_size, gamma=1.0) {
+        this.optimizer = optimizer;
+        this.step_size = step_size;
+        this.gamma = gamma;
+        this.last_epoch = 0;
+        this.base_lr = optimizer.lr;
+    }
+
+    step() {
+        this.last_epoch += 1;
+        if (this.last_epoch % this.step_size === 0) {
+            this.optimizer.lr *= this.gamma;
+        }
+    }
+
+    get_lr() {
+        return this.optimizer.lr;
+    }
+}
+
+export class LambdaLR {
+    constructor(optimizer, lr_lambda) {
+        this.optimizer = optimizer;
+        this.lr_lambda = lr_lambda;
+        this.last_epoch = 0;
+        this.base_lr = optimizer.lr;
+    }
+
+    step() {
+        this.last_epoch += 1;
+        this.optimizer.lr = this.base_lr * this.lr_lambda(this.last_epoch);
+    }
+
+    get_lr() {
+        return this.optimizer.lr;
+    }
+}
+
+// ---------------------- ELU Activation ----------------------
+export class ELU {
+    constructor(alpha=1.0) {
+        this.alpha = alpha;
+        this.out = null;
+    }
+
+    forward(x) {
+        this.out = x.map(row => 
+            row.map(v => v > 0 ? v : this.alpha * (Math.exp(v) - 1))
+        );
+        return this.out;
+    }
+
+    backward(grad) {
+        return grad.map((row, i) => 
+            row.map((v, j) => 
+                v * (this.out[i][j] > 0 ? 1 : this.alpha * Math.exp(this.out[i][j]))
+            )
+        );
+    }
+}
+
+// ---------------------- Mish Activation ----------------------
+export class Mish {
+    constructor() {
+        this.x = null;
+    }
+
+    forward(x) {
+        this.x = x;
+        return x.map(row => 
+            row.map(v => {
+                // Mish(x) = x * tanh(softplus(x)) = x * tanh(ln(1 + e^x))
+                const softplus = Math.log(1 + Math.exp(v));
+                return v * Math.tanh(softplus);
+            })
+        );
+    }
+
+    backward(grad) {
+        return grad.map((row, i) => 
+            row.map((v, j) => {
+                const x_val = this.x[i][j];
+                
+                // Gradient of Mish: 
+                // δ = ω * (4(x+1) + 4e^2x + e^3x + e^x(4x+6)) / (2e^x + e^2x + 2)^2
+                // where ω = sech^2(softplus(x))
+                
+                const exp_x = Math.exp(x_val);
+                const exp_2x = Math.exp(2 * x_val);
+                const exp_3x = Math.exp(3 * x_val);
+                const softplus = Math.log(1 + exp_x);
+                
+                const sech_softplus = 1 / Math.cosh(softplus);
+                const numerator = 4 * (x_val + 1) + 4 * exp_2x + exp_3x + exp_x * (4 * x_val + 6);
+                const denominator = Math.pow(2 * exp_x + exp_2x + 2, 2);
+                
+                const mish_grad = (sech_softplus * sech_softplus) * (numerator / denominator);
+                return v * mish_grad;
+            })
+        );
+    }
+}
+
+// ---------------------- SiLU Activation ----------------------
+export class SiLU {
+    constructor() {
+        this.x = null;
+    }
+
+    forward(x) {
+        this.x = x;
+        return x.map(row => 
+            row.map(v => v / (1 + Math.exp(-v)))  // x * sigmoid(x)
+        );
+    }
+
+    backward(grad) {
+        return grad.map((row, i) => 
+            row.map((v, j) => {
+                const x_val = this.x[i][j];
+                const sigmoid = 1 / (1 + Math.exp(-x_val));
+                return v * (sigmoid * (1 + x_val * (1 - sigmoid)));
+            })
+        );
+    }
+}
+
 export class SGD{ constructor(params,lr=0.01){ this.params=params; this.lr=lr; } step(){ this.params.forEach(p=>{ for(let i=0;i<p.param.length;i++) for(let j=0;j<(p.param[0].length||1);j++) p.param[i][j]-=this.lr*p.grad[i][j]; }); } }
 
+// ---------------------- BatchNorm2D ----------------------
+export class BatchNorm2d {
+    constructor(numFeatures, eps=1e-5, momentum=0.1, affine=true) {
+        this.numFeatures = numFeatures;
+        this.eps = eps;
+        this.momentum = momentum;
+        this.affine = affine;
+        
+        // Parameters
+        if (affine) {
+            this.weight = Array(numFeatures).fill(1);
+            this.bias = Array(numFeatures).fill(0);
+            this.gradWeight = Array(numFeatures).fill(0);
+            this.gradBias = Array(numFeatures).fill(0);
+        }
+        
+        // Running statistics
+        this.runningMean = Array(numFeatures).fill(0);
+        this.runningVar = Array(numFeatures).fill(1);
+        
+        // Training state
+        this.training = true;
+        this.x = null;
+        this.xCentered = null;
+        this.std = null;
+    }
+
+    forward(x) {
+        // x shape: [batch, channels, height, width]
+        this.x = x;
+        const batchSize = x.length;
+        const channels = x[0].length;
+        
+        if (this.training) {
+            // Calculate mean per channel
+            const means = Array(channels).fill(0);
+            for (let b = 0; b < batchSize; b++) {
+                for (let c = 0; c < channels; c++) {
+                    const channelData = x[b][c];
+                    let sum = 0;
+                    for (let i = 0; i < channelData.length; i++) {
+                        for (let j = 0; j < channelData[0].length; j++) {
+                            sum += channelData[i][j];
+                        }
+                    }
+                    means[c] += sum / (channelData.length * channelData[0].length);
+                }
+            }
+            means.forEach((_, c) => means[c] /= batchSize);
+            
+            // Calculate variance per channel
+            const variances = Array(channels).fill(0);
+            for (let b = 0; b < batchSize; b++) {
+                for (let c = 0; c < channels; c++) {
+                    const channelData = x[b][c];
+                    let sum = 0;
+                    for (let i = 0; i < channelData.length; i++) {
+                        for (let j = 0; j < channelData[0].length; j++) {
+                            sum += Math.pow(channelData[i][j] - means[c], 2);
+                        }
+                    }
+                    variances[c] += sum / (channelData.length * channelData[0].length);
+                }
+            }
+            variances.forEach((_, c) => variances[c] /= batchSize);
+            
+            // Update running statistics
+            for (let c = 0; c < channels; c++) {
+                this.runningMean[c] = this.momentum * means[c] + (1 - this.momentum) * this.runningMean[c];
+                this.runningVar[c] = this.momentum * variances[c] + (1 - this.momentum) * this.runningVar[c];
+            }
+            
+            // Normalize
+            this.xCentered = [];
+            this.std = Array(channels).fill(0).map(() => []);
+            
+            const output = [];
+            for (let b = 0; b < batchSize; b++) {
+                const batchOut = [];
+                for (let c = 0; c < channels; c++) {
+                    const channelData = x[b][c];
+                    const channelOut = zeros(channelData.length, channelData[0].length);
+                    const channelCentered = zeros(channelData.length, channelData[0].length);
+                    const channelStd = Math.sqrt(variances[c] + this.eps);
+                    this.std[c].push(channelStd);
+                    
+                    for (let i = 0; i < channelData.length; i++) {
+                        for (let j = 0; j < channelData[0].length; j++) {
+                            channelCentered[i][j] = channelData[i][j] - means[c];
+                            channelOut[i][j] = channelCentered[i][j] / channelStd;
+                            
+                            // Apply affine transformation if enabled
+                            if (this.affine) {
+                                channelOut[i][j] = channelOut[i][j] * this.weight[c] + this.bias[c];
+                            }
+                        }
+                    }
+                    
+                    batchOut.push(channelOut);
+                    if (b === 0) this.xCentered.push(channelCentered);
+                    else this.xCentered[c] = addMatrices(this.xCentered[c], channelCentered);
+                }
+                output.push(batchOut);
+            }
+            
+            return output;
+        } else {
+            // Inference mode - use running statistics
+            const output = [];
+            for (let b = 0; b < batchSize; b++) {
+                const batchOut = [];
+                for (let c = 0; c < channels; c++) {
+                    const channelData = x[b][c];
+                    const channelOut = zeros(channelData.length, channelData[0].length);
+                    const channelStd = Math.sqrt(this.runningVar[c] + this.eps);
+                    
+                    for (let i = 0; i < channelData.length; i++) {
+                        for (let j = 0; j < channelData[0].length; j++) {
+                            channelOut[i][j] = (channelData[i][j] - this.runningMean[c]) / channelStd;
+                            
+                            // Apply affine transformation if enabled
+                            if (this.affine) {
+                                channelOut[i][j] = channelOut[i][j] * this.weight[c] + this.bias[c];
+                            }
+                        }
+                    }
+                    
+                    batchOut.push(channelOut);
+                }
+                output.push(batchOut);
+            }
+            
+            return output;
+        }
+    }
+
+    backward(gradOutput) {
+        if (!this.training) {
+            throw new Error("Backward should only be called in training mode");
+        }
+        
+        const batchSize = gradOutput.length;
+        const channels = gradOutput[0].length;
+        
+        // Initialize gradients
+        const gradInput = this.x.map(batch => 
+            batch.map(channel => 
+                zeros(channel.length, channel[0].length)
+            )
+        );
+        
+        if (this.affine) {
+            this.gradWeight.fill(0);
+            this.gradBias.fill(0);
+        }
+        
+        for (let c = 0; c < channels; c++) {
+            let sumGradWeight = 0;
+            let sumGradBias = 0;
+            
+            for (let b = 0; b < batchSize; b++) {
+                const channelGrad = gradOutput[b][c];
+                const channelData = this.x[b][c];
+                
+                // Calculate gradients for bias and weight
+                if (this.affine) {
+                    for (let i = 0; i < channelGrad.length; i++) {
+                        for (let j = 0; j < channelGrad[0].length; j++) {
+                            sumGradBias += channelGrad[i][j];
+                            sumGradWeight += channelGrad[i][j] * (this.xCentered[c][i][j] / this.std[c][b]);
+                        }
+                    }
+                }
+                
+                // Calculate gradient for input
+                const n = channelData.length * channelData[0].length;
+                const stdInv = 1 / this.std[c][b];
+                
+                for (let i = 0; i < channelGrad.length; i++) {
+                    for (let j = 0; j < channelGrad[0].length; j++) {
+                        let grad = channelGrad[i][j];
+                        
+                        if (this.affine) {
+                            grad *= this.weight[c];
+                        }
+                        
+                        grad *= stdInv;
+                        gradInput[b][c][i][j] = grad;
+                    }
+                }
+            }
+            
+            if (this.affine) {
+                this.gradWeight[c] = sumGradWeight / batchSize;
+                this.gradBias[c] = sumGradBias / batchSize;
+            }
+        }
+        
+        return gradInput;
+    }
+
+    parameters() {
+        if (!this.affine) return [];
+        return [
+            { param: [this.weight], grad: [this.gradWeight] },
+            { param: [this.bias], grad: [this.gradBias] }
+        ];
+    }
+
+    train() { this.training = true; }
+    eval() { this.training = false; }
+}
+
 // ---------------------- Model Save/Load ----------------------
 export function saveModel(model){
     if(!(model instanceof Sequential)) throw new Error("saveModel supports only Sequential");
@@ -246,10 +590,10 @@ export function stack(tensors){ return tensors.map(t=>t.data); }
 export function eye(n){ return Array.from({length:n},(_,i)=>Array.from({length:n},(_,j)=>i===j?1:0)); }
 export function concat(a,b,axis=0){ /* concat along axis */ if(axis===0) return [...a,...b]; if(axis===1) return a.map((row,i)=>[...row,...b[i]]); }
 export function reshape(tensor, rows, cols) {
-    let flat = tensor.data.flat(); // flatten dulu
+    let flat = tensor.data.flat(); // flatten
     if(flat.length < rows*cols) throw new Error("reshape size mismatch");
     const out = Array.from({length: rows}, (_, i) =>
         flat.slice(i*cols, i*cols + cols)
     );
     return out;
-}
+}

package/src/startup.cpu

@@ -1,12 +1,15 @@
 // you can delete this files this files are not important for the engine runtime.
 
 e=run=[cpu[runtime]]
-e.set.runtime('beta')
-e.rnt()
-e.set()
-e.register('vanilla',expe='Experiments.js',main='MainEngine.js')
-l=e.prog('asm')
+devices=e.getdata[devices[5]]
+env.set.runtime('beta')
+env.rnt()
+env.set()
+env.register('vanilla',expe='Experiments.js',main='MainEngine.js',tgver=latest)
+resources=e.find(tag='resources')
+resources.ld(env)
+l=env.prog('asm')
 r=l.gv=[0xCAFEBABE]
-eng=e.load(register,r,'vanilla')
-eng.boot(e,r,'dp')
-eng.load()
+eng=env.load(register,r,'vanilla')
+eng.boot(env,r,'dp')
+eng.load(resources,runtime,devices)

package/tests/MakeModel.js

@@ -0,0 +1,38 @@
+// Example: Build and run a simple neural network model using mini-jstorch
+
+import { Sequential, Linear, ReLU, MSELoss, SGD, Tensor } from "../src/MainEngine.js";
+
+// Create dummy input and target data
+const input = new Tensor([[0.5, -1.0], [1.5, 2.0]]); // shape: [2,2]
+const target = new Tensor([[1.0, 0.0], [0.0, 1.0]]); // shape: [2,2]
+
+// Build a simple model: Linear -> ReLU -> Linear
+const model = new Sequential([
+    new Linear(2, 4),
+    new ReLU(),
+    new Linear(4, 2)
+]);
+
+const criterion = new MSELoss();
+const optimizer = new SGD(model.parameters(), 0.01);
+
+// Forward pass
+const output = model.forward(input.data);
+console.log("Model output:", output);
+
+// Compute loss
+const loss = criterion.forward(output, target.data);
+console.log("Loss:", loss);
+
+// Backward pass
+const grad = criterion.backward();
+model.backward(grad);
+
+// Optimizer step
+optimizer.step();
+console.log("Parameters updated!");
+
+// Run again to show change
+const output2 = model.forward(input.data);
+const loss2 = criterion.forward(output2, target.data);
+console.log("New Loss:", loss2);

package/tests/scheduler.js

@@ -0,0 +1,23 @@
+// Example: Test learning rate schedulers (StepLR and LambdaLR) with mini-jstorch optimizers
+
+import { SGD, StepLR, LambdaLR, Tensor } from "../src/MainEngine.js";
+
+const param = { param: [[1, 2], [3, 4]], grad: [[0, 0], [0, 0]] };
+const optimizer = new SGD([param], 0.1);
+
+// --- Test StepLR ---
+console.log("Testing StepLR...");
+const stepScheduler = new StepLR(optimizer, 3, 0.5);
+for (let epoch = 1; epoch <= 10; epoch++) {
+    stepScheduler.step();
+    console.log(`Epoch ${epoch}: LR = ${optimizer.lr.toFixed(4)}`);
+}
+
+// --- Test LambdaLR ---
+console.log("\nTesting LambdaLR...");
+optimizer.lr = 0.1; // Reset LR
+const lambdaScheduler = new LambdaLR(optimizer, epoch => 1.0 / (1 + epoch));
+for (let epoch = 1; epoch <= 5; epoch++) {
+    lambdaScheduler.step();
+    console.log(`Epoch ${epoch}: LR = ${optimizer.lr.toFixed(4)}`);
+}

package/MODULE.md

@@ -1,46 +0,0 @@
-## MODULE STATS ##
-
-New Files that automatically will All notable changes status state to *Mini-JSTorch* will be documented in this file.
-
-# MSG
-
-btw. This files is actually would be modified and note all changes system state automatically without manual i type with myself.
-
----
-
-**OFFICIAL RELEASE:** 2025-Monday-August-23 time: 2:22 AM (estimated time release)
-**VERSION:** 1.3.0
-**LICENSE:** MIT © 2025
-**AUTHOR:** Rizal
-**MODULE NAME:** mini-jstorch
-**MODULE DESC:** A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices Inspired by PyTorch.
-**MODULE TYPE:** module
-**ENGINE VERSIONS:** 1.2.0
-**UPDATE TITLE:** `MAJOR` update.
-**ADDED FILES/FOLDER:** {
-    "tests" //folder
-    "tests/proj1.js" //files
-    "tests/tests.js" //files [npmignore detected]
-    "tests/proj2.js" //files
-    "tests/proj3.js" //files
-    "N/A" //N/A [N/A]
-}
-
----
-
-**MODIFIED FILES:** {
-    "src" //folder
-    "src/startup.cpu" //files
-    "src/MainEngine.js" //files
-    "tests/tests.js" //files [npmignore detected]
-    "tests" //folder
-    "src/Dummy/exp.js" //files [npmignore detected]
-    "package.json" //files
-    "src/EngState.json" //files [npmignore detected]
-    "src/state.txt" //files [npmignore detected]
-    "README.md" //files
-    ".npmignore" //files [npmignore detected]
-    "N/A" //N/A [N/A]
-}
-
----

package/tests/proj1.js

@@ -1,85 +0,0 @@
-// TEST JSTORCH WHOLE SYSTEMS AT ONCE
-import { Tensor, Linear, Sequential, ReLU, Sigmoid, Tanh, LeakyReLU, GELU, Dropout, Conv2D, MSELoss, CrossEntropyLoss, Adam, SGD, saveModel, loadModel, flattenBatch, reshape, stack, concat, eye } from '../src/MainEngine.js';
-
-// ---------------------- Linear Test ----------------------
-console.log("=== Linear Test ===");
-const lin = new Linear(3,2);
-const linInput = [[1,2,3],[4,5,6]];
-const linOut = lin.forward(linInput);
-console.log("Linear forward:", linOut);
-const linGrad = [[0.1,0.2],[0.3,0.4]];
-const linBack = lin.backward(linGrad);
-console.log("Linear backward gradInput:", linBack);
-
-// ---------------------- Sequential + Activations Test ----------------------
-console.log("\n=== Sequential + Activations Test ===");
-const model = new Sequential([new Linear(2,2), new ReLU(), new Linear(2,1), new Sigmoid()]);
-const seqInput = [[0.5,1.0],[1.5,2.0]];
-const seqOut = model.forward(seqInput);
-console.log("Sequential forward:", seqOut);
-const seqGrad = [[0.1],[0.2]];
-const seqBack = model.backward(seqGrad);
-console.log("Sequential backward gradInput:", seqBack);
-
-// ---------------------- Conv2D Test ----------------------
-console.log("\n=== Conv2D Test ===");
-const conv = new Conv2D(1,1,3);
-const convInput = [[[ [1,2,3],[4,5,6],[7,8,9] ]]]; // batch=1, inC=1, HxW=3x3
-const convOut = conv.forward(convInput);
-console.log("Conv2D forward:", convOut);
-
-// Conv2D backward test
-const convGrad = [[[ [0.1,0.2,0.1],[0.2,0.3,0.2],[0.1,0.2,0.1] ]]];
-const convBack = conv.backward(convGrad);
-console.log("Conv2D backward gradInput:", convBack);
-
-// ---------------------- Tensor & Broadcast Test ----------------------
-console.log("\n=== Tensor & Broadcast Test ===");
-const a = Tensor.random(2,3);
-const b = Tensor.ones(2,3);
-const sum = a.add(b);
-console.log("Tensor add broadcast:", sum);
-
-// ---------------------- Loss + Optimizer Test ----------------------
-console.log("\n=== Loss + Optimizer Test ===");
-const lossModel = new Sequential([new Linear(2,2)]);
-const pred = lossModel.forward([[1,2]]);
-const target = [[0,1]];
-const ceLoss = new CrossEntropyLoss();
-const lval = ceLoss.forward(pred,target);
-console.log("CrossEntropyLoss value:", lval);
-
-const gradLoss = ceLoss.backward();
-lossModel.backward(gradLoss);
-
-const opt = new Adam(lossModel.parameters());
-opt.step();
-console.log("Updated parameters after Adam:", lossModel.parameters());
-
-// ---------------------- Dropout Test ----------------------
-console.log("\n=== Dropout Test ===");
-const drop = new Dropout(0.5);
-const dropInput = [[1,2],[3,4]];
-const dropOut = drop.forward(dropInput);
-console.log("Dropout forward:", dropOut);
-const dropBack = drop.backward([[0.1,0.2],[0.3,0.4]]);
-console.log("Dropout backward:", dropBack);
-
-// ---------------------- Save / Load Model Test ----------------------
-console.log("\n=== Save / Load Model Test ===");
-const modelSave = new Sequential([new Linear(2,2)]);
-const json = saveModel(modelSave);
-console.log("Saved model JSON:", json);
-const modelLoad = new Sequential([new Linear(2,2)]);
-loadModel(modelLoad,json);
-console.log("Loaded model parameters:", modelLoad.parameters());
-
-// ---------------------- Advanced Utils Test ----------------------
-console.log("\n=== Advanced Utils Test ===");
-const batch = [[[1,2],[3,4]],[[5,6],[7,8]]];
-console.log("Flatten batch:", flattenBatch(batch));
-console.log("Eye 3:", eye(3));
-console.log("Reshape:", reshape({data:[[1,2,3,4]]},2,2));
-console.log("Stack:", stack([Tensor.ones(2,2), Tensor.zeros(2,2)]));
-console.log("Concat axis0:", concat([[1,2],[3,4]], [[5,6],[7,8]], 0));
-console.log("Concat axis1:", concat([[1,2],[3,4]], [[5,6],[7,8]], 1));

package/tests/proj2.js

@@ -1,129 +0,0 @@
-/**
- * Mini-JSTorch Next Word Prediction Test (Self-contained Softmax)
- * - Softmax function defined in this file
- * - Beam search prediction
- * - Large vocab & diverse sentences
- * - Sequence length 2
- * - Training loop 2000 epochs
- */
-
-import { Sequential, Linear, ReLU, CrossEntropyLoss, Adam } from "../src/MainEngine.js";
-
-// ------------------------
-// Softmax Function
-// ------------------------
-function softmaxVector(logits) {
-  const maxVal = Math.max(...logits);
-  const exps = logits.map(v => Math.exp(v - maxVal));
-  const sumExps = exps.reduce((a,b)=>a+b, 0);
-  return exps.map(v => v/sumExps);
-}
-
-// ------------------------
-// Vocabulary & Tokenization
-// ------------------------
-const vocab = [
-  "i","you","he","she","we","they",
-  "like","love","hate","pizza","coding","game","movie","music","coffee","tea",
-  "run","walk","play","read","book","eat","drink","watch","listen","reads",
-  "drink","drinks"
-];
-
-const word2idx = Object.fromEntries(vocab.map((w,i)=>[w,i]));
-const idx2word = Object.fromEntries(vocab.map((w,i)=>[i,w]));
-
-function oneHot(index, vocabSize) {
-  return Array.from({length:vocabSize}, (_,i)=>i===index?1:0);
-}
-
-// ------------------------
-// Dataset (sequence length 2)
-// ------------------------
-const sentences = [
-  ["i","like","pizza"], ["i","like","coding"], ["i","love","music"],
-  ["i","read","book"], ["i","watch","movie"], ["you","like","pizza"],
-  ["you","love","coffee"], ["you","read","book"], ["you","play","game"],
-  ["he","hate","coffee"], ["he","like","music"], ["he","play","game"],
-  ["she","love","tea"], ["she","read","book"], ["she","watch","movie"],
-  ["we","play","game"], ["we","read","book"], ["we","love","coffee"],
-  ["they","eat","pizza"], ["they","play","game"], ["they","listen","music"],
-  ["i","drink","coffee"], ["he","drink","tea"], ["she","drink","coffee"],
-  ["we","drink","tea"], ["they","drink","coffee"], ["i","play","game"],
-  ["you","watch","movie"], ["he","read","book"], ["she","listen","music"],
-  ["he","reads","book"], ["you","read","book"], ["they","read","book"],
-  ["they","watch","movie"], ["we","listen","music"], ["we","watch","movie"],
-  ["we","reads","book"],["we","drinks","coffee"],["we","love","you"],
-  ["i","read","book"], ["i","love","you"]
-];
-
-// Convert to input-output pairs
-const X = [], Y = [];
-const seqLength = 2;
-for(const s of sentences){
-  for(let i=0;i<=s.length-seqLength;i++){
-    const inpSeq = s.slice(i,i+seqLength).map(w=>oneHot(word2idx[w],vocab.length)).flat();
-    const outWord = s[i+seqLength] ? oneHot(word2idx[s[i+seqLength]], vocab.length)
-                                    : oneHot(word2idx[s[i+seqLength-1]], vocab.length);
-    X.push(inpSeq);
-    Y.push(outWord);
-  }
-}
-
-// ------------------------
-// Model Definition
-// ------------------------
-const model = new Sequential([
-  new Linear(vocab.length*seqLength, 128),
-  new ReLU(),
-  new Linear(128, vocab.length)
-]);
-
-const lossFn = new CrossEntropyLoss();
-const optimizer = new Adam(model.parameters(), 0.01);
-
-// ------------------------
-// Training Loop
-// ------------------------
-for(let epoch=1; epoch<=1600; epoch++){
-  const pred = model.forward(X);
-  const loss = lossFn.forward(pred, Y);
-  const grad = lossFn.backward();
-  model.backward(grad);
-  optimizer.step();
-
-  if(epoch % 500 === 0) console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}`);
-}
-
-// ------------------------
-// Beam Search Prediction
-// ------------------------
-function beamSearch(inputWords, beamWidth=2, predLength=3){
-  let sequences = [{words:[...inputWords], score:1}];
-  for(let step=0; step<predLength; step++){
-    const allCandidates = [];
-    for(const seq of sequences){
-      const inp = seq.words.slice(-seqLength).map(w=>oneHot(word2idx[w],vocab.length)).flat();
-      const out = model.forward([inp])[0];
-      const probs = softmaxVector(out); // softmax applied here
-      const top = probs.map((v,i)=>[i,v]).sort((a,b)=>b[1]-a[1]).slice(0,beamWidth);
-      for(const [idx,score] of top){
-        allCandidates.push({words:[...seq.words, idx2word[idx]], score:seq.score*score});
-      }
-    }
-    sequences = allCandidates.sort((a,b)=>b.score-a.score).slice(0,beamWidth);
-  }
-  return sequences.map(s=>({sequence:s.words, score:s.score.toFixed(3)}));
-}
-
-// ------------------------
-// Test Predictions
-// ------------------------
-const testInputs = [
-  ["i","like"], ["you","love"], ["they","play"], ["he","hate"], ["she","reads"]
-];
-
-for(const inp of testInputs){
-  const results = beamSearch(inp, 2, 3); // beam width 2, predict next 3 words
-  console.log(`Input: ${inp.join(" ")}`);
-  results.forEach(r=>console.log(`  Sequence: ${r.sequence.join(" ")}, Score: ${r.score}`));
-}

package/tests/proj3.js

@@ -1,76 +0,0 @@
-// JSTORCH TESTS TEMPLATE FILES
-// CIRCLE CLASSIFICATION TESTS FILES
-import { Sequential, Linear, ReLU, Sigmoid, CrossEntropyLoss, Adam } from '../src/MainEngine.js';
-
-// === Generate Circle Dataset ===
-function generateCircleData(n) {
-  const X = [], Y = [];
-  for (let i = 0; i < n; i++) {
-    const x = Math.random() * 2 - 1; // -1 to 1
-    const y = Math.random() * 2 - 1;
-    const label = (x*x + y*y < 0.5*0.5) ? [1,0] : [0,1]; // inside circle radius 0.5
-    X.push([x, y]);
-    Y.push(label);
-  }
-  return { X, Y };
-}
-
-const { X, Y } = generateCircleData(300);
-
-// === Build Model (bigger hidden layers) ===
-const model = new Sequential([
-  new Linear(2, 16),
-  new ReLU(),
-  new Linear(16, 8),
-  new ReLU(),
-  new Linear(8, 2),
-  new Sigmoid()
-]);
-
-// Loss & Optimizer
-const lossFn = new CrossEntropyLoss();
-const optimizer = new Adam(model.parameters(), 0.01); // smaller LR for not causing the model get stuck.
-
-// === Training ===
-console.log("=== Circle Classification Training (Fixed) ===");
-const start = Date.now();
-for (let epoch = 1; epoch <= 2000; epoch++) {
-  const pred = model.forward(X);
-  const loss = lossFn.forward(pred, Y);
-  const gradLoss = lossFn.backward();
-  model.backward(gradLoss);
-  optimizer.step();
-
-  if (epoch % 500 === 0) {
-    console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}`);
-  }
-}
-console.log(`Training Time: ${((Date.now()-start)/1000).toFixed(3)}s`); // FOR real time monitoring while training if this module is literally lightweight and does not take a long time to train.
-
-// === Predictions ===
-console.log("\n=== Predictions ===");
-const testInputs = [
-  [0,0],
-  [0.7,0.7],
-  [0.2,0.2],
-  [0.9,0.1],
-  [-0.5,-0.5],
-  [0.6,-0.2]
-];
-testInputs.forEach(inp => {
-  const out = model.forward([inp])[0];
-  const predClass = out.indexOf(Math.max(...out));
-  console.log(
-    `Input: ${inp}, Pred: ${predClass}, Raw: ${out.map(v => v.toFixed(3))}`
-  );
-});
-
-/**
- * === How to Run On Your VS Code Projects ===
- * 1. Make sure Node.js (v20+ recommended) is installed on your system.
- * 2. Open this file in VS Code (or any editor).
- * 3. Open the terminal in the project root folder.
- * 4. Run: node tests/proj3.js
- *
- * You should see the training logs and prediction outputs directly.
- */