mini-jstorch 1.2.2 → 1.3.0

package/MODULE.md

@@ -0,0 +1,46 @@
+## MODULE STATS ##
+
+New Files that automatically will All notable changes status state to *Mini-JSTorch* will be documented in this file.
+
+# MSG
+
+btw. This files is actually would be modified and note all changes system state automatically without manual i type with myself.
+
+---
+
+**OFFICIAL RELEASE:** 2025-Monday-August-23 time: 2:22 AM (estimated time release)
+**VERSION:** 1.3.0
+**LICENSE:** MIT © 2025
+**AUTHOR:** Rizal
+**MODULE NAME:** mini-jstorch
+**MODULE DESC:** A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices Inspired by PyTorch.
+**MODULE TYPE:** module
+**ENGINE VERSIONS:** 1.2.0
+**UPDATE TITLE:** `MAJOR` update.
+**ADDED FILES/FOLDER:** {
+    "tests" //folder
+    "tests/proj1.js" //files
+    "tests/tests.js" //files [npmignore detected]
+    "tests/proj2.js" //files
+    "tests/proj3.js" //files
+    "N/A" //N/A [N/A]
+}
+
+---
+
+**MODIFIED FILES:** {
+    "src" //folder
+    "src/startup.cpu" //files
+    "src/MainEngine.js" //files
+    "tests/tests.js" //files [npmignore detected]
+    "tests" //folder
+    "src/Dummy/exp.js" //files [npmignore detected]
+    "package.json" //files
+    "src/EngState.json" //files [npmignore detected]
+    "src/state.txt" //files [npmignore detected]
+    "README.md" //files
+    ".npmignore" //files [npmignore detected]
+    "N/A" //N/A [N/A]
+}
+
+---

package/README.md

@@ -1,17 +1,136 @@
-# MINI JSTORCH #
+# Mini-JSTorch Major Update
 
----
-
-
-**Just Empty.. this versions only an note do not use it for your projects models.**
-**I ONLY WANNA GIVE A MESSAGES IF THE MAJOR UPDATE WOULD BE COMING AT TIME 2:22 AM JUST BE READY FOR IT GUYS!**
+# MAJOR UPDATE IS RELEASED NOW!! [got little late] 
 
-# HUGE THANKS 🎉 #
+---
 
+## 🙏 Big Thanks 🙏
 I genuinely appreciate everyone who has taken the time to install and try out this module. Every download isn’t just a number it’s a real person spending their time to explore, learn, or build something with this module!
 
 Reaching 400+ weekly downloads might sound small compared to giant frameworks, but for me it’s huge. It means this project is actually helping people out there, and that alone makes every late-night coding session worth it.
 
 Thank you for giving this module a chance. Your support and trust are what keep this project moving forward! 🚀
 
-# BE READY FOR IT... SET YOUR TIME CLOCK. #
+---
+
+## Overview
+
+Mini-JSTorch is a lightweight, high-performance JavaScript library for building neural networks that runs efficiently in both frontend and backend environments, including low-end devices. The library enables experimentation and learning in AI without compromising stability, accuracy, or training reliability.
+
+This release, **version 1.2.3**, is a **major update** that introduces full feature coverage, including convolutional layers, advanced activations, dropout, tensor broadcasting, and enhanced utilities. The engine remains pure JavaScript, lightweight, and compatible with both Node.js and browser environments.
+
+---
+
+## Major Updates in 1.2.3
+
+- Full **Conv2D support** with forward and backward operations.
+- **Tensor operations** now support broadcasting and reshaping.
+- Added new activations:  `LeakyReLU`, `GELU`.
+- Optimizers: `Adam` and `SGD` fully integrated with gradient updates.
+- Dropout layer added for regularization.
+- Advanced utilities for tensor manipulation: `flatten`, `stack`, `concat`, `eye`, `reshape`.
+- End-to-end training and prediction workflow now fully tested.
+- Save and load model functionality included for seamless persistence.
+- Optimized performance for both frontend and backend usage.
+- Maintained backward compatibility for previous core layers and activations.
+- folder tests files template that you can use it before make your models.
+
+---
+
+## Features
+
+- **Core Layers**: Linear, Conv2D  
+- **Activations**: ReLU, Sigmoid, Tanh, LeakyReLU, GELU  
+- **Loss Functions**: MSELoss, CrossEntropyLoss  
+- **Optimizers**: Adam, SGD  
+- **Utilities**: zeros, randomMatrix, softmax, crossEntropy, dot, addMatrices, reshape, stack, flatten, eye, concat  
+- **Model Container**: Sequential (for stacking layers with forward/backward passes)  
+
+---
+
+## Installation
+
+```bash
+npm install mini-jstorch
+# Node.js v20+ recommended for best performance
+```
+
+---
+
+## Quick Start Example
+
+```javascript
+import { Sequential, Linear, ReLU, Sigmoid, CrossEntropyLoss, Adam } from './src/MainEngine.js';
+
+// Build model
+const model = new Sequential([
+  new Linear(2,4),
+  new ReLU(),
+  new Linear(4,2),
+  new Sigmoid()
+]);
+
+// Sample XOR dataset
+const X = [
+  [0,0], [0,1], [1,0], [1,1]
+];
+const Y = [
+  [1,0], [0,1], [0,1], [1,0]
+];
+
+// Loss & optimizer
+const lossFn = new CrossEntropyLoss();
+const optimizer = new Adam(model.parameters(), 0.1);
+
+// Training loop
+for (let epoch = 1; epoch <= 100; epoch++) {
+  const pred = model.forward(X);
+  const loss = lossFn.forward(pred, Y);
+  const gradLoss = lossFn.backward();
+  model.backward(gradLoss);
+  optimizer.step();
+  if (epoch % 20 === 0) console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}`);
+}
+
+// Prediction
+const predTest = model.forward(X);
+predTest.forEach((p,i) => {
+  const predictedClass = p.indexOf(Math.max(...p));
+  console.log(`Input: ${X[i]}, Predicted class: ${predictedClass}, Raw output: ${p.map(v => v.toFixed(3))}`);
+});
+```
+
+---
+
+## Save & Load Models
+
+```javascript
+import { saveModel, loadModel } from 'mini-jstorch';
+
+const json = saveModel(model);
+const model2 = new Sequential([...]); // same architecture
+loadModel(model2, json);
+```
+
+---
+
+## Intended Use Cases
+ - Rapid prototyping of neural networks in frontend and backend.
+ - Learning and teaching foundational neural network concepts.
+ - Experimentation on low-end devices or mobile browsers.
+ - Lightweight AI projects without GPU dependency.
+
+---
+
+# License
+
+**MIT © 2025 Rizal**
+
+---
+
+## Facts
+
+- **This module is implemented entirely in pure JavaScript.**
+- **The `Dummy` folder contains modules used for development, testing, and debugging before integration into the main engine.**
+- **files startup.cpu is actually an some random files lol.**
+- **This module was created by a `single` developer.**

package/package.json

@@ -1,13 +1,16 @@
 {
   "name": "mini-jstorch",
-  "version": "1.2.2",
+  "version": "1.3.0",
   "type": "module", 
   "description": "A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices Inspired by PyTorch.",
   "main": "index.js",
   "keywords": [
     "neural-network",
     "javascript",
+    "lightweight-torch",
     "lightweight",
+    "small",
+    "javascript-torch",
     "ai",
     "jstorch",
     "pytorch",

package/src/MainEngine.js

@@ -1 +1,255 @@
-console.log("MAJOR UPDATE IN 2:22 AM BE READY GUYS! this was empty because this versions only for an major note.")
+// MINI JSTORCH ENGINE - MAJOR ULTRA OPTIMIZED 1.2.3
+// LICENSE: MIT (C) Rizal 2025
+
+// ---------------------- Utilities ----------------------
+function zeros(rows, cols) { return Array.from({length:rows},()=>Array(cols).fill(0)); }
+function ones(rows, cols) { return Array.from({length:rows},()=>Array(cols).fill(1)); }
+function randomMatrix(rows, cols, scale=0.1){ return Array.from({length:rows},()=>Array.from({length:cols},()=> (Math.random()*2-1)*scale)); }
+function transpose(matrix){ return matrix[0].map((_,i)=>matrix.map(row=>row[i])); }
+function addMatrices(a,b){ return a.map((row,i)=>row.map((v,j)=>v+(b[i] && b[i][j]!==undefined?b[i][j]:0))); }
+function dot(a,b){ const res=zeros(a.length,b[0].length); for(let i=0;i<a.length;i++) for(let j=0;j<b[0].length;j++) for(let k=0;k<a[0].length;k++) res[i][j]+=a[i][k]*b[k][j]; return res; }
+function softmax(x){ const m=Math.max(...x); const exps=x.map(v=>Math.exp(v-m)); const s=exps.reduce((a,b)=>a+b,0); return exps.map(v=>v/s); }
+function crossEntropy(pred,target){ const eps=1e-12; return -target.reduce((sum,t,i)=>sum+t*Math.log(pred[i]+eps),0); }
+
+// ---------------------- Tensor ----------------------
+export class Tensor {
+    constructor(data){ this.data=data; this.grad=zeros(data.length,data[0].length); }
+    shape(){ return [this.data.length,this.data[0].length]; }
+    add(t){ return t instanceof Tensor?this.data.map((r,i)=>r.map((v,j)=>v+t.data[i][j])):this.data.map(r=>r.map(v=>v+t)); }
+    sub(t){ return t instanceof Tensor?this.data.map((r,i)=>r.map((v,j)=>v-t.data[i][j])):this.data.map(r=>r.map(v=>v-t)); }
+    mul(t){ return t instanceof Tensor?this.data.map((r,i)=>r.map((v,j)=>v*t.data[i][j])):this.data.map(r=>r.map(v=>v*t)); }
+    matmul(t){ if(t instanceof Tensor) return dot(this.data,t.data); else throw new Error("matmul requires Tensor"); }
+    transpose(){ return transpose(this.data); }
+    flatten(){ return this.data.flat(); }
+    static zeros(r,c){ return new Tensor(zeros(r,c)); }
+    static ones(r,c){ return new Tensor(ones(r,c)); }
+    static random(r,c,scale=0.1){ return new Tensor(randomMatrix(r,c,scale)); }
+}
+
+// ---------------------- Layers ----------------------
+export class Linear {
+    constructor(inputDim,outputDim){
+        this.W=randomMatrix(inputDim,outputDim);
+        this.b=Array(outputDim).fill(0);
+        this.gradW=zeros(inputDim,outputDim);
+        this.gradb=Array(outputDim).fill(0);
+        this.x=null;
+    }
+
+    forward(x){
+        this.x=x;
+        const out=dot(x,this.W);
+        return out.map((row,i)=>row.map((v,j)=>v+this.b[j]));
+    }
+
+    backward(grad){
+        for(let i=0;i<this.W.length;i++) for(let j=0;j<this.W[0].length;j++)
+            this.gradW[i][j]=this.x.reduce((sum,row,k)=>sum+row[i]*grad[k][j],0);
+        for(let j=0;j<this.b.length;j++)
+            this.gradb[j]=grad.reduce((sum,row)=>sum+row[j],0);
+
+        const gradInput=zeros(this.x.length,this.W.length);
+        for(let i=0;i<this.x.length;i++)
+            for(let j=0;j<this.W.length;j++)
+                for(let k=0;k<this.W[0].length;k++)
+                    gradInput[i][j]+=grad[i][k]*this.W[j][k];
+        return gradInput;
+    }
+
+    parameters(){ return [ {param:this.W,grad:this.gradW}, {param:[this.b],grad:[this.gradb]} ]; }
+}
+
+// ---------------------- Conv2D ----------------------
+export class Conv2D {
+    constructor(inC,outC,kernel,stride=1,padding=0){
+        this.inC=inC; this.outC=outC; this.kernel=kernel;
+        this.stride=stride; this.padding=padding;
+        this.W=Array(outC).fill(0).map(()=>Array(inC).fill(0).map(()=>randomMatrix(kernel,kernel)));
+        this.gradW=Array(outC).fill(0).map(()=>Array(inC).fill(0).map(()=>zeros(kernel,kernel)));
+        this.x=null;
+    }
+
+    pad2D(input,pad){
+        return input.map(channel=>{
+            const rows=channel.length+2*pad;
+            const cols=channel[0].length+2*pad;
+            const out=Array.from({length:rows},()=>Array(cols).fill(0));
+            for(let i=0;i<channel.length;i++) for(let j=0;j<channel[0].length;j++) out[i+pad][j+pad]=channel[i][j];
+            return out;
+        });
+    }
+
+    conv2DSingle(input,kernel){
+        const rows=input.length-kernel.length+1;
+        const cols=input[0].length-kernel[0].length+1;
+        const out=zeros(rows,cols);
+        for(let i=0;i<rows;i++) for(let j=0;j<cols;j++)
+            for(let ki=0;ki<kernel.length;ki++) for(let kj=0;kj<kernel[0].length;kj++)
+                out[i][j]+=input[i+ki][j+kj]*kernel[ki][kj];
+        return out;
+    }
+
+    forward(batch){
+        this.x=batch;
+        return batch.map(sample=>{
+            const channelsOut=[];
+            for(let oc=0;oc<this.outC;oc++){
+                let outChan=zeros(sample[0].length,sample[0][0].length);
+                for(let ic=0;ic<this.inC;ic++){
+                    let inputChan=sample[ic];
+                    if(this.padding>0) inputChan=this.pad2D([inputChan],this.padding)[0];
+                    const conv=this.conv2DSingle(inputChan,this.W[oc][ic]);
+                    outChan=addMatrices(outChan,conv);
+                }
+                channelsOut.push(outChan);
+            }
+            return channelsOut;
+        });
+    }
+
+    backward(grad) {
+    const batchSize = this.x.length;
+    const gradInput = this.x.map(sample => sample.map(chan => zeros(chan.length, chan[0].length)));
+    const gradW = this.W.map(oc => oc.map(ic => zeros(this.kernel,this.kernel)));
+
+    for (let b = 0; b < batchSize; b++) {
+        const xPadded = this.pad2D(this.x[b], this.padding);
+        const gradInputPadded = xPadded.map(chan => zeros(chan.length, chan[0].length));
+
+        for (let oc = 0; oc < this.outC; oc++) {
+            for (let ic = 0; ic < this.inC; ic++) {
+                const outGrad = grad[b][oc];
+                const inChan = xPadded[ic];
+
+                // Compute gradW
+                for (let i = 0; i < this.kernel; i++) {
+                    for (let j = 0; j < this.kernel; j++) {
+                        let sum = 0;
+                        for (let y = 0; y < outGrad.length; y++) {
+                            for (let x = 0; x < outGrad[0].length; x++) {
+                                const inY = y * this.stride + i;
+                                const inX = x * this.stride + j;
+                                if (inY < inChan.length && inX < inChan[0].length) {
+                                    sum += inChan[inY][inX] * outGrad[y][x];
+                                }
+                            }
+                        }
+                        gradW[oc][ic][i][j] += sum;
+                    }
+                }
+
+                // Compute gradInput
+                const flippedKernel = this.W[oc][ic].map(row => [...row].reverse()).reverse();
+                for (let y = 0; y < outGrad.length; y++) {
+                    for (let x = 0; x < outGrad[0].length; x++) {
+                        for (let i = 0; i < this.kernel; i++) {
+                            for (let j = 0; j < this.kernel; j++) {
+                                const inY = y * this.stride + i;
+                                const inX = x * this.stride + j;
+                                if (inY < gradInputPadded[ic].length && inX < gradInputPadded[ic][0].length) {
+                                    gradInputPadded[ic][inY][inX] += flippedKernel[i][j] * outGrad[y][x];
+                                }
+                            }
+                        }
+                    }
+                }
+            }
+        }
+
+        // Remove padding from gradInput
+        if (this.padding > 0) {
+            for (let ic = 0; ic < this.inC; ic++) {
+                const padded = gradInputPadded[ic];
+                const cropped = padded.slice(this.padding, padded.length - this.padding)
+                    .map(row => row.slice(this.padding, row.length - this.padding));
+                gradInput[b][ic] = cropped;
+            }
+        } else {
+            for (let ic = 0; ic < this.inC; ic++) gradInput[b][ic] = gradInputPadded[ic];
+        }
+    }
+
+    this.gradW = gradW;
+    return gradInput;
+}
+
+    parameters(){ return this.W.flatMap((w,oc)=>w.map((wc,ic)=>({param:wc,grad:this.gradW[oc][ic]}))); }
+}
+
+// ---------------------- Sequential ----------------------
+export class Sequential {
+    constructor(layers=[]){ this.layers=layers; }
+    forward(x){ return this.layers.reduce((acc,l)=>l.forward(acc), x); }
+    backward(grad){ return this.layers.reduceRight((g,l)=>l.backward(g), grad); }
+    parameters(){ return this.layers.flatMap(l=>l.parameters?l.parameters():[]); }
+}
+
+// ---------------------- Activations ----------------------
+export class ReLU{ constructor(){ this.out=null; } forward(x){ this.out=x.map(r=>r.map(v=>Math.max(0,v))); return this.out; } backward(grad){ return grad.map((r,i)=>r.map((v,j)=>v*(this.out[i][j]>0?1:0))); } }
+export class Sigmoid{ constructor(){ this.out=null; } forward(x){ const fn=v=>1/(1+Math.exp(-v)); this.out=x.map(r=>r.map(fn)); return this.out; } backward(grad){ return grad.map((r,i)=>r.map((v,j)=>v*this.out[i][j]*(1-this.out[i][j]))); } }
+export class Tanh{ constructor(){ this.out=null; } forward(x){ this.out=x.map(r=>r.map(v=>Math.tanh(v))); return this.out; } backward(grad){ return grad.map((r,i)=>r.map((v,j)=>v*(1-this.out[i][j]**2))); } }
+export class LeakyReLU{ constructor(alpha=0.01){ this.alpha=alpha; this.out=null; } forward(x){ this.out=x.map(r=>r.map(v=>v>0?v:v*this.alpha)); return this.out; } backward(grad){ return grad.map((r,i)=>r.map((v,j)=>v*(this.out[i][j]>0?1:this.alpha))); } }
+export class GELU{ constructor(){ this.out=null; } forward(x){ const fn=v=>0.5*v*(1+Math.tanh(Math.sqrt(2/Math.PI)*(v+0.044715*v**3))); this.out=x.map(r=>r.map(fn)); return this.out; } backward(grad){ return grad.map((r,i)=>r.map(v=>v*1)); } }
+
+// ---------------------- Dropout ----------------------
+export class Dropout{ constructor(p=0.5){ this.p=p; } forward(x){ return x.map(r=>r.map(v=>v*Math.random()>=this.p?v:0)); } backward(grad){ return grad.map(r=>r.map(v=>v*(1-this.p))); } }
+
+// ---------------------- Losses ----------------------
+export class MSELoss{ forward(pred,target){ this.pred=pred; this.target=target; const losses=pred.map((row,i)=>row.reduce((sum,v,j)=>sum+(v-target[i][j])**2,0)/row.length); return losses.reduce((a,b)=>a+b,0)/pred.length; } backward(){ return this.pred.map((row,i)=>row.map((v,j)=>2*(v-this.target[i][j])/row.length)); } }
+export class CrossEntropyLoss{ forward(pred,target){ this.pred=pred; this.target=target; const losses=pred.map((p,i)=>crossEntropy(softmax(p),target[i])); return losses.reduce((a,b)=>a+b,0)/pred.length; } backward(){ return this.pred.map((p,i)=>{ const s=softmax(p); return s.map((v,j)=>(v-this.target[i][j])/this.pred.length); }); } }
+
+// ---------------------- Optimizers ----------------------
+export class Adam{
+    constructor(params,lr=0.001,b1=0.9,b2=0.999,eps=1e-8){
+        this.params=params; this.lr=lr; this.beta1=b1; this.beta2=b2; this.eps=eps;
+        this.m=params.map(p=>zeros(p.param.length,p.param[0].length||1));
+        this.v=params.map(p=>zeros(p.param.length,p.param[0].length||1));
+        this.t=0;
+    }
+    step(){
+        this.t++;
+        this.params.forEach((p,idx)=>{
+            for(let i=0;i<p.param.length;i++)
+                for(let j=0;j<(p.param[0].length||1);j++){
+                    const g=p.grad[i][j];
+                    this.m[idx][i][j]=this.beta1*this.m[idx][i][j]+(1-this.beta1)*g;
+                    this.v[idx][i][j]=this.beta2*this.v[idx][i][j]+(1-this.beta2)*g*g;
+                    const mHat=this.m[idx][i][j]/(1-Math.pow(this.beta1,this.t));
+                    const vHat=this.v[idx][i][j]/(1-Math.pow(this.beta2,this.t));
+                    p.param[i][j]-=this.lr*mHat/(Math.sqrt(vHat)+this.eps);
+                }
+        });
+    }
+}
+
+export class SGD{ constructor(params,lr=0.01){ this.params=params; this.lr=lr; } step(){ this.params.forEach(p=>{ for(let i=0;i<p.param.length;i++) for(let j=0;j<(p.param[0].length||1);j++) p.param[i][j]-=this.lr*p.grad[i][j]; }); } }
+
+// ---------------------- Model Save/Load ----------------------
+export function saveModel(model){
+    if(!(model instanceof Sequential)) throw new Error("saveModel supports only Sequential");
+    const weights=model.layers.map(layer=>({weights:layer.W||null,biases:layer.b||null}));
+    return JSON.stringify(weights);
+}
+
+export function loadModel(model,json){
+    if(!(model instanceof Sequential)) throw new Error("loadModel supports only Sequential");
+    const weights=JSON.parse(json);
+    model.layers.forEach((layer,i)=>{
+        if(layer.W && weights[i].weights) layer.W=weights[i].weights;
+        if(layer.b && weights[i].biases) layer.b=weights[i].biases;
+    });
+}
+
+// ---------------------- Advanced Utils ----------------------
+export function flattenBatch(batch){ return batch.flat(2); }
+export function stack(tensors){ return tensors.map(t=>t.data); }
+export function eye(n){ return Array.from({length:n},(_,i)=>Array.from({length:n},(_,j)=>i===j?1:0)); }
+export function concat(a,b,axis=0){ /* concat along axis */ if(axis===0) return [...a,...b]; if(axis===1) return a.map((row,i)=>[...row,...b[i]]); }
+export function reshape(tensor, rows, cols) {
+    let flat = tensor.data.flat(); // flatten dulu
+    if(flat.length < rows*cols) throw new Error("reshape size mismatch");
+    const out = Array.from({length: rows}, (_, i) =>
+        flat.slice(i*cols, i*cols + cols)
+    );
+    return out;
+}

package/tests/proj1.js

@@ -0,0 +1,85 @@
+// TEST JSTORCH WHOLE SYSTEMS AT ONCE
+import { Tensor, Linear, Sequential, ReLU, Sigmoid, Tanh, LeakyReLU, GELU, Dropout, Conv2D, MSELoss, CrossEntropyLoss, Adam, SGD, saveModel, loadModel, flattenBatch, reshape, stack, concat, eye } from '../src/MainEngine.js';
+
+// ---------------------- Linear Test ----------------------
+console.log("=== Linear Test ===");
+const lin = new Linear(3,2);
+const linInput = [[1,2,3],[4,5,6]];
+const linOut = lin.forward(linInput);
+console.log("Linear forward:", linOut);
+const linGrad = [[0.1,0.2],[0.3,0.4]];
+const linBack = lin.backward(linGrad);
+console.log("Linear backward gradInput:", linBack);
+
+// ---------------------- Sequential + Activations Test ----------------------
+console.log("\n=== Sequential + Activations Test ===");
+const model = new Sequential([new Linear(2,2), new ReLU(), new Linear(2,1), new Sigmoid()]);
+const seqInput = [[0.5,1.0],[1.5,2.0]];
+const seqOut = model.forward(seqInput);
+console.log("Sequential forward:", seqOut);
+const seqGrad = [[0.1],[0.2]];
+const seqBack = model.backward(seqGrad);
+console.log("Sequential backward gradInput:", seqBack);
+
+// ---------------------- Conv2D Test ----------------------
+console.log("\n=== Conv2D Test ===");
+const conv = new Conv2D(1,1,3);
+const convInput = [[[ [1,2,3],[4,5,6],[7,8,9] ]]]; // batch=1, inC=1, HxW=3x3
+const convOut = conv.forward(convInput);
+console.log("Conv2D forward:", convOut);
+
+// Conv2D backward test
+const convGrad = [[[ [0.1,0.2,0.1],[0.2,0.3,0.2],[0.1,0.2,0.1] ]]];
+const convBack = conv.backward(convGrad);
+console.log("Conv2D backward gradInput:", convBack);
+
+// ---------------------- Tensor & Broadcast Test ----------------------
+console.log("\n=== Tensor & Broadcast Test ===");
+const a = Tensor.random(2,3);
+const b = Tensor.ones(2,3);
+const sum = a.add(b);
+console.log("Tensor add broadcast:", sum);
+
+// ---------------------- Loss + Optimizer Test ----------------------
+console.log("\n=== Loss + Optimizer Test ===");
+const lossModel = new Sequential([new Linear(2,2)]);
+const pred = lossModel.forward([[1,2]]);
+const target = [[0,1]];
+const ceLoss = new CrossEntropyLoss();
+const lval = ceLoss.forward(pred,target);
+console.log("CrossEntropyLoss value:", lval);
+
+const gradLoss = ceLoss.backward();
+lossModel.backward(gradLoss);
+
+const opt = new Adam(lossModel.parameters());
+opt.step();
+console.log("Updated parameters after Adam:", lossModel.parameters());
+
+// ---------------------- Dropout Test ----------------------
+console.log("\n=== Dropout Test ===");
+const drop = new Dropout(0.5);
+const dropInput = [[1,2],[3,4]];
+const dropOut = drop.forward(dropInput);
+console.log("Dropout forward:", dropOut);
+const dropBack = drop.backward([[0.1,0.2],[0.3,0.4]]);
+console.log("Dropout backward:", dropBack);
+
+// ---------------------- Save / Load Model Test ----------------------
+console.log("\n=== Save / Load Model Test ===");
+const modelSave = new Sequential([new Linear(2,2)]);
+const json = saveModel(modelSave);
+console.log("Saved model JSON:", json);
+const modelLoad = new Sequential([new Linear(2,2)]);
+loadModel(modelLoad,json);
+console.log("Loaded model parameters:", modelLoad.parameters());
+
+// ---------------------- Advanced Utils Test ----------------------
+console.log("\n=== Advanced Utils Test ===");
+const batch = [[[1,2],[3,4]],[[5,6],[7,8]]];
+console.log("Flatten batch:", flattenBatch(batch));
+console.log("Eye 3:", eye(3));
+console.log("Reshape:", reshape({data:[[1,2,3,4]]},2,2));
+console.log("Stack:", stack([Tensor.ones(2,2), Tensor.zeros(2,2)]));
+console.log("Concat axis0:", concat([[1,2],[3,4]], [[5,6],[7,8]], 0));
+console.log("Concat axis1:", concat([[1,2],[3,4]], [[5,6],[7,8]], 1));

package/tests/proj2.js

@@ -0,0 +1,129 @@
+/**
+ * Mini-JSTorch Next Word Prediction Test (Self-contained Softmax)
+ * - Softmax function defined in this file
+ * - Beam search prediction
+ * - Large vocab & diverse sentences
+ * - Sequence length 2
+ * - Training loop 2000 epochs
+ */
+
+import { Sequential, Linear, ReLU, CrossEntropyLoss, Adam } from "../src/MainEngine.js";
+
+// ------------------------
+// Softmax Function
+// ------------------------
+function softmaxVector(logits) {
+  const maxVal = Math.max(...logits);
+  const exps = logits.map(v => Math.exp(v - maxVal));
+  const sumExps = exps.reduce((a,b)=>a+b, 0);
+  return exps.map(v => v/sumExps);
+}
+
+// ------------------------
+// Vocabulary & Tokenization
+// ------------------------
+const vocab = [
+  "i","you","he","she","we","they",
+  "like","love","hate","pizza","coding","game","movie","music","coffee","tea",
+  "run","walk","play","read","book","eat","drink","watch","listen","reads",
+  "drink","drinks"
+];
+
+const word2idx = Object.fromEntries(vocab.map((w,i)=>[w,i]));
+const idx2word = Object.fromEntries(vocab.map((w,i)=>[i,w]));
+
+function oneHot(index, vocabSize) {
+  return Array.from({length:vocabSize}, (_,i)=>i===index?1:0);
+}
+
+// ------------------------
+// Dataset (sequence length 2)
+// ------------------------
+const sentences = [
+  ["i","like","pizza"], ["i","like","coding"], ["i","love","music"],
+  ["i","read","book"], ["i","watch","movie"], ["you","like","pizza"],
+  ["you","love","coffee"], ["you","read","book"], ["you","play","game"],
+  ["he","hate","coffee"], ["he","like","music"], ["he","play","game"],
+  ["she","love","tea"], ["she","read","book"], ["she","watch","movie"],
+  ["we","play","game"], ["we","read","book"], ["we","love","coffee"],
+  ["they","eat","pizza"], ["they","play","game"], ["they","listen","music"],
+  ["i","drink","coffee"], ["he","drink","tea"], ["she","drink","coffee"],
+  ["we","drink","tea"], ["they","drink","coffee"], ["i","play","game"],
+  ["you","watch","movie"], ["he","read","book"], ["she","listen","music"],
+  ["he","reads","book"], ["you","read","book"], ["they","read","book"],
+  ["they","watch","movie"], ["we","listen","music"], ["we","watch","movie"],
+  ["we","reads","book"],["we","drinks","coffee"],["we","love","you"],
+  ["i","read","book"], ["i","love","you"]
+];
+
+// Convert to input-output pairs
+const X = [], Y = [];
+const seqLength = 2;
+for(const s of sentences){
+  for(let i=0;i<=s.length-seqLength;i++){
+    const inpSeq = s.slice(i,i+seqLength).map(w=>oneHot(word2idx[w],vocab.length)).flat();
+    const outWord = s[i+seqLength] ? oneHot(word2idx[s[i+seqLength]], vocab.length)
+                                    : oneHot(word2idx[s[i+seqLength-1]], vocab.length);
+    X.push(inpSeq);
+    Y.push(outWord);
+  }
+}
+
+// ------------------------
+// Model Definition
+// ------------------------
+const model = new Sequential([
+  new Linear(vocab.length*seqLength, 128),
+  new ReLU(),
+  new Linear(128, vocab.length)
+]);
+
+const lossFn = new CrossEntropyLoss();
+const optimizer = new Adam(model.parameters(), 0.01);
+
+// ------------------------
+// Training Loop
+// ------------------------
+for(let epoch=1; epoch<=1600; epoch++){
+  const pred = model.forward(X);
+  const loss = lossFn.forward(pred, Y);
+  const grad = lossFn.backward();
+  model.backward(grad);
+  optimizer.step();
+
+  if(epoch % 500 === 0) console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}`);
+}
+
+// ------------------------
+// Beam Search Prediction
+// ------------------------
+function beamSearch(inputWords, beamWidth=2, predLength=3){
+  let sequences = [{words:[...inputWords], score:1}];
+  for(let step=0; step<predLength; step++){
+    const allCandidates = [];
+    for(const seq of sequences){
+      const inp = seq.words.slice(-seqLength).map(w=>oneHot(word2idx[w],vocab.length)).flat();
+      const out = model.forward([inp])[0];
+      const probs = softmaxVector(out); // softmax applied here
+      const top = probs.map((v,i)=>[i,v]).sort((a,b)=>b[1]-a[1]).slice(0,beamWidth);
+      for(const [idx,score] of top){
+        allCandidates.push({words:[...seq.words, idx2word[idx]], score:seq.score*score});
+      }
+    }
+    sequences = allCandidates.sort((a,b)=>b.score-a.score).slice(0,beamWidth);
+  }
+  return sequences.map(s=>({sequence:s.words, score:s.score.toFixed(3)}));
+}
+
+// ------------------------
+// Test Predictions
+// ------------------------
+const testInputs = [
+  ["i","like"], ["you","love"], ["they","play"], ["he","hate"], ["she","reads"]
+];
+
+for(const inp of testInputs){
+  const results = beamSearch(inp, 2, 3); // beam width 2, predict next 3 words
+  console.log(`Input: ${inp.join(" ")}`);
+  results.forEach(r=>console.log(`  Sequence: ${r.sequence.join(" ")}, Score: ${r.score}`));
+}

package/tests/proj3.js

@@ -0,0 +1,76 @@
+// JSTORCH TESTS TEMPLATE FILES
+// CIRCLE CLASSIFICATION TESTS FILES
+import { Sequential, Linear, ReLU, Sigmoid, CrossEntropyLoss, Adam } from '../src/MainEngine.js';
+
+// === Generate Circle Dataset ===
+function generateCircleData(n) {
+  const X = [], Y = [];
+  for (let i = 0; i < n; i++) {
+    const x = Math.random() * 2 - 1; // -1 to 1
+    const y = Math.random() * 2 - 1;
+    const label = (x*x + y*y < 0.5*0.5) ? [1,0] : [0,1]; // inside circle radius 0.5
+    X.push([x, y]);
+    Y.push(label);
+  }
+  return { X, Y };
+}
+
+const { X, Y } = generateCircleData(300);
+
+// === Build Model (bigger hidden layers) ===
+const model = new Sequential([
+  new Linear(2, 16),
+  new ReLU(),
+  new Linear(16, 8),
+  new ReLU(),
+  new Linear(8, 2),
+  new Sigmoid()
+]);
+
+// Loss & Optimizer
+const lossFn = new CrossEntropyLoss();
+const optimizer = new Adam(model.parameters(), 0.01); // smaller LR for not causing the model get stuck.
+
+// === Training ===
+console.log("=== Circle Classification Training (Fixed) ===");
+const start = Date.now();
+for (let epoch = 1; epoch <= 2000; epoch++) {
+  const pred = model.forward(X);
+  const loss = lossFn.forward(pred, Y);
+  const gradLoss = lossFn.backward();
+  model.backward(gradLoss);
+  optimizer.step();
+
+  if (epoch % 500 === 0) {
+    console.log(`Epoch ${epoch}, Loss: ${loss.toFixed(4)}`);
+  }
+}
+console.log(`Training Time: ${((Date.now()-start)/1000).toFixed(3)}s`); // FOR real time monitoring while training if this module is literally lightweight and does not take a long time to train.
+
+// === Predictions ===
+console.log("\n=== Predictions ===");
+const testInputs = [
+  [0,0],
+  [0.7,0.7],
+  [0.2,0.2],
+  [0.9,0.1],
+  [-0.5,-0.5],
+  [0.6,-0.2]
+];
+testInputs.forEach(inp => {
+  const out = model.forward([inp])[0];
+  const predClass = out.indexOf(Math.max(...out));
+  console.log(
+    `Input: ${inp}, Pred: ${predClass}, Raw: ${out.map(v => v.toFixed(3))}`
+  );
+});
+
+/**
+ * === How to Run On Your VS Code Projects ===
+ * 1. Make sure Node.js (v20+ recommended) is installed on your system.
+ * 2. Open this file in VS Code (or any editor).
+ * 3. Open the terminal in the project root folder.
+ * 4. Run: node tests/proj3.js
+ *
+ * You should see the training logs and prediction outputs directly.
+ */