mini-jstorch 1.1.7 → 1.2.0

package/CHANGELOG.md

@@ -0,0 +1,32 @@
+## Changelog ##
+
+**Official Release:** 2025-Monday-August-18  
+**Version:** 1.1.9
+
+New Files that will All notable changes to *Mini-JSTorch* will be documented in this file.
+
+---
+
+## [1.1.9] - 2025-Monday-August-18
+
+### Added
+- Complete overhaul of core engine for stability and full feature support.
+- Adam optimizer fully integrated with batch gradient handling.
+- NaN-safe operations for forward/backward passes and loss calculations.
+- Sequential container with robust parameter tracking.
+- Utility functions enhanced for numeric safety and matrix operations.
+- Ready-to-use example scripts with multiple hidden layers for frontend experimentation.
+- Full handling for lightweight devices, optimized for speed and low memory usage.
+
+### Fixed
+- bugs in ReLU and gradient propagation causing NaN loss values.
+- Backward propagation errors for batch matrices corrected.
+- Critical Error while startup Engine.
+
+### Changed
+- Layer and activation API standardized for consistency.
+- CrossEntropyLoss now safely handles edge cases in probability computations.
+- Improvements systems for make experiences more stable and not heavy.
+- Minor numeric stability improvements.
+
+---

package/README.md

@@ -1,17 +1,51 @@
-## INTRODUCING MINI-JSTORCH ##
+## Mini-JSTorch ##
 
-**Mini-JSTorch** is a lightweight JavaScript Module Packages where it was inspired by **PyTorch** and this Module was designed to be used at any devices and any like frontend or backend. This JSTorch finally at their versions and not *BETA* again.
-This Module only Friendly at Frontend may he not run well while running at GPU or Backend We're gonna still update it so it still stable at backend and GPU.
+## SORRY GUYS FOR THE DISTURB BECAUSE VERSION 1.1.9 I FORGOT PLACE THE FULL SYSTEM FOR THE UPDATE NOT PLACED ON MAINENGINE BUT AT EXPERIMENTS FILES 
 
+# IMPORTANT!
+
+This Module will not gonna Run *Well* In at GPU or a Backend.
+We're will gonna *Optimize* So all User can still use this Module.
+
+---
+
+**Mini JSTorch** is a lightweight, high-performance JavaScript library designed for rapid prototyping of neural networks directly in the frontend environment and for low-end devices. Its core purpose is to enable experimentation and learning on devices with limited computing resources, without compromising stability or training reliability.
 
 ---
 
-## Key Features
+## Vision
 
-- **Custom Model Creation:** Configure layers, neurons, learning rate, and more.  
-- **Manual & Auto Gradient:** Toggle gradient computation mode per experiment.  
-- **Sequence Handling:** Supports input sequences for NLP or time series tasks.  
-- **Lightweight & Frontend-Friendly:** Pure JS implementation, no GPU dependency.
+Our vision is to democratize AI experimentation by providing a tool that is:
+
+- **Accessible**: Runs efficiently on *low-end* laptops or mobile devices.
+- **Stable**: Handles gradient calculations and NaN values robustly.
+- **Educational**: Clear abstractions for neural network layers, activations, and optimizers.
+- **Fast**: Immediate output without lengthy training delays.
+
+*Mini JSTorch* aims to bridge the gap between conceptual AI learning and hands-on experimentation, making it possible to explore neural network behavior interactively, even on minimal hardware.
+
+---
+
+## Features
+
+- **Core Layers**
+  - `Linear`: Fully connected layer with weight and bias support.
+- **Activations**
+  - `ReLU`: Rectified Linear Unit activation.
+  - `Sigmoid`: Sigmoid activation function.
+- **Loss Functions**
+  - `CrossEntropyLoss`: Combines softmax and cross-entropy for classification tasks.
+- **Optimizers**
+  - `Adam`: Adaptive optimizer for stable and efficient training.
+- **Utilities**
+  - `zeros(rows, cols)`: Generate a zero matrix.
+  - `randomMatrix(rows, cols, scale)`: Random initialization for weights.
+  - `softmax(x)`: Compute softmax probabilities.
+  - `crossEntropy(pred, target)`: Compute cross-entropy loss.
+  - `dot(a, b)`: Matrix multiplication.
+  - `addMatrices(a, b)`: Element-wise addition of matrices.
+- **Model Container**
+  - `Sequential`: Container to stack layers sequentially with easy forward and backward passes.
 
 ---
 
@@ -19,16 +53,81 @@ This Module only Friendly at Frontend may he not run well while running at GPU o
 
 ```bash
 npm install mini-jstorch
+# Node.js v20+ recommended for a good experience while using module
 ```
 
-## Patch v1.1.7
+---
+
+## Example Scripts
+
+```javascript
+import { Sequential, Linear, ReLU, Sigmoid, CrossEntropy, Adam } from './engine/MainEngine.js'; // use import to import the modules
+
+// Construct a model with two hidden layers
+const model = new Sequential([
+    new Linear(2, 4),
+    new ReLU(),
+    new Linear(4, 4),
+    new ReLU(),
+    new Linear(4, 2),
+    new Sigmoid()
+]);
+
+// Training data (XOR)
+const inputs = [
+    [0,0], [0,1], [1,0], [1,1]
+];
+const targets = [
+    [1,0], [0,1], [0,1], [1,0]
+];
+
+// Optimizer and loss
+const optimizer = new Adam(model.parameters(), 0.01);
+const lossFunc = CrossEntropy;
+
+// Training loop
+for(let epoch = 1; epoch <= 5000; epoch++) {
+    let totalLoss = 0;
+    for(let i = 0; i < inputs.length; i++) {
+        const output = model.forward([inputs[i]]);
+        const loss = lossFunc(output, [targets[i]]);
+        totalLoss += loss;
+        model.zeroGrad();
+        model.backward(lossFunc.backward());
+        optimizer.step();
+    }
+    if(epoch % 500 === 0) console.log(`Epoch ${epoch}, Loss: ${totalLoss.toFixed(6)}`);
+}
+
+// Predictions
+inputs.forEach(inp => {
+    const pred = model.forward([inp]);
+    console.log(`${inp} -> ${pred.map(p => p.toFixed(4))}`);
+});
+```
+---
+
+## Intended Use Cases
+
+- **Experimentation on low-end devices or mobile browsers.**
+- **Learning and teaching foundational neural network concepts.**
+- **Testing small to medium feedforward models in real-time.**
+- **Quick prototyping without GPU dependency or complex setup.**
+
+---
+
+## Roadmap
 
-- *ONLY* Adding a README.md Because i'm forgot btw.
+- **Browser-based interactive playground.**
+- **Additional activation functions and loss options.**
+- **Visualization of training metrics and loss curves in real-time.**
+- **Support for small convolutional networks in frontend environments.**
 
 ---
 
-## Notes
+## Facts
 
-- **One More. This Module are would not Run *WELL* at GPU or Backend**
-- **The Features from the Module are Nearly little near same system Feature Engine like *PyTorch***
-- **If you confusing about file 'startup.cpu' because the syntax are not similar to a program Language it was probably used for booting the Engine**
+- **This module is implemented entirely in pure JavaScript.**
+- **The `Dummy` folder contains modules used for development, testing, and debugging before integration into the main engine.**
+- **This module was created by a `single` developer.**
+- **Date 23 Would be `CRAZY`.**

package/engine/MainEngine.js

@@ -1,148 +1,209 @@
-// ================================
-// MINI JS AI ENGINE v1
-// ================================
 
-// Basic tensor ops
-const zeros = n => Array(n).fill(0);
-const randn = () => (Math.random() * 2 - 1) * 0.1;
+// MAINENGINE FILES [PACK IN AT ONE FILES]
+// CURRENT VERSIONS: 0.0.4
+// AUTHOR: Rizal
+// LICENSE: MIT(R)
 
-const dot = (a,b) => a.map(row=>b[0].map((_,j)=>row.reduce((sum,v,k)=>sum+v*b[k][j],0)));
-const add = (a,b) => a.map((row,i)=>row.map((v,j)=>v+b[i][j]));
-const sub = (a,b) => a.map((row,i)=>row.map((v,j)=>v-b[i][j]));
-const mulScalar = (a,s) => a.map(row=>row.map(v=>v*s));
-const transpose = m => m[0].map((_,i)=>m.map(row=>row[i]));
+// Utils
+export function zeros(rows, cols) {
+    return Array.from({length: rows}, () => Array(cols).fill(0));
+}
 
-// Activations
-const Activations = {
-    relu: x => x.map(v=>Math.max(0,v)),
-    linear: x => x,
-    leakyRelu: (x, alpha=0.01) => x.map(v=>v>0?v:alpha*v)
-};
-const dActivations = {
-    relu: x => x.map(v=>v>0?1:0),
-    linear: x => x.map(_=>1),
-    leakyRelu: (x, alpha=0.01) => x.map(v=>v>0?1:alpha)
-};
-
-// Dense layer with manual grad & auto-grad
-class Dense {
-    constructor(inputSize, outputSize, activation='linear'){
-        this.inputSize = inputSize;
-        this.outputSize = outputSize;
-        this.activation = activation;
-        this.W = Array.from({length:inputSize},()=>Array.from({length:outputSize},()=>randn()*Math.sqrt(2/inputSize)));
-        this.b = Array(outputSize).fill(0);
-
-        // Adam variables
-        this.mW = mulScalar(this.W,0);
-        this.vW = mulScalar(this.W,0);
-        this.mb = Array(outputSize).fill(0);
-        this.vb = Array(outputSize).fill(0);
-        this.lastInput = null;
-        this.lastOutput = null;
-    }
-
-    forward(X){
-        this.lastInput = X;
-        let output = dot(X,this.W);
-        output = output.map((row,i)=>row.map((v,j)=>v+this.b[j]));
-        this.lastOutput = output.map(row => Activations[this.activation](row));
-        return this.lastOutput;
-    }
-
-    backward(dLoss, lr=0.001, beta1=0.9, beta2=0.999, t=1){
-        const flatOut = this.lastOutput.flat();
-        const actGrad = dActivations[this.activation](flatOut);
-        const dOut = dLoss.flat().map((v,i)=>v*actGrad[i]);
-
-        const gradW = Array.from({length:this.inputSize},()=>Array(this.outputSize).fill(0));
-        const gradB = Array(this.outputSize).fill(0);
-
-        for(let k=0;k<this.lastInput.length;k++){
-            for(let i=0;i<this.inputSize;i++){
-                for(let j=0;j<this.outputSize;j++){
-                    gradW[i][j] += this.lastInput[k][i]*dOut[j]/this.lastInput.length;
-                }
-            }
-        }
-        for(let j=0;j<this.outputSize;j++) gradB[j] = dOut[j]/this.lastInput.length;
-
-        // Adam update with bias correction
-        for(let i=0;i<this.inputSize;i++){
-            for(let j=0;j<this.outputSize;j++){
-                this.mW[i][j] = beta1*this.mW[i][j]+(1-beta1)*gradW[i][j];
-                this.vW[i][j] = beta2*this.vW[i][j]+(1-beta2)*gradW[i][j]*gradW[i][j];
-                const mHat = this.mW[i][j]/(1-Math.pow(beta1,t));
-                const vHat = this.vW[i][j]/(1-Math.pow(beta2,t));
-                this.W[i][j] -= lr*mHat/(Math.sqrt(vHat)+1e-8);
-            }
-        }
+export function randomMatrix(rows, cols, scale=0.1) {
+    return Array.from({length: rows}, () =>
+        Array.from({length: cols}, () => (Math.random()*2-1)*scale)
+    );
+}
 
-        for(let j=0;j<this.outputSize;j++){
-            this.mb[j] = beta1*this.mb[j]+(1-beta1)*gradB[j];
-            this.vb[j] = beta2*this.vb[j]+(1-beta2)*gradB[j]*gradB[j];
-            const mHat = this.mb[j]/(1-Math.pow(beta1,t));
-            const vHat = this.vb[j]/(1-Math.pow(beta2,t));
-            this.b[j] -= lr*mHat/(Math.sqrt(vHat)+1e-8);
-        }
+export function softmax(x) {
+    const maxVal = Math.max(...x);
+    const exps = x.map(v => Math.exp(v - maxVal));
+    const sumExps = exps.reduce((a,b)=>a+b, 0);
+    return exps.map(v => v / sumExps);
+}
 
-        // Return dLoss for next layer (manual grad)
-        const dNext = Array(this.lastInput.length).fill(0).map(_=>Array(this.inputSize).fill(0));
-        for(let i=0;i<this.inputSize;i++){
-            for(let j=0;j<this.outputSize;j++){
-                for(let k=0;k<this.lastInput.length;k++){
-                    dNext[k][i] += dOut[j]*this.W[i][j];
-                }
-            }
+export function crossEntropy(pred, target) {
+    const eps = 1e-12;
+    return -target.reduce((sum, t, i) => sum + t * Math.log(pred[i] + eps), 0);
+}
+
+export function addMatrices(a, b) {
+    return a.map((row, i) =>
+        row.map((v, j) => v + (b[i] && b[i][j] !== undefined ? b[i][j] : 0))
+    );
+}
+
+export function dot(a, b) {
+    const result = zeros(a.length, b[0].length);
+    for (let i=0;i<a.length;i++) {
+        for (let j=0;j<b[0].length;j++) {
+            let sum=0;
+            for (let k=0;k<a[0].length;k++) sum += a[i][k]*b[k][j];
+            result[i][j]=sum;
         }
-        return dNext;
     }
+    return result;
 }
 
-// Sequential model
-class Seq {
-    constructor(layers){
-        this.layers = layers;
-        this.logs = [];
+// Layers
+export class Linear {
+    constructor(inputDim, outputDim) {
+        this.W = randomMatrix(inputDim, outputDim);
+        this.b = Array(outputDim).fill(0);
+        this.gradW = zeros(inputDim, outputDim);
+        this.gradb = Array(outputDim).fill(0);
+        this.x = null;
     }
 
-    forward(X){
-        return this.layers.reduce((inp,layer)=>layer.forward(inp), X);
+    forward(x) {
+        this.x = x;
+        const out = dot(x, this.W);
+        return out.map((row,i) => row.map((v,j)=>v+this.b[j]));
     }
 
-    train(X,y,epochs=200, lr=0.001){
-        for(let epoch=1;epoch<=epochs;epoch++){
-            const yPred = this.forward(X);
-            const dLoss = yPred.map((row,i)=>row.map((v,j)=>2*(v - y[i][j])/y.length));
+    backward(grad) {
+        // grad shape: batch x outputDim
+        for (let i=0;i<this.W.length;i++)
+            for (let j=0;j<this.W[0].length;j++)
+                this.gradW[i][j] = this.x.reduce((sum,row,k)=>sum+row[i]*grad[k][j],0);
+        
+        for (let j=0;j<this.b.length;j++)
+            this.gradb[j] = grad.reduce((sum,row)=>sum+row[j],0);
+
+        // propagate to input
+        const gradInput = zeros(this.x.length, this.W.length);
+        for (let i=0;i<this.x.length;i++)
+            for (let j=0;j<this.W.length;j++)
+                for (let k=0;k<this.W[0].length;k++)
+                    gradInput[i][j]+=grad[i][k]*this.W[j][k];
+        return gradInput;
+    }
 
-            let grad = dLoss;
-            for(let l=this.layers.length-1;l>=0;l--){
-                grad = this.layers[l].backward(grad, lr, 0.9, 0.999, epoch);
-            }
+    parameters() {
+        return [
+            {param: this.W, grad: this.gradW},
+            {param: [this.b], grad: [this.gradb]} // wrap b in array for consistency
+        ];
+    }
+}
 
-            const loss = yPred.map((row,i)=>row.map((v,j)=>Math.pow(v - y[i][j],2))).flat().reduce((a,b)=>a+b,0)/y.length;
-            if(epoch%50===0) console.log(`Epoch ${epoch}, Loss ${loss.toFixed(4)}, Pred sample ${yPred[0][0].toFixed(2)}`);
-        }
+// Activations
+export class ReLU {
+    constructor() { this.out=null; }
+    forward(x) {
+        this.out = Array.isArray(x[0]) ? x.map(r=>r.map(v=>Math.max(0,v))) : x.map(v=>Math.max(0,v));
+        return this.out;
+    }
+    backward(grad) {
+        return Array.isArray(grad[0])
+            ? grad.map((r,i)=>r.map((v,j)=>v*(this.out[i][j]>0?1:0)))
+            : grad.map((v,i)=>v*(this.out[i]>0?1:0));
+    }
+}
+
+export class Sigmoid {
+    constructor() { this.out=null; }
+    forward(x) {
+        const sigmoidFn = v=>1/(1+Math.exp(-v));
+        this.out = Array.isArray(x[0]) ? x.map(r=>r.map(sigmoidFn)) : x.map(sigmoidFn);
+        return this.out;
+    }
+    backward(grad) {
+        return Array.isArray(grad[0])
+            ? grad.map((r,i)=>r.map((v,j)=>v*this.out[i][j]*(1-this.out[i][j])))
+            : grad.map((v,i)=>v*this.out[i]*(1-this.out[i]));
+    }
+}
+
+// Loss wrapper
+export class CrossEntropyLoss {
+    forward(pred, target) {
+        // pred: batch x classes, target: batch x classes
+        const losses = pred.map((p,i)=>crossEntropy(softmax(p), target[i]));
+        this.pred = pred;
+        this.target = target;
+        return losses.reduce((a,b)=>a+b,0)/pred.length;
     }
 
-    predict(X){
-        return this.forward(X);
+    backward() {
+        // gradient of softmax + CE
+        const grad = [];
+        for (let i=0;i<this.pred.length;i++) {
+            const s = softmax(this.pred[i]);
+            grad.push(s.map((v,j)=>v-this.target[i][j]));
+        }
+        return grad.map(r=>r.map(v=>v/this.pred.length));
     }
 }
 
-// ================================
-// EXAMPLE USAGE
-// ================================
+// Optimizer: Adam
+export class Adam {
+    constructor(params, lr=0.001, beta1=0.9, beta2=0.999, eps=1e-8) {
+        this.params = params;
+        this.lr = lr;
+        this.beta1 = beta1;
+        this.beta2 = beta2;
+        this.eps = eps;
+        this.m = params.map(p=>zeros(p.param.length, p.param[0].length || 1));
+        this.v = params.map(p=>zeros(p.param.length, p.param[0].length || 1));
+        this.t=0;
+    }
 
-const model = new Seq([
-    new Dense(2,16,'relu'),
-    new Dense(16,12,'relu'),
-    new Dense(12,1,'linear')
-]);
+    step() {
+        this.t++;
+        this.params.forEach((p,idx)=>{
+            for (let i=0;i<p.param.length;i++){
+                for (let j=0;j<(p.param[0].length||1);j++){
+                    const g = p.grad[i][j];
+                    this.m[idx][i][j] = this.beta1*this.m[idx][i][j] + (1-this.beta1)*g;
+                    this.v[idx][i][j] = this.beta2*this.v[idx][i][j] + (1-this.beta2)*g*g;
+                    const mHat = this.m[idx][i][j]/(1-Math.pow(this.beta1,this.t));
+                    const vHat = this.v[idx][i][j]/(1-Math.pow(this.beta2,this.t));
+                    p.param[i][j]-= this.lr*mHat/(Math.sqrt(vHat)+this.eps);
+                }
+            }
+        });
+    }
+}
 
-const X = [[1,2],[2,3],[3,4]];
-const y = [[3],[5],[7]];
+// Sequential container
+export class Sequential {
+    constructor(layers=[]) { this.layers=layers; }
+    forward(x) {
+        return this.layers.reduce((input, layer)=>layer.forward(input), x);
+    }
+    backward(grad) {
+        return this.layers.reduceRight((g,layer)=>layer.backward(g), grad);
+    }
+    parameters() {
+        return this.layers.flatMap(l=>l.parameters ? l.parameters() : []);
+    }
+}
 
-model.train(X,y,200,0.01);
+// Tambahin ke akhir file utama mini-jstorch
+     export function saveModel(model) {
+       if (!(model instanceof Sequential)) {
+         throw new Error("saveModel only supports Sequential models");
+       }
+       const weights = model.layers.map(layer => ({
+         weights: layer.W ? layer.W : null,
+         biases: layer.b ? layer.b : null
+       }));
+       return JSON.stringify(weights);
+     }
+
+     export function loadModel(model, json) {
+       if (!(model instanceof Sequential)) {
+         throw new Error("loadModel only supports Sequential models");
+       }
+       const weights = JSON.parse(json);
+       model.layers.forEach((layer, i) => {
+         if (layer.W && weights[i].weights) {
+           layer.W = weights[i].weights;
+         }
+         if (layer.b && weights[i].biases) {
+           layer.b = weights[i].biases;
+         }
+       });
+     }
 
-console.log('Prediction [7,8]:', model.predict([[7,8]]));

package/index.js

@@ -1,4 +1,4 @@
-// Entry point of the library, export main classes and functions
+// Entry point of the library, export main classes and functions [DEPRECATED]
 export { Seq } from './models/seq.js';
 export { Dense } from './layers/dense.js';
 export * as act from './act/linear.js';

package/package.json

@@ -1,16 +1,18 @@
 {
   "name": "mini-jstorch",
-  "version": "1.1.7",
+  "version": "1.2.0",
   "type": "module", 
-  "description": "A lightweight JavaScript neural network framework for browser & Node.js, inspired by PyTorch.",
+  "description": "A lightweight JavaScript neural network library for rapid frontend AI experimentation on low-resource devices Inspired by PyTorch.",
   "main": "index.js",
   "keywords": [
     "neural-network",
     "javascript",
     "lightweight",
     "ai",
+    "jstorch",
+    "pytorch",
+    "front-end",
     "machine-learning",
-    "browser",
     "mini"
   ],
   "author": "Rizal",