volvoxai 0.1.0 → 0.3.0

package/README.md

@@ -1,144 +1,321 @@
 # VolvoxAI
 
-**A zero-dependency, bare-metal deep learning inference engine for the browser,
-Node.js, and native Windows/Linux/macOS/Android targets.**
+**A zero-dependency deep-learning runtime for browsers, Node.js, and native
+Windows, Linux, macOS, and Android targets.**
 
-VolvoxAI runs exported neural-network graphs without shipping a full ML runtime.
-Train in PyTorch or another framework, export to a small Volvox blueprint plus
-safetensors weights, and run inference through WebNN, WebGPU, WASM SIMD, pure JS,
-or a freestanding native C binary.
+VolvoxAI runs compact graph packages without embedding a general-purpose ML
+framework. It supports WebNN, WebGPU, WASM SIMD, JavaScript CPU, native CPU,
+Vulkan, OpenGL, optional CUDA, Metal, and NNAPI integrations.
 
-The project is inference-only. It is built for small, inspectable model packages,
-constrained web apps, extensions, local tools, and edge devices where heavyweight
-runtimes such as ONNX Runtime Web or TensorFlow.js are too large or too opaque.
+The repository is also a from-scratch textbook:
+
+- [Idea, Build, and Deep tracks](docs/textbook/README.md)
+- [Architecture](ARCHITECTURE.md)
+- [Runtime design](REFACTORING.md)
 
 ## Highlights
 
-- Browser tiers: WebNN, WebGPU, WASM SIMD, and pure-JS CPU fallback.
-- Zero runtime dependency path: vanilla JS plus WGSL with optional freestanding WASM.
-- Small runtime artifacts: about 167 KiB minified JS, 55 KiB WASM, and a
-  roughly 695 KiB native Linux executable in current builds.
-- Native runtime: freestanding C with dynamic Vulkan, OpenGL, Metal, and Android
-  NNAPI loading.
-- Optimization-focused internals: operator fusion, weight packing, quantized CPU
-  islands, vectorized WGSL/C kernels, and backend-specific dispatch.
-- Model format: `config.json` graph blueprint plus `model.safetensors` weights.
-- Shader source: WGSL for browser WebGPU and native shader generation.
-- Examples: EfficientDet Lite0 object detection and TinyStories-1M text generation.
-- License: MIT.
+- One explicit inference lifecycle: Runtime → Model → CompiledModel →
+  ExecutionContext → ExecutionResult.
+- Stable named outputs on every backend. Host reads return caller-owned arrays;
+  WebGPU results may also expose result-owned device buffers.
+- Independent execution and decode contexts with immutable compiled model and
+  weight revisions.
+- Required or preferred backend policy with independent operator-fallback
+  control and machine-readable reports.
+- A single context-aware provider contract for built-in and external devices.
+- A full profile with a retained Trainer for CPU, WebGPU, or strict WASM
+  training.
+- Inspectable model packages using graph.json and safetensors.
+- Strict inference/training composition boundaries in JavaScript, WASM, and
+  native builds.
 
 ## Install
 
-```bash
+~~~bash
 npm install volvoxai
-```
+~~~
 
-For local development from this repository:
+For repository development:
 
-```bash
+~~~bash
 npm install
+npm run typecheck
 npm run build:all
-```
-
-The browser bundle is emitted to:
+~~~
 
-```text
-dist/volvoxai.js
-dist/volvoxai.min.js
-```
+## Release artifacts
 
-The WASM tier also needs `volvoxai.wasm`. The reproducible Docker build copies it
-into `dist/`:
+The fixed browser release files for package version 0.3.0 are:
 
-```bash
-make build_web
-```
+~~~text
+dist/0.3.0/volvoxai.js
+dist/0.3.0/volvoxai.min.js
+dist/0.3.0/volvoxai.full.js
+dist/0.3.0/volvoxai.full.min.js
+dist/0.3.0/volvoxai.wasm.js
+dist/0.3.0/volvoxai.wasm.min.js
+dist/0.3.0/volvoxai.wasm
+dist/0.3.0/volvoxai.full.wasm
+~~~
 
-## Browser Usage
+The standard JavaScript entry is inference-only and resolves the forward-only
+WASM sidecar. The full entry adds training and resolves volvoxai.full.wasm.
+The WASM-only JavaScript entry contains strict WASM inference and training but
+no CPU, WebNN, WebGPU, WGSL, or Node filesystem implementation.
 
-```javascript
-import { VolvoxAI } from './volvoxai.js';
+Build all browser artifacts reproducibly with:
 
-const engine = await VolvoxAI.init(); // auto: WebNN, WebGPU, WASM, CPU
-const graph = await engine.loadGraph('./models/my-model/model.safetensors');
-const executor = await engine.compile(graph);
-
-const inputs = {
+~~~bash
+make build_web
+~~~
+
+## Model packages
+
+An inference package contains:
+
+~~~text
+graph.json
+model.safetensors
+~~~
+
+Every graph root, including named subgraphs, must carry the exact
+case-sensitive discriminator:
+
+~~~json
+{
+  "format": "volvox-graph/v1"
+}
+~~~
+
+The loader rejects a missing or different discriminator before allocating
+weights or backend resources. Every node input must resolve to a declared graph
+input, a named weight, or an earlier node output.
+
+## Inference
+
+~~~javascript
+import { VolvoxAI } from 'volvoxai';
+
+const runtime = await VolvoxAI.createRuntime({
+  backends: ['webnn', 'webgpu', 'wasm', 'cpu'],
+  onDiagnostic(event) {
+    console.debug(event.kind, event.report ?? event);
+  },
+});
+
+const model = await runtime.loadModel(
+  './models/my-model/model.safetensors',
+);
+const compiled = await model.compile({
+  backend: {
+    mode: 'prefer',
+    order: ['webgpu', 'wasm', 'cpu'],
+    operatorFallback: 'allow',
+  },
+});
+const context = await compiled.createContext();
+
+const result = await context.execute({
   images: new Float32Array(1 * 224 * 224 * 3),
-};
-
-const output = await executor.execute(inputs);
-```
-
-On WebGPU, `execute()` currently returns a `GPUBuffer` for the final node's first
-output. WASM and CPU return a map keyed by `graph.outputNames`.
-
-## Native Usage
-
-```bash
+});
+const scores = await result.output('scores').read();
+
+await result.close();
+await context.close();
+await compiled.close();
+await model.close();
+await runtime.close();
+~~~
+
+Runtime loading resolves graph.json beside the first safetensors URL. Pass
+graphUrl in the loader options when the graph is stored elsewhere; its basename
+must be `graph.json` or a named `*.graph.json` document.
+
+Compilation pins an immutable topology and weight revision. Create multiple
+contexts from one compiled model for independent request or decode state. Each
+context serializes its own accepted operations, while different contexts may
+progress concurrently.
+
+ExecutionResult owns a stable snapshot of every declared graph output. A result
+remains usable after later executions and after its context closes. Each read()
+returns a fresh typed array. A device result may expose deviceBuffer; that
+buffer remains owned by the result and must not be destroyed by the caller.
+
+Use a strict policy when execution must stay on one provider:
+
+~~~javascript
+const compiled = await model.compile({
+  backend: {
+    mode: 'require',
+    backend: 'webgpu',
+    operatorFallback: 'forbid',
+  },
+});
+~~~
+
+Backend selection finishes during compilation. Execution failure is reported
+and is never retried on another provider.
+
+## Training
+
+Training is available only from the full and WASM-only profiles. Trainer owns
+gradients, optimizer slots, accumulation, and a private working
+revision. `trainStep()` mutates only that private revision. `commit()` atomically
+publishes it as a new Model weight revision; already compiled models and
+contexts remain pinned to their original revision.
+
+~~~javascript
+import {
+  ModelBuilder,
+  VolvoxAI,
+} from 'volvoxai/full';
+
+const builder = new ModelBuilder();
+const x = builder.input('x', [1, 4]);
+const weight = builder.weight('projection', [4, 8], 'float32', {
+  initializer: { type: 'xavierUniform', seed: 17 },
+});
+const logits = builder.addOp(
+  'MatMul',
+  { input: x, weight },
+  { out: { name: 'logits', shape: [1, 8] } },
+  {},
+  { id: 'projection', wLayout: 'din' },
+).out;
+builder.outputs(logits);
+const graph = builder.build();
+
+const runtime = await VolvoxAI.createRuntime({ backends: ['cpu'] });
+const model = runtime.createModel(graph);
+const trainer = await VolvoxAI.createTrainer(model, {
+  backend: 'cpu',
+});
+
+const step = await trainer.trainStep({
+  inputs: { x: new Float32Array([1, 2, 3, 4]) },
+  logitsTensor: 'logits',
+  targets: new Int32Array([3]),
+  trainableTensors: ['projection'],
+  updateMode: 'adamw',
+  optimizer: { learningRate: 1e-3, maxGradNorm: 1 },
+});
+await trainer.commit();
+
+await trainer.close();
+await model.close();
+await runtime.close();
+~~~
+
+The same Trainer contract accepts backend: 'webgpu' or backend: 'wasm'. WASM
+training is strict and rejects an unsupported graph before mutating weights.
+There is no implicit publication: call `commit()` before compiling inference
+against the update, or `rollback()` to restore the last committed baseline.
+The full profile also exports training builders, checkpoints, gradient
+accumulation controls, LoRA helpers, and PTQ authoring tools. See
+[model construction and training](docs/model_builder_training.md) and the
+[operation matrix](docs/operation_list.md).
+
+## WASM-only browser extensions
+
+For a Manifest V3 extension, package one WASM-only JavaScript variant, the full
+sidecar, and the model:
+
+~~~text
+vendor/volvoxai.wasm.min.js
+vendor/volvoxai.full.wasm
+model/graph.json
+model/model.safetensors
+~~~
+
+~~~javascript
+import { VolvoxAI } from './vendor/volvoxai.wasm.min.js';
+
+const runtime = await VolvoxAI.createRuntime({
+  wasmUrl: chrome.runtime.getURL('vendor/volvoxai.full.wasm'),
+});
+const model = await runtime.loadModel(
+  chrome.runtime.getURL('model/model.safetensors'),
+  { graphUrl: chrome.runtime.getURL('model/graph.json') },
+);
+~~~
+
+Extension pages need wasm-unsafe-eval in their content security policy. The
+WASM-only release has no dynamic import and contains no alternate backend.
+See [Browser and Node runtime](docs/browser-runtime.md).
+
+## Native use
+
+~~~bash
 make build_native
 
-./native/volvoxai generate models/tinystories_1m \
-  --prompt "Once upon a time, Lily" \
-  --max-new 50
-```
+./native/volvoxai --help
+./native/volvoxai-full --help
+~~~
 
-Generic tensor execution:
+The inference executable provides model-agnostic tensor execution. The full
+executable additionally provides training:
 
-```bash
+~~~bash
 ./native/volvoxai run models/tinystories_1m \
   --input tokens=models/tinystories_1m/tokens.i32 \
   --input positions=models/tinystories_1m/positions.i32 \
-  --output logits=out.f32 \
-  --last-token 4
-```
+  --output logits=out.f32
+~~~
+
+Raw files use a storage suffix matching their declared dtype: .f32, .i32,
+.i8, or .u8. Outputs contain the complete declared tensor; applications
+select task-specific rows or slices. Model-specific tokenization, image
+decoding, generation, and postprocessing live under examples/.
 
-## Example Models
+Native releases use embedded shaders. For shader development,
+VOLVOXAI_SHADER_DIR may point to generated spv/, glsl/, gles/, and metal/
+directories; VolvoxAI logs once when that external override is actually used.
 
-Model weights are not committed. Regenerate the example packages from public
-sources:
+## Example models
 
-```bash
+Weights are not committed. Recreate the example packages from public sources:
+
+~~~bash
 make models_deps
 make models_efficientdet
 make models_tinystories
-```
-
-See [docs/models.md](docs/models.md) for export details.
-
-## Repository Layout
-
-```text
-js/          Browser and Node engine implementation
-shaders/     WGSL compute shaders
-native/      Freestanding C engine and native backends
-runtime/     Rust service wrapper around the C engine
-examples/    Browser examples
-tools/       Export, shader, and test utilities
-docs/        Detailed documentation
-```
+make validate_model_packages
+~~~
+
+See [Models and exporters](docs/models.md).
+
+## Repository layout
+
+~~~text
+ts/core/             graph/data objects and runtime ownership
+ts/ops/              operators, validation, and normalization
+ts/backends/         backend providers and device resources
+ts/training/         Trainer, autograd, optimizers, checkpoints, and PTQ
+examples/            model-specific applications and integrations
+shaders/             authoritative WGSL source
+native/include/      public opaque inference/provider and full Trainer/PTQ C APIs
+native/src/runtime/  runtime/model/context/result implementation
+native/src/kernels/  portable and optimized CPU/WASM kernels
+native/src/backends/ native device integrations
+native/src/training/ full-profile training implementation
+runtime/             optional in-process Synurang FFI plugin
+~~~
 
 ## Documentation
 
 - [Quickstart](docs/quickstart.md)
 - [Browser and Node runtime](docs/browser-runtime.md)
 - [Native runtime](docs/native-runtime.md)
+- [Backend SDK](docs/backend-sdk.md)
 - [Model format](docs/model-format.md)
-- [Models and exporters](docs/models.md)
+- [Graph exporter and optimizer design](docs/graph-optimizer-design.md)
+- [Typed PTQ](docs/typed-ptq.md)
+- [Model construction and training](docs/model_builder_training.md)
 - [Operation support matrix](docs/operation_list.md)
 - [Testing and validation](docs/testing.md)
-- [Roadmap](docs/roadmap.md)
-- [Textbook walkthrough](docs/textbook/README.md) ([한국어](docs/textbook/ko/README.md))
-- [EfficientDet benchmark notes](docs/efficientdet_tflite_vs_volvoxai.md)
-
-## Current Status
-
-VolvoxAI can run real browser and native inference paths, but it is still early.
-Known gaps include WebGPU multi-output readback, broader WebNN coverage, additional
-native/WebGPU shaders for a few fallback ops, browser-side generation helpers, and
-formal CI wiring for the existing smoke tests.
-
-See [docs/roadmap.md](docs/roadmap.md) for the detailed list.
+- [Models and exporters](docs/models.md)
+- [Textbook](docs/textbook/README.md)
 
 ## License
 

package/bin/volvox.js

@@ -1,17 +1,23 @@
 #!/usr/bin/env node
 
 import { Command } from 'commander';
-import { VolvoxAI } from '../js/index.js';
 import fs from 'fs';
 import path from 'path';
 import { fileURLToPath } from 'url';
 
+const packageVersion = JSON.parse(
+  fs.readFileSync(new URL('../package.json', import.meta.url), 'utf8'),
+).version;
+const { VolvoxAI } = await import(
+  new URL(`../dist/${packageVersion}/volvoxai.js`, import.meta.url)
+);
+
 const program = new Command();
 
 program
   .name('volvox')
   .description('Volvox AI CLI for running local inference')
-  .version('0.1.0');
+  .version(packageVersion);
 
 program
   .command('run')
@@ -21,10 +27,13 @@ program
   .action(async (options) => {
     try {
       installFileFetchShim();
-      console.log(`[Volvox CLI] Initializing Engine (Backend: ${options.backend})...`);
+      console.log(`[Volvox CLI] Initializing runtime (backend: ${options.backend})...`);
       const cliDir = path.dirname(fileURLToPath(import.meta.url));
-      const wasmPath = path.resolve(cliDir, '..', 'volvoxai.wasm');
-      const engine = await VolvoxAI.init(options.backend, wasmPath);
+      const wasmPath = path.resolve(cliDir, '..', 'dist', packageVersion, 'volvoxai.wasm');
+      const runtime = await VolvoxAI.createRuntime({
+        backends: [options.backend],
+        wasmUrl: wasmPath,
+      });
 
       console.log(`[Volvox CLI] Loading Model: ${options.model}`);
       if (!fs.existsSync(options.model)) {
@@ -33,14 +42,20 @@ program
       }
       
       const modelUrl = 'file://' + path.resolve(options.model);
-      const graph = await engine.loadGraph(modelUrl);
-      
-      console.log(`[Volvox CLI] Compiling Model...`);
-      const executor = await engine.compile(graph, modelUrl);
+      const model = await runtime.loadModel(modelUrl);
 
-      console.log(`[Volvox CLI] Model Ready! Provide input data via scripts for full inference.`);
-      // TODO: In a real CLI, we would parse JSON inputs or specific tensors,
-      // and call executor.execute(inputs).
+      console.log('[Volvox CLI] Compiling model...');
+      const compiled = await model.compile({
+        backend: {
+          mode: 'require',
+          backend: options.backend,
+          operatorFallback: 'forbid',
+        },
+      });
+      console.log(`[Volvox CLI] Model ready on ${compiled.backend}.`);
+      await compiled.close();
+      await model.close();
+      await runtime.close();
       
     } catch (err) {
       console.error('[Volvox CLI] Error during execution:', err);