diskformer 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +44 -0
- package/package.json +13 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Chirag Patnaik
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
<h1 align="center">diskformer.js</h1>
|
|
2
|
+
|
|
3
|
+
**Keep model weights on disk in the browser, and page into WebGPU only what each step needs.**
|
|
4
|
+
A disk tier for in-browser inference — not another engine.
|
|
5
|
+
|
|
6
|
+
Plain ES modules. Chromium with WebGPU. No server, no build step, no telemetry.
|
|
7
|
+
|
|
8
|
+
> **Status: pre-release.** The code runs today inside [LocalMind](https://github.com/NakliTechie/LocalMind).
|
|
9
|
+
> v0.1 extracts it here with no behaviour change. `0.0.x` on npm is a name placeholder.
|
|
10
|
+
|
|
11
|
+
## Install
|
|
12
|
+
|
|
13
|
+
| How | Command |
|
|
14
|
+
|---|---|
|
|
15
|
+
| npm | `npm install diskformer` *(from v0.1)* |
|
|
16
|
+
| CDN | `import { … } from 'https://cdn.jsdelivr.net/npm/diskformer/+esm'` *(from v0.1)* |
|
|
17
|
+
|
|
18
|
+
TODO(v0.1): the first call — copy a weights file into OPFS once, open a GPU row cache or expert pool on your
|
|
19
|
+
`GPUDevice`, and read what a step needs.
|
|
20
|
+
|
|
21
|
+
## Why
|
|
22
|
+
|
|
23
|
+
Your model does not fit on the GPU, or it fits only by crowding out everything else. Browser engines load every
|
|
24
|
+
weight onto the GPU at load time, including the ones a step never touches: Gemma 4's per-layer embedding table is
|
|
25
|
+
read one row per token, and a mixture-of-experts model uses under a tenth of its weights per token.
|
|
26
|
+
|
|
27
|
+
diskformer.js keeps those weights in OPFS — the browser's private file system, read through sync access handles in
|
|
28
|
+
workers — and pages into a GPU cache only what each step needs. What it does today, inside LocalMind:
|
|
29
|
+
|
|
30
|
+
- **Gemma 4 E2B:** the 1.2 GB per-layer embedding table on disk; Chrome's GPU process drops from 4.27 GB to 2.07 GB on
|
|
31
|
+
the live site, with identical output.
|
|
32
|
+
- **Qwen3.6-35B-A3B (36.9 GB), experimental:** experts paged from OPFS on a 24 GB Mac; greedy output matches
|
|
33
|
+
llama.cpp's Metal backend 8/8 on 4- and 16-layer cuts of the same GGUF.
|
|
34
|
+
|
|
35
|
+
## Context
|
|
36
|
+
|
|
37
|
+
Extracted from LocalMind's SSD-streaming work (2026-10-02 to 2026-10-04): `opfs-reader.js` (OPFS reader pool and
|
|
38
|
+
writer), `moe-expert-stream.js` (GPU slot pool with eviction, pins, prefetch and mapped staging-ring uploads) and
|
|
39
|
+
`ple-opfs.js` (row file and GPU row cache). v0.1 scope is those two layers — store and residency — moved here with no
|
|
40
|
+
behaviour change and verified by LocalMind's existing gates.
|
|
41
|
+
|
|
42
|
+
## License
|
|
43
|
+
|
|
44
|
+
MIT · LocalMind: [github.com/NakliTechie/LocalMind](https://github.com/NakliTechie/LocalMind)
|
package/package.json
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "diskformer",
|
|
3
|
+
"version": "0.0.1",
|
|
4
|
+
"description": "A disk tier for in-browser inference: keep model weights in OPFS and page into WebGPU only what each step needs. Pre-release name placeholder; v0.1 is being extracted from LocalMind.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"license": "MIT",
|
|
7
|
+
"author": "Chirag Patnaik",
|
|
8
|
+
"homepage": "https://github.com/NakliTechie/diskformer.js#readme",
|
|
9
|
+
"repository": { "type": "git", "url": "git+https://github.com/NakliTechie/diskformer.js.git" },
|
|
10
|
+
"bugs": { "url": "https://github.com/NakliTechie/diskformer.js/issues" },
|
|
11
|
+
"keywords": ["webgpu", "opfs", "llm", "inference", "offload", "moe", "browser", "transformers"],
|
|
12
|
+
"files": ["README.md", "LICENSE"]
|
|
13
|
+
}
|