neurarch 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,96 @@
+Sustainable Use License
+
+Version 1.0
+
+## Acceptance
+
+By using the software, you agree to all of the terms and conditions below.
+
+## Copyright License
+
+The licensor grants you a non-exclusive, royalty-free, worldwide,
+non-sublicensable, non-transferable license to use, copy, distribute, make
+available, and prepare derivative works of the software, in each case subject
+to the limitations below.
+
+## Limitations
+
+You may use or modify the software only for your own internal business purposes
+or for non-commercial or personal use. You may distribute the software or
+provide it to others only if you do so free of charge for non-commercial
+purposes. You may not alter, remove, or obscure any licensing, copyright, or
+other notices of the licensor in the software. Any use of the licensor's
+trademarks is subject to applicable law.
+
+## Patents
+
+The licensor grants you a license, under any patent claims the licensor can
+license, or becomes able to license, to make, have made, use, sell, offer for
+sale, import and have imported the software, in each case subject to the
+limitations and conditions in this license. This license does not cover any
+patent claims that you cause to be infringed by modifications or additions to
+the software. If you or your company make any written claim that the software
+infringes or contributes to infringement of any patent, your patent license for
+the software granted under these terms ends immediately. If your company makes
+such a claim, your patent license ends immediately for work on behalf of your
+company.
+
+## Notices
+
+You must ensure that anyone who gets a copy of any part of the software from you
+also gets a copy of these terms.
+
+If you modify the software, you must include in any modified copies of the
+software prominent notices stating that you have modified the software.
+
+## No Other Rights
+
+These terms do not imply any licenses other than those expressly granted in
+these terms.
+
+## Termination
+
+If you use the software in violation of these terms, such use is not licensed,
+and your licenses will automatically terminate. If the licensor provides you
+with a notice of your violation, and you cease all violation of this license no
+later than 30 days after you receive that notice, your licenses will be
+reinstated retroactively. However, if you violate these terms after such
+reinstatement, any additional violation of these terms will cause your licenses
+to terminate automatically and permanently.
+
+## No Liability
+
+As far as the law allows, the software comes as is, without any warranty or
+condition, and the licensor will not be liable to you for any damages arising
+out of these terms or the use or nature of the software, under any kind of
+legal claim.
+
+## Definitions
+
+The "licensor" is the entity offering these terms.
+
+The "software" is the software the licensor makes available under these terms,
+including any portion of it.
+
+"You" refers to the individual or entity agreeing to these terms.
+
+"Your company" is any legal entity, sole proprietorship, or other kind of
+organization that you work for, plus all organizations that have control over,
+are under the control of, or are under common control with that organization.
+"Control" means ownership of substantially all the assets of an entity, or the
+power to direct its management and policies by vote, contract, or otherwise.
+Control can be direct or indirect.
+
+"Your licenses" are all the licenses granted to you for the software under these
+terms.
+
+"Use" means anything you do with the software requiring one of your licenses.
+
+"Trademark" means trademarks, service marks, and similar rights.
+
+---
+
+Copyright (c) 2026 neurarch-ai
+
+This license is adapted from the Sustainable Use License, which is published by
+n8n GmbH under CC BY-SA 4.0 and is intended to be reused.

package/README.md

@@ -1,3 +1,248 @@
-# Temporary Holding Version
+# neurarch
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+What a GPU training job needs before it takes a reservation: the peak memory at
+its batch, the smallest card that holds it, and what one run costs, read from a
+Hugging Face `config.json` (or a traced graph). No account, no download, no GPU.
+
+```
+npx neurarch rightsize jobs.csv          # each past job's request next to what it needed
+npx neurarch Qwen/Qwen3-8B --batch 8     # the training peak for one model
+npx neurarch oom error.txt --model Qwen/Qwen3-8B --batch 8   # why it ran out of memory, and the least invasive fix
+npx neurarch Qwen/Qwen3-8B --inference --batch 4 --seq-len 4096 --gen 512   # weights + KV cache for a generation job
+```
+
+The memory number is calibrated against measured runs, and every run is public
+next to `accelerate estimate-memory`: <https://neurarch.com/memory-calibration.html>.
+
+It also reads any Hugging Face model as an architecture:
+
+```
+npx neurarch microsoft/Phi-3-mini-4k-instruct
+```
+
+```
+microsoft/Phi-3-mini-4k-instruct  phi3 · text-generation
+
+  published (safetensors.total)  3.82B  3,821,079,552
+  derived (config.json alone)    3.82B  3,821,111,616
+  difference                     +0.00%  within 2%: the reconstruction matches the checkpoint
+
+  layers (198, showing 12; --layers for all)
+      1  Input                 Input                 1 × 4096
+      2  Embedding             Embedding             1 × 4096 × 3072
+      ...
+
+  checks  clean
+```
+
+## The number this exists to print
+
+The second one. `derived` is computed from `config.json` alone: the layer graph
+is reconstructed from the config, shapes are propagated through it, and the
+parameters are counted. `published` is `safetensors.total` from the Hub API,
+the scalars actually stored in the weight files.
+
+They come from different places, so the gap between them is the only honest
+statement about whether the reconstruction is worth anything. Everything else
+the tool prints, the layer table and the checks, is downstream of that.
+
+The tolerance is 2%, chosen before the numbers were looked at because it is
+smaller than one layer in every model in the reference set. Agreement across
+the published set lives at <https://neurarch.com/m/>.
+
+## What one training run would cost
+
+```
+npx neurarch bert-base-uncased --data 3M --gpu h100-80gb
+```
+
+```
+  one training run over 3,000,000 samples
+    quoted          $31  on H100 (80GB), 8.0 h
+    fits            T4  A100_40  H100_80  (training peak ~4.2 GB at batch 32, 512 tokens, 1.15x headroom)
+    tokens          512 assumed (default 2048, or the context if shorter); --seq-len to change
+    attention       SDPA, scores not stored; eager would add 8.4 GB
+    derived, not measured: nothing here was trained
+```
+
+`--data` is the size of YOUR training set (`3M`, `500k`, `2B`). It is required
+for a price, and that is deliberate: with no workload the estimate defaults to
+50,000 rows, which is nobody's training set and would make the largest number
+on the screen the one nobody stated. Same model, same card, three sizes:
+
+| `--data` | quoted | wall clock |
+|---|---|---|
+| 50k | $0.59 | 0.1 h |
+| 3M | $31 | 8.0 h |
+| 40M | $418 | 4.4 days |
+
+`--gpu` is one of `t4-16gb`, `l4-24gb`, `a10g-24gb`, `l40s-48gb`, `a100-40gb`,
+`a100-80gb`, `h100-80gb`; a prefix works when it is unambiguous (`t4`, `h100`),
+and `a100` is refused with both options rather than resolved by array order.
+
+`--batch 8` sizes the training peak at that batch (default 32). `--attention` is
+`sdpa` (the default, and Hugging Face's own), `flash` or `eager`. Only eager
+keeps the attention score matrices for backward, so only eager adds them to the
+peak; under the other two the line says how many GB eager would add. On a token
+model at 2048 tokens that term is most of the activation memory, so it is worth
+setting when your training script sets `attn_implementation="eager"`. The
+same option is the `attention` field on `POST /api/v1/plan` and the `attention`
+column of `rightsize`, so all three size one model the same way.
+
+`--seq-len 8192` sizes a text model at that many tokens per sample. The default
+is 2048, or the model's context when that is shorter (BERT: 512). It is not
+`max_position_embeddings`: that is the longest input the model accepts, and
+Qwen3-0.6B declares 40,960, which put a ~191 GB peak at batch 8 on a model
+people fine-tune on one card. At 2048 tokens the same line reads ~21 GB. The
+default is the one `rightsize` uses for a blank `seq_len` and the `seq_len`
+field on `POST /api/v1/plan` (which the MCP `plan` tool calls), and the line
+always says the length it used. `--batch`, `--seq-len` or `--attention` without
+`--data` print the training peak with no price.
+
+`--inference` sizes a generation job instead: weights at `--precision` (bf16 by
+default; fp16, fp32, int8, int4) plus the KV cache at the last generated token,
+`--seq-len` prompt tokens plus `--gen` generated ones (default 256) at `--batch`
+(default 1), with the same 1.15x headroom. The KV cache is the same per-token
+formula the change record's serving line uses. It has no activation term, and it
+prints what that leaves out (the prefill working set and prompt logits, the CUDA
+context and any framework reserve). `npx neurarch oom --inference` compares an
+error against this peak instead of the training one.
+
+A repo with no safetensors index (InternLM-7B, xlnet-large-cased) has no
+published count, so the plain read refuses it. With any sizing flag it is still
+sized from `config.json`, under a line saying there was nothing to compare.
+
+Everything in that block is arithmetic over the reconstructed graph at list GPU
+prices. Nothing was trained and nothing was measured, which is why the verb is
+"quoted": what it is good for is deciding whether to start the run, not
+predicting what comes out of it.
+
+## Which modules a LoRA can sit on
+
+```
+npx neurarch Qwen/Qwen3-0.6B --lora-targets
+```
+
+```
+  LoRA targets   base 596.0M params, 196 linear modules
+    attention    r=8 +2.3M (0.38%)  r=16 +4.6M (0.77%)  over 112 modules
+                 target_modules=["q_proj","k_proj","v_proj","o_proj"]
+    all-linear   r=8 +5.0M (0.85%)  r=16 +10.1M (1.69%)  over 196 modules
+                 target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"]
+    by role      q_proj 28 · k_proj 28 · v_proj 28 · o_proj 28 · gate 28 · up 28 · down 28
+```
+
+Every Linear in the reconstructed graph, with attention opened into its q, k,
+v and o projections (GQA's narrower k and v included) and SwiGLU into gate, up
+and down. A rank-r adapter adds r x (in + out) per module; the output head is
+left out of all-linear, as PEFT does. `--json --lora-targets` returns the
+per-module list. It says what each preset costs, not which one trains better.
+
+## Requested vs needed, over your past jobs
+
+```
+npx neurarch rightsize jobs.csv --out report.md
+```
+
+```
+6 jobs, 174 GPU-hours, $507.85 spent; a right-sized request would have saved $404.64 (80% of spend), claiming nothing on 1 row marked under or unknown.
+```
+
+For each job in the CSV: what it requested (card x count), what it needed
+(the training peak the plan predicts at that batch), the smallest card that
+holds it, a verdict, and the dollars a right-sized request would have saved
+over the hours the reservation was held.
+
+| verdict | means |
+|---|---|
+| `right-sized` | the request is the smallest capacity that holds the peak with 1.15x headroom, or one size up |
+| `over` | a smaller card or fewer cards would have held it; the saving is priced on the cheapest one that does |
+| `under` | the predicted peak does not fit what was requested: the job OOMed or relied on something the graph does not declare. No saving claimed |
+| `unknown` | no graph (bad id, unreadable config, a card name it cannot resolve); the row says why. No saving claimed |
+
+The CSV has a header row; column order is free. A sample is in
+[`examples/jobs.sample.csv`](examples/jobs.sample.csv):
+
+```csv
+job_id,model,gpu,batch,hours,seq_len,lora_rank
+ft-qwen-small,Qwen/Qwen3-0.6B,8xH100,8,6,2048,
+lora-qwen-8b,Qwen/Qwen3-8B,A100-80,4,12,2048,16
+bert-classifier,bert-base-uncased,A100-80,32,3,512,
+tinyllama-lora,TinyLlama/TinyLlama-1.1B-Chat-v1.0,H100,8,5,2048,8
+phi3-sft,microsoft/Phi-3-mini-4k-instruct,A100-40,8,10,4096,
+gpt2-pretrain,openai-community/gpt2,4xA100-80,16,24,1024,
+```
+
+| column | required | |
+|---|---|---|
+| `job_id` | yes | any label |
+| `model` | yes | a Hugging Face id, or a path to a `.neurarch.json` graph written by `neurarch-trace` |
+| `gpu` | yes | `T4`, `L4`, `A10G`, `A100-40`, `A100-80`, `L40S`, `H100`, `H200`, or a count like `8xH100`. A bare `A100` is refused: the two differ by 2x in memory |
+| `gpus` | no | card count, instead of `8xH100` |
+| `hours` | yes | wall hours the job held the reservation |
+| `batch` | no | per-device batch, default 32 |
+| `seq_len` | no | sequence length for token models; default 2048, or the model's context if shorter |
+| `lora_rank` | no | size the base frozen in bf16 (2 bytes per parameter) and only adapters (on every Linear but the head) as trainable |
+| `attention` | no | `sdpa` (default), `flash` or `eager`. Only eager keeps the attention score matrices for backward |
+
+`--prices prices.json` overrides the default list prices per GPU-hour
+(`{"H100": 2.49, "A100-80": 1.79}`); cards it does not name keep theirs, and the
+report says which were overridden. `--json` returns every row with the plan's
+own `peakGb`, `smallestFitGpu`, `maxBatch` and `twoCardSplit`.
+
+**The memory model.** The peak is `buildPlan`'s training peak under AdamW with
+bf16 autocast: weights, gradients and optimizer state plus activations (every
+layer's output, the intermediates inside each layer, recurrent gate states, the
+largest tensor's backward copy and, under eager attention, stored scores),
+fitted on 178 measured runs. Held out one architecture at a time, the median
+error is 7% on runs of 1 GB or more and it flagged 113 of the 131 times a run
+did not fit a card; on 25 runs at the edge of a 24 GB card measured after the
+fit, the median miss is 14%. Inference jobs and other optimizers will differ. A multi-card job
+is sized as one model sharded across its cards (FSDP / ZeRO-3): the peak at the
+global batch against the summed capacity. Default prices are Modal's on-demand
+list prices, the same table the Neurarch app prices training with.
+
+**What it does not know**: data loader memory, custom kernels beyond the
+attention column, activation checkpointing (a checkpointed job needs less),
+multi-node communication buffers, and throughput: the saving assumes the same
+wall hours, so a request cut from 8 cards to 1 is capacity reserved and not
+needed, not a promise the same job finishes as fast. The report prints all of
+this next to the numbers.
+
+## What it refuses to do
+
+- **No safetensors index on the Hub** → it prints nothing and says so. A
+  derived count with nothing to check it against looks authoritative and means
+  nothing.
+- **A quantized checkpoint** → the counts are reported and explicitly marked
+  not comparable. Packed storage counts stored elements under a packing scheme,
+  not logical parameters, so the delta measures the packing, not us.
+- **A multi-tower config** (CLIP, LLaVA) → same, with the sub-models named. The
+  checkpoint carries every tower; the graph carries what the reader
+  reconstructs, so a gap is a statement about what we read.
+- **Weight sharing** (`num_hidden_groups`) → the graph counts blocks in the
+  forward pass, so a structural count is necessarily larger than the file.
+
+## Usage
+
+```
+npx neurarch <model-id>            # bert-base-uncased, Qwen/Qwen3-8B, ...
+npx neurarch <model-id> --layers   # every layer, not the first 12
+npx neurarch <model-id> --json     # machine-readable
+npx neurarch rightsize jobs.csv    # requested vs needed GPU over past jobs
+```
+
+Exit code is `1` when the model cannot be read or a check blocks, `0`
+otherwise, so it composes in a script.
+
+Set `HF_TOKEN` to read a gated repo. The token is sent only to
+`huggingface.co`. Nothing is uploaded anywhere and nothing is stored.
+
+## Related
+
+- <https://neurarch.com/compare> — the same comparison for two models, in a
+  browser, no account
+- `POST https://www.neurarch.com/api/v1/check` — run the checks on a graph of
+  your own
+- [`neurarch-lint`](https://www.npmjs.com/package/neurarch-lint) — the same
+  checks against PyTorch source