economize 0.3.0__tar.gz

economize-0.3.0/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.pyc
+.pytest_cache/
+*.egg-info/
+build/
+dist/
+.venv*/
+.mypy_cache/
+.ruff_cache/

economize-0.3.0/CHANGELOG.md

@@ -0,0 +1,13 @@
+# Changelog
+
+## 0.3.0 (2026-10-11)
+
+First release on PyPI. The library was developed as `effkit` and is renamed to `economize`: the import name, the
+distribution, the entry-point groups (`economize.techniques`, `economize.formats`, ...), the checkpoint files
+(`economize_config.json`, `economize_model.safetensors`), the exported drafter block (`economize_jump`) and the vLLM
+draft architecture (`EconomizeJumpEagle3`) all carry the new name.
+
+Checkpoints saved under the old name (`effkit_config.json`, `effkit_model.safetensors`) load as before through the
+`"effkit"` checkpoint format, and `JumpDrafterSpec.from_hf_config` still reads an `effkit_jump` block. Drafters
+exported for vLLM before the rename name the old draft architecture: export them again with
+`economize.backends.export_drafter`.

economize-0.3.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

economize-0.3.0/PKG-INFO

@@ -0,0 +1,372 @@
+Metadata-Version: 2.5
+Name: economize
+Version: 0.3.0
+Summary: Composable efficiency techniques (cross-layer jump drafting, early exit, speculative decoding, ...) for Hugging Face causal LMs.
+Project-URL: Homepage, https://github.com/Thiggel/economize
+Project-URL: Repository, https://github.com/Thiggel/economize
+Project-URL: Issues, https://github.com/Thiggel/economize/issues
+Project-URL: Documentation, https://github.com/Thiggel/economize/tree/main/docs
+Author-email: Filipe Laitenberger <f.laitenberger@gmail.com>
+License-Expression: Apache-2.0
+License-File: LICENSE
+Keywords: early-exit,efficiency,huggingface,inference,kv-cache,layer-skipping,llm,pruning,quantization,speculative-decoding,transformers,vllm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: huggingface-hub>=1.0
+Requires-Dist: safetensors>=0.4
+Requires-Dist: torch>=2.4
+Requires-Dist: transformers>=5.0
+Provides-Extra: all
+Requires-Dist: accelerate>=1.14; extra == 'all'
+Requires-Dist: compressed-tensors>=0.19; extra == 'all'
+Requires-Dist: datasets>=5.0; extra == 'all'
+Requires-Dist: kernels>=0.17; extra == 'all'
+Requires-Dist: lm-eval[hf]>=0.4.13; extra == 'all'
+Requires-Dist: peft>=0.21; extra == 'all'
+Requires-Dist: torchao>=0.18; extra == 'all'
+Requires-Dist: trl>=1.14; extra == 'all'
+Provides-Extra: bench
+Requires-Dist: datasets>=5.0; extra == 'bench'
+Provides-Extra: dev
+Requires-Dist: mypy==2.4.0; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff==0.16.10; extra == 'dev'
+Provides-Extra: eval
+Requires-Dist: lm-eval[hf]>=0.4.13; extra == 'eval'
+Provides-Extra: kernels
+Requires-Dist: kernels>=0.17; extra == 'kernels'
+Provides-Extra: peft
+Requires-Dist: peft>=0.21; extra == 'peft'
+Provides-Extra: quant
+Requires-Dist: compressed-tensors>=0.19; extra == 'quant'
+Requires-Dist: torchao>=0.18; extra == 'quant'
+Provides-Extra: sglang
+Requires-Dist: sglang; extra == 'sglang'
+Provides-Extra: test
+Requires-Dist: pytest>=8; extra == 'test'
+Provides-Extra: train
+Requires-Dist: accelerate>=1.14; extra == 'train'
+Requires-Dist: datasets>=5.0; extra == 'train'
+Requires-Dist: trl>=1.14; extra == 'train'
+Provides-Extra: vllm
+Requires-Dist: vllm>=0.30; extra == 'vllm'
+Description-Content-Type: text/markdown
+
+# economize
+
+Composable efficiency techniques for Hugging Face causal language models.
+
+economize attaches techniques to an unmodified `transformers` model. The base model keeps its module tree and its
+`forward`. Drafters and other added components bring their own parameters and act only while decoding; weight
+transforms (pruning, low rank, quantization) rewrite the base weights in place, once, when they attach. Any set of
+registered techniques can be combined in one call, and every architecture adapter, technique, verifier, cache and
+checkpoint format is added by registration.
+
+Version 0.3 ships cross-layer jump drafting, static depth pruning, Wanda / SparseGPT pruning, SVD and SVD-LLM low
+rank, GPTQ / AWQ / round-to-nearest quantization, TEAL activation sparsity and draft-only quantization, on 44 model
+types (dense, MoE, and hybrid attention plus Mamba / linear-attention models), with batched decoding of ragged
+prompts, draft trees and lossless speculative sampling.
+
+## Install
+
+```bash
+pip install economize                       # torch>=2.4, transformers>=5.0, safetensors, huggingface_hub
+pip install "economize[all]"                # every optional integration below except vLLM and SGLang
+```
+
+From source:
+
+```bash
+git clone https://github.com/Thiggel/economize.git
+pip install -e "./economize[dev]"
+```
+
+| Extra | Installs | For |
+|---|---|---|
+| `test` | pytest | the test suite |
+| `peft` | peft | PEFT-wrapped base models and LoRA training of the base |
+| `quant` | torchao, compressed-tensors | quantized kernels, compressed-tensors export |
+| `kernels` | kernels | Hugging Face `kernels` hub kernels (`economize.integrations.kernels`) |
+| `train` | accelerate, datasets, trl | drafter training and online RL rollouts |
+| `eval` | lm-eval[hf] | `economize-lm-eval`, lm-evaluation-harness with economize's decoding loop |
+| `bench` | datasets | prompt suites of `economize.bench` (Spec-Bench, GSM8K, HumanEval) |
+| `vllm`, `sglang` | vllm, sglang | serving backends (`economize.backends`; drafters are written by `economize.backends.export_drafter`) |
+
+Installing the package registers the vLLM plugin (`vllm.general_plugins`), the compressed-tensors quantization
+backend and the `economize-lm-eval` command through entry points.
+
+## Quick start
+
+```python
+import torch
+import economize
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B", dtype=torch.bfloat16, device_map="cuda")
+model = economize.load(base, "path/to/drafter")            # a trained drafter: a folder or a Hub repo id
+# or in one step, loading the base model named in the checkpoint:
+model = economize.from_pretrained("path/to/drafter", dtype=torch.bfloat16, device_map="cuda")
+
+tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B")
+ids = tokenizer("The water cycle describes", return_tensors="pt").input_ids.cuda()
+out = model.generate(ids, max_new_tokens=128)             # lossless: the same tokens as base.generate
+```
+
+`model.generate` accepts the usual `transformers` arguments (logits processors such as `repetition_penalty`,
+`eos_token_id`, `return_dict_in_generate`, ...) and routes them through economize's decoding loop. `model.decode`
+returns the tokens together with a log of every model call and the acceptance of each cycle:
+
+```python
+result = model.decode(ids[0], 128, decoding=economize.DecodeConfig(cycle_length=4))
+result.cost_summary()      # exact passes, draft steps, layer applications relative to plain decoding
+result.mean_accepted       # accepted drafts per cycle
+```
+
+## Jump drafting
+
+A draft step for the next token predicts its hidden state after layer `m` from the current token's state after
+layer `u` and the next token's embedding, runs only layers `m+1..r` and reads out logits. Layers `1..m` (and
+`r+1..L`) are skipped. One *cycle* of `N = cycle_length` tokens runs the unmodified model once, in parallel, over
+every token not yet computed exactly, which also yields one exact token, followed by up to `N-1` draft steps.
+
+```python
+config = economize.JumpConfig(
+    target_layer=12,            # m
+    source_layer=18,            # u
+    exit_layer=None,            # r (default: the last layer; below it a readout adapter is added)
+    adaptation="frozen",        # or "lora" (LoRA active only on the jump path) or "full" (trainable layer copies)
+)
+model = economize.apply(base, config)
+list(model.technique_parameters())   # the added parameters only; the base model is untouched
+```
+
+`DecodeConfig` chooses how drafts are used:
+
+| Option | Meaning |
+|---|---|
+| `cycle_length` | Tokens per cycle `N` (`1` is ordinary decoding). |
+| `lossless=True` | Speculative decoding: drafts are verified by the exact pass, the output equals greedy decoding. |
+| `lossless=False` | Every draft is kept and the exact pass only refreshes the cache (lossy, cheaper). |
+| `min_confidence` | Stop drafting in a cycle once the drafter's top probability falls below this value. |
+| `verifier` | A registered verifier by name: `"greedy"`, `"rejection"` (lossless speculative sampling at T > 0, chains and trees), `"typical"` (Medusa's typical acceptance, lossy) or `"accept_all"`. |
+| `verifier_options` | Keyword options of the verifier, e.g. `{"threshold": 0.09, "alpha": 0.3}` for `"typical"`. |
+| `cuda_graphs` | Replay the exact passes and the draft steps from CUDA graphs (see below). |
+| `sync_interval` | Ordinary decoding reads tokens back and checks stop conditions every this many steps (default 8; the output is the same as reading every step). |
+
+Other conditioning variants are part of `JumpConfig`: several source layers (`extra_source_layers`) combined by a
+registered `mixer` (`"concat"`, `"fused_linear"`, `"layer_attention"`), the predictor (`"mlp"`, `"linear"`, its
+width and depth, or `"flow"` / `"flow_linear"`: that mean followed by a few Euler steps of a rectified flow, trained by
+flow matching, with `flow_steps`, `flow_sigma`, `flow_width`), and ablations (`use_state`, `use_token`,
+`token_residual`).
+
+Several prompts of different lengths decode as one batch, each row accepting its own number of drafts per cycle.
+With `verifier="rejection"` and a sampling selector the output follows the model's own sampling distribution
+(speculative sampling):
+
+```python
+results = model.decode_batch([ids_a, ids_b, ids_c], 128)          # one DecodeResult per prompt
+sampled = model.decode_batch(
+    [ids_a, ids_b], 128,
+    decoding=economize.DecodeConfig(cycle_length=4, verifier="rejection"),
+    selector=economize.TopPSampler(temperature=0.7, top_p=0.9, seed=0, device="cuda"),
+)
+```
+
+Each row may also have its own selector, stop condition and token limit (`selectors=`, `stops=`, and a list for
+`max_new_tokens`). Rows that share a selector object are drafted and verified together.
+
+At batch size 1 a draft step and a short exact pass are bound by kernel launches. With
+`DecodeConfig(cuda_graphs=True)` the decoder records its exact chain passes and the jump drafter's steps as CUDA
+graphs on a static KV cache and replays them. The model keeps one decoder per configuration, so the graphs recorded
+in the first generation serve the following ones; `model.refresh()` drops them after the weights or the techniques
+change. Draft trees, dynamic-depth policies, KV-cache techniques and models with recurrent layers decode eagerly
+whatever the flag says.
+
+## Combining techniques
+
+`economize.apply` takes any number of configurations and attaches them in a fixed order, whatever the order of the
+arguments: weight transforms (by their `order`: depth pruning, pruning, low rank, quantization), activation hooks,
+KV-cache wrappers, dynamic depth, the drafter, then the rest. Techniques that need data calibrate on the model as it
+is at that point, so quantization sees the pruned weights.
+
+```python
+from economize.techniques.prune import PruneConfig
+from economize.techniques.quant import QuantConfig
+from economize.techniques.sparsity import TealConfig
+
+model = economize.apply(
+    base,
+    PruneConfig(method="wanda", sparsity="2:4"),
+    QuantConfig(format="int4", method="gptq", group_size=128),
+    TealConfig(sparsity=0.4),
+    economize.JumpConfig(target_layer=12, source_layer=18),
+    calibration=calibration_ids,          # LongTensor [N, S], an iterable of batches, or a callable
+)
+```
+
+See [docs/weights.md](https://github.com/Thiggel/economize/blob/main/docs/weights.md) for every weight and activation technique and its options.
+
+## Training
+
+`economize.train` trains what the techniques add (drafters, readout adapters, LoRA on a drafter's path, early-exit
+heads) and, where a technique needs it, the base model:
+
+```python
+from economize.train import fit
+
+model = economize.apply(base, economize.JumpConfig(target_layer=16, source_layer=17, exit_layer=17))
+fit(model, "data/qwen/train.bin", steps=2000, output_dir="runs/jump_m16_u17")   # uint32 token file
+model = economize.load(base, "runs/jump_m16_u17/final")
+```
+
+`fit` picks the objective registered for the attached drafter (`jump_rollout` for `jump`, the objective of
+`tskip/train.py`). An existing `transformers.Trainer` pipeline uses `EffTrainer` with the same objectives:
+
+```python
+from economize.train import EffTrainer, JumpRollout, economize_training_args
+
+objective = JumpRollout(horizon=16)
+args = economize_training_args(objective, "runs/jump_trainer", max_steps=2000, per_device_train_batch_size=8)
+EffTrainer(model, objective, args, train_dataset=dataset).train()
+```
+
+`pip install "economize[train]"` adds `accelerate`, `datasets` and `trl` (`fit` itself needs only torch).
+[docs/training.md](https://github.com/Thiggel/economize/blob/main/docs/training.md) lists the objectives, the data formats, data-parallel runs under `accelerate launch` and how
+to register an objective.
+
+## Ecosystem
+
+| Library | Entry point | Details |
+|---|---|---|
+| `transformers` | `model.generate(...)`, `custom_generate` Hub recipe | logits processors, stopping criteria, `return_dict_in_generate` |
+| Hugging Face Hub | `model.save_pretrained`, `push_to_hub`, `economize.from_pretrained` | only the technique weights are uploaded |
+| PEFT | `economize.apply(peft_model, ...)` | the adapter finds the base model inside a `PeftModel` |
+| `kernels` | `economize.integrations.kernels.kernelize(model)`, `op(name)` | hub kernels for the base model and every added module |
+| lm-evaluation-harness | `economize-lm-eval run --model economize --model_args pretrained=...,economize=...` | generation runs economize's decoding loop |
+| TRL | `economize.integrations.trl.attach_rollouts` | in-process GRPO / RLOO rollouts with the drafter refreshed after each step |
+| vLLM | `economize.backends.vllm` | drafter export (`export_drafter(model, dir, backend="vllm")`), plugin entry point, jump drafting in the V2 model runner (tested with vLLM 0.30) |
+| SGLang | `economize.backends.sglang` | interface only (`JumpDraftBackend`); its registered exporter raises `UnsupportedBackendError`, and nothing is patched in SGLang yet |
+
+[docs/ecosystem.md](https://github.com/Thiggel/economize/blob/main/docs/ecosystem.md) describes each integration and its tests.
+
+## Saving and loading
+
+```python
+model.save_pretrained("my_drafter")       # economize_config.json + technique weights in safetensors
+model = economize.load(base, "my_drafter")
+```
+
+Only the technique weights are saved. Checkpoints of the research code (`tskip`, a `jump.pt` file or its run
+directory) are detected and converted on load. Further formats are added through the `FORMATS` registry.
+
+## Extending economize
+
+Everything that varies is a registry, and new entries are added by registration, never by editing economize:
+
+| Registry | Module | Entry-point group |
+|---|---|---|
+| `TECHNIQUES` | `economize.core` | `economize.techniques` |
+| `ARCHITECTURES`, `LAYER_KINDS` | `economize.arch` | `economize.architectures`, `economize.layer_kinds` |
+| `FORMATS` | `economize.core` | `economize.formats` |
+| `VERIFIERS`, `POLICIES` | `economize.decoding` | `economize.verifiers`, `economize.draft_policies` |
+| `CACHES`, `MASKS`, `MASK_FORMATS` | `economize.runtime` | `economize.caches`, `economize.masks`, `economize.mask_formats` |
+| `PREDICTORS`, `MIXERS` | `economize.techniques.jump` | `economize.jump_predictors`, `economize.jump_mixers` |
+| `SKIP_SEARCHES`, `SKIP_METRICS` | `economize.techniques.layer_skip` | `economize.skip_searches`, `economize.skip_metrics` |
+| `MEDUSA_TREES` | `economize.techniques.medusa` | `economize.medusa_trees` |
+| `MTP_FAMILIES` | `economize.techniques.vendor_mtp` | `economize.mtp_families` |
+| `CONFIDENCE_MEASURES`, `FLEXI_ROUTERS` | `economize.techniques.dynamic_depth` | `economize.calm_confidence`, `economize.flexi_routers` |
+| `PRUNE_METHODS`, `DEPTH_CRITERIA` | `economize.techniques.prune` | `economize.prune_methods`, `economize.depth_criteria` |
+| `QUANT_METHODS`, `QUANT_BACKENDS`, `QUANT_FORMATS`, `AWQ_SCALE_GROUPS`, `TORCHAO_TENSORS` | `economize.techniques.quant` | `economize.quant_methods`, `economize.quant_backends`, `economize.quant_formats`, `economize.awq_scale_groups`, `economize.torchao_tensors` |
+| `FACTORIZERS` | `economize.techniques.lowrank` | `economize.factorizers` |
+| `TEAL_BACKENDS` | `economize.techniques.sparsity` | `economize.teal_backends` |
+| `EVICTION_METHODS` | `economize.techniques.kv_evict` | `economize.kv_eviction` |
+| `KERNEL_OPS` | `economize.integrations.kernels` | `economize.kernel_ops` |
+| `OBJECTIVES`, `DIVERGENCES`, `SCHEDULES`, `STATE_LOSSES` | `economize.train` | `economize.objectives`, `economize.divergences`, `economize.schedules`, `economize.state_losses` |
+| `PROMPT_SOURCES`, `HARDWARE`, `SHAPE_READERS` | `economize.bench` | `economize.prompt_sources`, `economize.hardware`, `economize.shape_readers` |
+| `EXPORTERS` | `economize.backends` | `economize.exporters` |
+| `VLLM_DRAFTERS` | `economize.backends.vllm` | `economize.vllm_drafters` |
+| `CT_SCHEMES` | `economize.integrations.compressed_tensors` | `economize.ct_schemes` |
+
+A new predictor, for example:
+
+```python
+from economize.techniques.jump import PREDICTORS, Predictor
+
+@PREDICTORS.register("residual_linear")
+class ResidualLinear(Predictor):
+    def __init__(self, hidden_size, config):
+        super().__init__(hidden_size, config)
+        self.proj = torch.nn.Linear(2 * hidden_size, hidden_size)
+
+    def forward(self, source, token_embedding):
+        return token_embedding.float() + self.proj(torch.cat([source, token_embedding], -1).float())
+
+model = economize.apply(base, economize.JumpConfig(target_layer=12, source_layer=18, predictor="residual_linear"))
+```
+
+An installed package can contribute the same entry without an import, through the entry-point group in its
+`pyproject.toml`:
+
+```toml
+[project.entry-points."economize.jump_predictors"]
+residual_linear = "my_pkg.predictors:ResidualLinear"
+```
+
+A new technique subclasses `economize.Technique` with a frozen `TechniqueConfig` dataclass and is registered with
+`@economize.register_technique`. A drafting technique declares `Capability.DRAFTER` and returns an object that follows
+the `economize.decoding.Drafter` protocol. A new model family registers an `ArchAdapter` under its `model_type`;
+most decoder-only families share the built-in `StandardDecoderAdapter`. Family facts that techniques and the cost
+model need are set on the adapter: `layer_layout = "llama"` gives AWQ the Llama scale groups, `shape_fields`
+corrects the cost model's config reading, and `module_path(module)` names modules for exporters. A drafter is
+written for a serving backend by a `DrafterExporter` registered with
+`@economize.backends.register_exporter("<backend>", "<technique>")`; a new backend is one such class.
+[docs/ARCHITECTURE.md](https://github.com/Thiggel/economize/blob/main/docs/ARCHITECTURE.md) describes every contract.
+
+## Scope of version 0.2
+
+- Techniques: jump drafting (frozen, LoRA, full; one or several source layers mixed by concatenation, a fused
+  linear map or layer attention; MLP or linear predictors), `depth_prune`, `prune`, `lowrank`, `quant`, `teal`,
+  `draft_quant`.
+- Decoding: batched ragged prompts with per-row selectors, stops and limits, chains and draft trees, greedy
+  verification, lossless speculative sampling on chains and trees, typical acceptance, lossy drafting, fixed or
+  confidence-based draft length, CUDA graphs for chain decoding, a log of every model call for cost accounting.
+- Training: `economize.train` (`fit`, `EffTrainer`, registered objectives).
+- Checkpoints record a fingerprint of the base weights; loading onto different weights warns
+  (`economize.TeacherMismatchWarning`, off with `check_teacher=False`).
+- Not yet: flex-attention masks for draft trees (trees use `sdpa` or `eager` masks), draft trees on models with
+  recurrent layers, and the vLLM / SGLang draft step.
+
+## Tests
+
+```bash
+cd TJEPA-Eff && source env.sh
+PYTHONPATH=economize/src:economize/tests:. $PY -m pytest economize/tests            # unit, technique, integration, oracle (CPU)
+PYTHONPATH=economize/src:economize/tests:. $PY -m pytest economize/tests -m slow    # trained checkpoints on Qwen2.5-0.5B
+PYTHONPATH=economize/src:economize/tests:. $PY -m pytest economize/tests -m gpu     # batched decoding on a CUDA device
+```
+
+The lint (`uvx ruff check src tests hub`) and type (`uvx mypy --python-executable <python> src/economize`) gates and
+the design rules a change must follow are in [docs/CONTRIBUTING.md](https://github.com/Thiggel/economize/blob/main/docs/CONTRIBUTING.md).
+
+The unit tests check that the exact path equals the model's own logits on every supported family, that batched
+lossless decoding equals greedy decoding of each prompt alone (chains, trees and jump drafters, with partial
+acceptance and hybrid recurrent layers), that rejection sampling outputs the target distribution (chi-square
+tests with a negative control), the recipe's ordering and rollback, and the checkpoint round trips. The oracle
+tests check that economize produces the same tokens, the same log of model calls and the same acceptance as the
+research decoder `tskip.decode.Decoder`, on random drafters of a tiny model and on three trained drafters of
+Qwen2.5-0.5B.
+
+## License
+
+Apache License 2.0, see [LICENSE](https://github.com/Thiggel/economize/blob/main/LICENSE).