hipengine 0.1.0__tar.gz → 0.2.0__tar.gz

hipengine-0.2.0/.gitattributes

@@ -0,0 +1,10 @@
+# WORKLOG.md is append-only: when two branches both add new entries, git's
+# built-in union driver concatenates both sides instead of emitting conflict
+# markers. For an append-only file this is the correct semantics (common
+# prefix + ours-tail + theirs-tail). See scripts/resolve_worklog_conflict.py
+# for the recovery path when conflict markers already exist in the file.
+WORKLOG.md merge=union
+
+hipengine/kernels/hip_gfx1100/attention/aotriton_runtime/** -whitespace
+hipengine/kernels/hip_gfx1100/attention/aotriton_runtime/**/*.aks2 filter=lfs diff=lfs merge=lfs -text
+hipengine/kernels/hip_gfx1100/attention/aotriton_runtime/**/*.so* filter=lfs diff=lfs merge=lfs -text

hipengine-0.2.0/.github/workflows/publish.yml

@@ -0,0 +1,103 @@
+name: Publish to PyPI
+
+on:
+  push:
+    tags:
+      - "v*"
+
+permissions: {}
+
+jobs:
+  build:
+    runs-on: ubuntu-24.04
+    permissions:
+      contents: read
+    outputs:
+      version: ${{ steps.meta.outputs.version }}
+
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          persist-credentials: false
+          lfs: true
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
+
+      - name: Set up Python 3.12
+        run: |
+          uv python install 3.12
+          uv venv --python 3.12
+
+      - name: Extract version metadata
+        id: meta
+        run: |
+          version="$(uv run --no-sync python - <<'PY'
+          from pathlib import Path
+          import tomllib
+
+          data = tomllib.loads(Path("pyproject.toml").read_text())
+          print(data["project"]["version"])
+          PY
+          )"
+          echo "version=${version}" >> "$GITHUB_OUTPUT"
+
+      - name: Verify tag matches package version
+        env:
+          RELEASE_TAG: ${{ github.ref_name }}
+          RELEASE_VERSION: ${{ steps.meta.outputs.version }}
+        run: |
+          expected="v${RELEASE_VERSION}"
+          if [ "${RELEASE_TAG}" != "${expected}" ]; then
+            echo "::error::Tag '${RELEASE_TAG}' does not match package version '${expected}'"
+            exit 1
+          fi
+
+      - name: Install dependencies
+        run: uv pip install -e '.[dev]' build twine
+
+      - name: Run release validation
+        run: uv run --no-sync pytest -q
+
+      - name: Remove stale build artifacts
+        run: rm -rf dist/
+
+      - name: Build
+        run: uv run --no-sync python -m build
+
+      - name: Verify package metadata
+        run: uv run --no-sync python -m twine check dist/*
+
+      - name: Upload build artifacts
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: dist
+          path: |
+            dist/hipengine-*.whl
+            dist/hipengine-*.tar.gz
+
+  publish:
+    needs: build
+    runs-on: ubuntu-24.04
+    environment: pypi-publish
+    permissions:
+      id-token: write
+      contents: read
+      attestations: write
+
+    steps:
+      - name: Download build artifacts
+        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
+        with:
+          name: dist
+          path: dist/
+
+      - name: Generate artifact attestations
+        uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
+        with:
+          subject-path: |
+            dist/hipengine-*.whl
+            dist/hipengine-*.tar.gz
+
+      - name: Publish to PyPI (trusted publishing)
+        uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0

{hipengine-0.1.0 → hipengine-0.2.0}/AGENTS.md

@@ -64,6 +64,7 @@ Do not drift these casually. They define what hipEngine is.
 
 - Keep changes scoped to one logical unit (one kernel family, one plugin, one doc, one phase milestone).
 - Write or update the targeted test/fixture before implementation when behavior or math changes. If RED-first is impractical, record why in `WORKLOG.md`.
+- When adding tests that call HIP/ROCm runtime, `hipcc`, or GPU kernels, add an explicit HIP-availability guard (for example `ctypes.CDLL("libamdhip64.so")` + `pytest.skip`) so no-ROCm CI/publish runners skip them instead of failing release validation.
 - Log non-trivial decisions, measurements, and dependency additions in `WORKLOG.md` as they happen.
 - When profiling Python/ctypes JIT-built kernels with `rocprofv3`, prebuild the `.so` outside the profiler and run the profiled command with a precomputed compiler-version file plus `require_cached`; do not let the profiled process spawn `hipcc`/clang.
 - Do not silently add `import torch`, `flash_attn`, or other CUDA-only deps to hot-path modules.
@@ -144,6 +145,7 @@ Working tree is shared state. Other agents or the human may be editing concurren
 - **High-conflict files:** `AGENTS.md`, `CLAUDE.md`, `docs/PLAN.md`, `docs/BENCHMARK.md`, `docs/TESTING.md`, `docs/KERNELS.md`, `docs/IMPLEMENTATION.md`, `WORKLOG.md`, `pyproject.toml`, `hipengine/kernels/registry.py`, `hipengine/quant/registry.py`, `hipengine/models/registry.py`, `hipengine/dispatch/fusion.py`, `hipengine/core/*`.
 - Same-file contention: stop and coordinate. The designated agent stages and commits their scoped hunks first to unblock others.
 - `WORKLOG.md` appends are expected and not a conflict unless there are actual conflict markers or interleaved garbled lines. Re-read the live tail, append after it, commit with your logical unit.
+- `WORKLOG.md` is configured with git's built-in `merge=union` driver (see `.gitattributes`), so concurrent appends auto-resolve as `common prefix + ours-tail + theirs-tail` with no conflict markers. If markers do appear (e.g. a stash or a rebase started before this was configured), run `python3 scripts/resolve_worklog_conflict.py WORKLOG.md` (add `--sort-by-date` to re-order `## YYYY-MM-DD` sections in each resolved block; `--check` for a pre-commit gate). The script only touches conflict blocks; content outside markers is left exactly as-is.
 - Do not clean up another agent's benchmark outputs, staged files, or local artifacts unless the task explicitly asks for that cleanup.
 
 ## External Reference Repos

hipengine-0.2.0/CHANGELOG.md

@@ -0,0 +1,112 @@
+# Changelog
+
+All notable user-facing changes for hipEngine releases are documented here.
+
+This changelog is for package/API releases. Performance rollup history remains in
+[`benchmarks/CHANGELOG.md`](benchmarks/CHANGELOG.md), with detailed benchmark
+evidence under [`benchmarks/results/`](benchmarks/results/).
+
+## v0.2.0 - 2026-05-25
+
+Minor release for the GGUF runtime path and W7900 benchmark refresh. GGUF is a
+meaningful new model-loading surface rather than a patch-level fix, so this
+supersedes the previously planned v0.1.2 patch.
+
+### Added
+
+- Added Qwen3.6 35B MoE GGUF support for `Q4_K_M` and `Q4_K_S` model files,
+  including resident GGUF loading, bulk prefill, graph-replay decode,
+  decode-repacked T16 layouts, and WMMA/GEMV fast-path controls used by the
+  W7900 benchmark profile.
+- Added `docs/ENVS.md` as the canonical environment-variable reference, including
+  TheRock ROCm process setup, cached-build profiling guidance, and safe GGUF
+  benchmark profiles.
+- Added a persistent README sweep harness that loads each hipEngine model once
+  and runs repeated in-session workload measurements, matching llama-bench-style
+  repetition without multiplying model load/decode-repack time by every shape.
+
+### Changed
+
+- Refreshed W7900 README performance tables with 5-run persistent-session medians
+  for packed PARO and GGUF Q4_K_S while keeping the existing llama.cpp HIP/Vulkan
+  comparison rows unchanged.
+- Documented the current GGUF tradeoffs: higher one-time load cost and resident
+  memory from decode-repack, Q4_K_S preferred for tighter VRAM budgets, and
+  performance still behind PARO on some shapes while already competitive in the
+  broader W7900 comparison.
+
+### Fixed
+
+- Fixed the PARO resident prefill workspace-overlap regression that shipped in
+  v0.1.1: short and mid prompts now keep prefill workspaces resident through
+  32K tokens, restoring 512/128-class prefill throughput while retaining the
+  long-context memory-saving path for prompts above 32K when active chunking
+  splits the prompt.
+- Fixed GGUF non-split full-attention decode in max-context persistent sessions
+  by launching the context kernel with the active decode context instead of the
+  session's maximum allocation length.
+
+### Known limitations
+
+- GGUF support remains alpha: production correctness and performance coverage is
+  strongest for the documented Qwen3.6 35B MoE Q4_K_M/Q4_K_S paths on gfx1100,
+  and other GGUF quants/models require local validation.
+- GGUF model load is slower than packed PARO on the same host because current
+  decode-repack happens on load and is not yet cached on disk.
+
+## v0.1.1 - 2026-05-19
+
+Patch release focused on long-context memory documentation and the INT8 KV cache
+bring-up that landed after v0.1.0.
+
+### Added
+
+- INT8 KV cache policy controls and dispatch coverage for Qwen/PARO resident
+  inference paths, including CPU/layer/E2E correctness gates and memory audits.
+- Documented Qwen3.6 packed PARO memory rows for 128K BF16 KV, 128K INT8 KV, and
+  256K INT8 KV on W7900/gfx1100, with retained-KV and loaded-weight VRAM notes.
+
+### Changed
+
+- Reduced the 256K INT8 KV tracked allocator high-water mark below the 24 GiB
+  class target by releasing/reusing prefill scratch and AOTriton query buffers.
+- Clarified that packed vs unstripped PARO checkpoint size does not translate to
+  meaningfully different resident model-weight VRAM for the current text runtime.
+
+### Known limitations
+
+- INT8 KV correctness is gated by deterministic fixtures and layer probes; it is
+  not yet a long-rollout perplexity or compounding-error study.
+- Qwen3.6 packed throughput rows remain diagnostic pending a promoted public
+  `LLM.generate()` correctness/repetition gate.
+
+## v0.1.0 - 2026-05-18
+
+Initial public alpha release.
+
+### Added
+
+- Torch-free Python runtime hot path for local ROCm inference bring-up.
+- Plugin registries keyed by model/backend/quant/layer variants.
+- HIP backends for `gfx1100` and `gfx1151`, plus `backend="auto"` detection with
+  `HIPENGINE_BACKEND` force override guidance for nearby targets.
+- Qwen3.5/Qwen3.6 PARO W4 runtime path, JIT HIP build/cache plumbing, AOTriton
+  prefill runtime packaging, and OpenAI-compatible server entry point.
+- CPU reference kernels and focused correctness/performance documentation.
+
+### Packaging
+
+- PyPI project name: `hipengine`.
+- Python import package: `hipengine`.
+- Canonical repository/wordmark: `hipEngine`.
+- Release wheels are Linux x86-64 `manylinux_2_39` platform wheels because the
+  package bundles a ROCm/AOTriton shared-library runtime; ROCm runtime libraries
+  remain external system dependencies.
+
+### Known limitations
+
+- Alpha-quality API and model coverage; expect sharp edges outside the documented
+  Qwen/PARO paths.
+- Default supported GPU targets are `gfx1100` and `gfx1151`; other AMD targets
+  require explicit backend forcing and local validation.
+- Model weights are not distributed with the package.

{hipengine-0.1.0 → hipengine-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hipengine
-Version: 0.1.0
+Version: 0.2.0
 Summary: ROCm-native local LLM inference engine with a torch-free runtime hot path
 Project-URL: Homepage, https://github.com/shisa-ai/hipEngine
 Project-URL: Repository, https://github.com/shisa-ai/hipEngine
@@ -75,52 +75,143 @@ supported GPUs and models.
 
 ## Status
 
-**v0.1.x.** The runtime hot path is torch-free by construction; kernel
-families and registry plumbing are landing under
-[`hipengine/kernels/hip_gfx1100/`](hipengine/kernels/hip_gfx1100/). Current
-single-model tuning targets
+**v0.2.0 alpha.** The runtime hot path is torch-free by construction, and the
+first two 35B-class model-loading surfaces are now available on gfx1100:
 [shisa-ai/Qwen3.6-35B-A3B-PARO-full4096-e5-packed](https://huggingface.co/shisa-ai/Qwen3.6-35B-A3B-PARO-full4096-e5-packed)
 (19.07 GiB, 4.68 bpw) in packed
-[ParoQuant](https://github.com/shisa-ai/paroquant) format.
+[ParoQuant](https://github.com/shisa-ai/paroquant) format, plus Qwen3.6 GGUF
+`Q4_K_M` / `Q4_K_S` files through the new resident GGUF path.
 
-## gfx1100 (Radeon RX 7900 XTX / Radeon Pro W7900)
+- INT8 KV cache support has been added for PARO. Qwen 3 MoE's full 256K context window can fit in <24GB tracked memory; see [Memory Usage](#memory-usage).
+- Qwen 3.6 [Q4_K_M](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF?show_file_info=Qwen3.6-35B-A3B-UD-Q4_K_M.gguf) and [Q4_K_S](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF?show_file_info=Qwen3.6-35B-A3B-UD-IQ4_XS.gguf) GGUF support has landed (W7900 Q4_K_S sweep is in [Performance](#performance) alongside packed PARO and llama.cpp Q4_K_M HIP/Vulkan baselines). GGUF uses a substantial GGUF-specific runtime path with bulk prefill, graph decode, and on-load decode-repack into T16 tile layouts. Q4_K_S is recommended on 24 GiB cards because Q4_K_M is bigger; on the 48 GiB W7900 Q4_K_S fits all the way to 128K context, while on 24 GiB cards expect roughly 64K. GGUF also has a higher per-session load cost (~60 s vs ~24 s for PARO packed on the same hardware) for the same decode-repack reason.
+- Current gfx1100 performance snapshots are summarized in [Performance](#performance) and compared against recent llama.cpp Q4_K_M baselines.
 
-While we are far from [gfx1100 roofline](https://github.com/shisa-ai/hipEngine/blob/main/docs/ROOFLINE.md), the current gfx1100 implementation does well compared to Q4_K_M quants of recent llama.cpp builds (`b9042`) on the same model family. The latest W7900 packed rows use the default prefill policy: 512-token prompts stay unchunked and prompts above 1K use `1024/1024/4096/1024/1024` chunks.
 
-### Prefill tok/s
+## Hardware targets
+
+| Backend | Hardware | Status |
+| --- | --- | --- |
+| `cpu_reference` | Any CPU, numpy | Correctness oracle; CI without GPU |
+| `hip_gfx1100` | AMD Radeon Pro W7900 / RX 7900 XTX (RDNA3) | Active backend |
+| `hip_gfx1151` | AMD Ryzen AI MAX+ 395 / Radeon 8060S (Strix Halo, RDNA3.5) | Active backend |
+| `cuda_sm86` | NVIDIA Ampere consumer (3090-class) | Planned peer backend |
 
-| Workload | hipEngine shisa Qwen3.6 packed PARO | llama.cpp HIP | llama.cpp Vulkan |
+`backend="auto"` is the public API/server default. It maps exact `gfx1100` and
+`gfx1151` detections to the matching HIP backend; unknown ROCm targets warn and
+select `cpu_reference` where a CPU implementation exists. Users on nearby targets
+such as `gfx1101`/`gfx1102` can force a backend with `backend="hip_gfx1100"`,
+`--backend hip_gfx1100`, or `HIPENGINE_BACKEND=hip_gfx1100` after validating
+correctness/performance.
+
+Wave32 is the default for `hip_gfx1100` device code; wave64 is treated as an
+isolated experiment with its own gates (see
+[`docs/PLAN.md`](docs/PLAN.md#rdna3-wavefront-and-scheduling-caveat)).
+
+## Memory Usage
+
+With BF16 KV cache, hipEngine running the packed Qwen 3.6 PARO model fits a
+128K context window in a 24GB-class memory budget. The INT8 KV cache option
+(with FP16 per-token/per-head scales) uses the
+`--kv-storage int8_per_token_head` flag and lets the **full 256K context** fit
+under 24 GiB tracked allocator peak.
+
+The numbers below are for
+`shisa-ai/Qwen3.6-35B-A3B-PARO-full4096-e5-packed` on W7900/gfx1100 with q3072
+full-attention prefill chunks:
+
+| Model                | Context | KV cache | Sampled peak | Allocator peak | Retained KV | Prefill      | Decode     |
+| -------------------- | ------: | -------- | -----------: | -------------: | ----------: | -----------: | ---------: |
+| Qwen3.6 35B-A3B PARO |    128K | BF16     |    21.04 GiB |      21.88 GiB |    2.69 GiB | 1091.9 tok/s | 62.2 tok/s |
+| Qwen3.6 35B-A3B PARO |    128K | INT8     |    19.80 GiB |      20.89 GiB |    1.36 GiB | 1076.5 tok/s | 60.0 tok/s |
+| Qwen3.6 35B-A3B PARO |    256K | INT8     |    21.96 GiB |      23.71 GiB |    2.71 GiB |  670.2 tok/s | 40.3 tok/s |
+
+Regardless of the difference in PARO weight storage (legacy or packed),
+loaded-weight memory is about the same — approximately 16.4 GiB in VRAM.
+
+The INT8 KV correctness gate is currently the deterministic Qwen3.5 PARO
+fixture `fixtures/qwen35_paro/parent_512_32_seed1234.json` (512-token prompt,
+32 greedy decode tokens): `max_kl=0.015328`, `mean_kl=0.001639`, top-1 agreement
+100%, and generated IDs match BF16 KV exactly. Layer attention probes at context
+64 and 520 also had top-1 agreement 100% with max quantized-vs-BF16 KL
+`2.34e-7`. This is a fixture/regression gate, not a long-rollout perplexity
+study, so long context generations may have unmeasured compounding errors.
+
+The same 128K/128 Qwen3.5 BF16-vs-INT8 run measured -0.99% prefill tok/s and
+-3.20% decode tok/s for INT8 KV, so speed loss is also very small.
+
+See
+[`benchmarks/results/2026-05-19-hipengine-qwen36-packed-int8-kv-readme-memory-diagnostic.json`](benchmarks/results/2026-05-19-hipengine-qwen36-packed-int8-kv-readme-memory-diagnostic.json),
+[`benchmarks/README.md`](benchmarks/README.md#blocked--diagnostic-benchmark-attempts),
+and [`docs/KVCACHE.md`](docs/KVCACHE.md) for commands, artifacts, and the full
+no-shadow memory audit.
+
+### llama.cpp
+
+When run with `q8_0` kvcache, llama.cpp can also fit in 24GB:
+
+```bash
+--flash-attn on -ctk q8_0 -ctv q8_0 -c 262144 -b 128 -ub 128
+```
+
+Results:
+
+| Model | llama.cpp model buffer | KV cache | Compute buffer | rocm-smi VRAM used | Free VRAM |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| Q4_K_M | 20583 MiB | 2720 MiB | 203 MiB | 24017 MiB / 23.45 GiB | ~543 MiB |
+| Q4_K_S | 19399 MiB | 2720 MiB | 203 MiB | 22832 MiB / 22.30 GiB | ~1728 MiB |
+
+With `-ub 512`:
+
+| Model | Compute buffer | rocm-smi VRAM used | Free VRAM |
 | --- | ---: | ---: | ---: |
-| 512/128 | **2500.565** | 2436.049 | 1816.927 |
-| 4K/128 | **2899.685** | 2176.905 | 1705.093 |
-| 32K/128 | **2115.050** | 1496.409 | 1128.554 |
-| 128K/128 | **1054.291** | 710.213 | 480.539 |
+| Q4_K_M | 812 MiB | 24540 MiB | ~20 MiB |
+| Q4_K_S | 812 MiB | 23443 MiB | ~1117 MiB |
+
+- Note Q4_K_M is incredibly tight with only 20 MiB of headroom and you may either need to resize down or set `-b 512 -ub 128`.
+- Q4_K_S does not need small `-b`/`-ub`; `-ub 512` fits fine, and can even increase to `-b 2048` (but `-ub` is the more important VRAM knob that controls the physical microbatch / compute buffer size for llama.cpp).
+
+## Performance
+
+### gfx1100 (Radeon RX 7900 XTX / Radeon Pro W7900)
+
+While we are far from [gfx1100 roofline](https://github.com/shisa-ai/hipEngine/blob/main/docs/ROOFLINE.md), the current gfx1100 implementation does well compared to Q4_K_M quants of recent llama.cpp builds (`b9042`) on the same model family. The latest W7900 hipEngine rows use TheRock ROCm 7.13 and load each resident model once for 1 warmup + 5 measured in-session repetitions per shape. PARO uses the default prefill policy: 512-token prompts stay unchunked and prompts above 1K use `1024/1024/4096/1024/1024` chunks. The `hipEngine GGUF Q4_K_S` column uses the same chunked-prefill policy plus the WMMA prefill + GEMV decode fast paths and the persistent on-load decode-repack into T16 tile layouts.
+
+### Prefill tok/s
+
+| Workload | hipEngine PARO | hipEngine GGUF Q4_K_S | llama.cpp HIP | llama.cpp Vulkan |
+| --- | ---: | ---: | ---: | ---: |
+| 512/128 | **2718.497** | 2258.847 | 2436.049 | 1816.927 |
+| 4K/128 | **2838.773** | 2576.673 | 2176.905 | 1705.093 |
+| 32K/128 | **2074.699** | 1893.967 | 1496.409 | 1128.554 |
+| 128K/128 | **1055.454** | 998.143 | 710.213 | 480.539 |
 
 ### Decode tok/s
 
-| Workload | hipEngine shisa Qwen3.6 packed PARO | llama.cpp HIP | llama.cpp Vulkan |
-| --- | ---: | ---: | ---: |
-| 512/128 | 111.516 | 85.487 | **127.515** |
-| 4K/128 | 113.094 | 87.375 | **120.163** |
-| 32K/128 | 97.594 | 76.994 | **98.073** |
-| 128K/128 | 62.027 | 57.341 | **64.478** |
+| Workload | hipEngine PARO | hipEngine GGUF Q4_K_S | llama.cpp HIP | llama.cpp Vulkan |
+| --- | ---: | ---: | ---: | ---: |
+| 512/128 | 103.460 | 109.152 | 85.487 | **127.515** |
+| 4K/128 | 101.964 | 100.048 | 87.375 | **120.163** |
+| 32K/128 | 90.438 | 86.774 | 76.994 | **98.073** |
+| 128K/128 | 59.598 | 57.954 | 57.341 | **64.478** |
 
 ### Peak GiB
 
-| Workload | hipEngine shisa Qwen3.6 packed PARO | llama.cpp HIP | llama.cpp Vulkan |
-| --- | ---: | ---: | ---: |
-| 512/128 | **18.123** | 21.125 | 20.844 |
-| 4K/128 | **19.455** | 21.197 | 20.969 |
-| 32K/128 | **20.267** | 21.738 | 21.533 |
-| 128K/128 | **23.235** | 23.605 | 23.596 |
+| Workload | hipEngine PARO | hipEngine GGUF Q4_K_S | llama.cpp HIP | llama.cpp Vulkan |
+| --- | ---: | ---: | ---: | ---: |
+| 512/128 | 20.962 | 25.108 | 21.125 | **20.844** |
+| 4K/128 | 21.906 | 25.108 | 21.197 | **20.969** |
+| 32K/128 | 22.016 | 25.108 | 21.738 | **21.533** |
+| 128K/128 | **22.122** | 25.108 | 23.605 | 23.596 |
+
+hipEngine W7900 row source: [`benchmarks/results/2026-05-25-w7900-hipengine-readme-persistent-5run-diagnostic.json`](benchmarks/results/2026-05-25-w7900-hipengine-readme-persistent-5run-diagnostic.json). Both hipEngine columns are 5-run medians from one resident session allocated for the maximum requested context (`128K/128`), so the peak-memory column is a max-context persistent-session high-water mark rather than each shape's minimum allocation. Existing W7900 llama.cpp HIP/Vulkan Q4_K_M rows are reused unchanged. The hipEngine GGUF Q4_K_S column is compared against the existing llama.cpp Q4_K_M baselines because that is the lineage of measured baselines we have on this host; cross-quant comparisons should be read as approximate.
 
-## gfx1151 (AMD Ryzen AI MAX+ 395 / Radeon 8060S)
+### gfx1151 (AMD Ryzen AI MAX+ 395 / Radeon 8060S)
 
 The gfx1151 backend is a native `--offload-arch=gfx1151` peer backend using the same registry-keyed kernel surface. The Strix Halo snapshot below uses 256-row prefill chunks, which removed the 4K prefill gap without hurting long-context decode.
 
 ### Prefill tok/s
 
-| Workload | hipEngine shisa Qwen3.6 packed PARO | llama.cpp HIP | llama.cpp Vulkan |
+| Workload | hipEngine PARO | llama.cpp HIP | llama.cpp Vulkan |
 | --- | ---: | ---: | ---: |
 | 512/128 | 983.206 | **1058.738** | 638.008 |
 | 4K/128 | **1029.402** | 1004.220 | 595.400 |
@@ -129,7 +220,7 @@ The gfx1151 backend is a native `--offload-arch=gfx1151` peer backend using the
 
 ### Decode tok/s
 
-| Workload | hipEngine shisa Qwen3.6 packed PARO | llama.cpp HIP | llama.cpp Vulkan |
+| Workload | hipEngine PARO | llama.cpp HIP | llama.cpp Vulkan |
 | --- | ---: | ---: | ---: |
 | 512/128 | **62.060** | 50.537 | 57.615 |
 | 4K/128 | **63.605** | 49.379 | 55.027 |
@@ -141,25 +232,26 @@ On Strix Halo, `rocm-smi` / sysfs expose only a 512 MiB VRAM aperture, so cross-
 See [`benchmarks/README.md`](benchmarks/README.md) for full protocol details,
 correctness status, source-lineage targets, and external comparison baselines.
 
-## Hardware targets
+## GGUF Support
 
-| Backend | Hardware | Status |
-| --- | --- | --- |
-| `hip_gfx1100` | AMD Radeon Pro W7900 / RX 7900 XTX (RDNA3) | Primary, in active bring-up |
-| `hip_gfx1151` | AMD Ryzen AI MAX+ 395 / Radeon 8060S (Strix Halo, RDNA3.5) | Active backend |
-| `cuda_sm86` | NVIDIA Ampere consumer (3090-class) | Planned peer backend |
-| `cpu_reference` | Any CPU, numpy | Correctness oracle; CI without GPU |
+As of v0.2.0, hipEngine includes resident Qwen3.6 GGUF support for `Q4_K_M` and
+`Q4_K_S` model files (with more formats planned). This is a major runtime path,
+not just a loader shim: GGUF has its own quant readers, bulk-prefill path,
+decode-repacked T16 layouts, and fast-path controls.
 
-`backend="auto"` is the public API/server default. It maps exact `gfx1100` and
-`gfx1151` detections to the matching HIP backend; unknown ROCm targets warn and
-select `cpu_reference` where a CPU implementation exists. Users on nearby targets
-such as `gfx1101`/`gfx1102` can force a backend with `backend="hip_gfx1100"`,
-`--backend hip_gfx1100`, or `HIPENGINE_BACKEND=hip_gfx1100` after validating
-correctness/performance.
+Current caveats:
+
+- PARO models take ~24s to load on the W7900 test host; GGUF currently takes
+  about 60s because decode-repack happens on load. On-disk caching could reduce
+  startup time later, but would require additional storage for repacked layouts.
+- GGUF has higher resident memory than packed PARO. In the current W7900 README
+  sweep, the max-context Q4_K_S session peaks at ~25.1 GiB tracked, so 128K is
+  W7900/48 GiB territory; on 24 GiB cards, expect roughly 64K context with
+  Q4_K_S.
+- GGUF is close enough to PARO to share some high-level scheduling ideas, but in
+  practice it needs substantial GGUF-only kernels and dispatch. The goal for
+  future releases is to keep closing the remaining PARO/GGUF speed gap.
 
-Wave32 is the default for `hip_gfx1100` device code; wave64 is treated as an
-isolated experiment with its own gates (see
-[`docs/PLAN.md`](docs/PLAN.md#rdna3-wavefront-and-scheduling-caveat)).
 
 ## Architecture at a glance
 
@@ -185,7 +277,7 @@ isolated experiment with its own gates (see
 │  KERNELS (backend-keyed, 120 __global__ in the Qwen/PARO port)  │
 │  kernels/hip_gfx1100/  attention / linear_attn / moe / quant    │
 │                        wmma / norm / rotary / fused             │
-│  kernels/hip_gfx1151/  native target-arch peer backend           │
+│  kernels/hip_gfx1151/  native target-arch peer backend          │
 │  kernels/cuda_sm86/    (future)                                 │
 │  kernels/cpu_reference/ correctness oracle, no GPU required     │
 └─────────────────────────────────────────────────────────────────┘
@@ -262,6 +354,7 @@ current limitations.
 | [`docs/BENCHMARK.md`](docs/BENCHMARK.md) | Benchmark protocols, baselines, correctness gate, artifact format |
 | [`docs/TESTING.md`](docs/TESTING.md) | RED/GREEN workflow, correctness oracles, fixture policy |
 | [`docs/KERNELS.md`](docs/KERNELS.md) | Kernel catalog, source-lineage drift workflow, JIT cache gotchas, build profiles |
+| [`docs/ENVS.md`](docs/ENVS.md) | Environment variables, TheRock setup, benchmark/profiling profiles |
 | [`docs/ROOFLINE.md`](docs/ROOFLINE.md) | RDNA3 / W7900 performance model and decision tree |
 | [`docs/IMPLEMENTATION.md`](docs/IMPLEMENTATION.md) | Implementation status and concrete milestones |
 | [`docs/API.md`](docs/API.md) | OpenAI-compatible server usage and endpoint support |