mxflashattn 1.4.0__tar.gz → 1.5.0__tar.gz

{mxflashattn-1.4.0 → mxflashattn-1.5.0}/MANIFEST.in

@@ -1,4 +1,5 @@
 include README.md
+include README_PYPI.md
 include LICENSE
 include CONTRIBUTING.md
 include requirements-c500.txt

mxflashattn-1.5.0/PKG-INFO

@@ -0,0 +1,188 @@
+Metadata-Version: 2.4
+Name: mxflashattn
+Version: 1.5.0
+Summary: Observable FlashAttention-compatible decode integration for MetaX C500 and MXMACA
+License-Expression: Apache-2.0
+Classifier: Development Status :: 3 - Alpha
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.4
+Requires-Dist: PyYAML<7,>=6.0
+Provides-Extra: dev
+Requires-Dist: pytest<10,>=8; extra == "dev"
+Requires-Dist: ruff<1,>=0.6; extra == "dev"
+Provides-Extra: vllm
+Requires-Dist: vllm==0.17.0; extra == "vllm"
+Provides-Extra: reports
+Requires-Dist: matplotlib<4,>=3.8; extra == "reports"
+Dynamic: license-file
+
+# MXFlashAttn
+
+**MXFlashAttn 1.5.0** provides a FlashAttention-compatible Python API and an
+observable paged-decode integration for **MetaX C500 / MXMACA**.
+
+The package combines:
+
+- validated dense, varlen, and KV-cache forward entry points;
+- explicit validation, dispatch information, and opt-in reference fallback;
+- an M1 Split-KV decode path for C500 paged KV cache;
+- an exact-profile MMA Split-KV path for calibrated FP16 decode shapes;
+- vLLM EngineCore registration and per-dispatch evidence.
+
+Native C500 extensions are compiled inside the target MetaX/MACA environment.
+The PyPI artifact supplies the Python API, integration layer, profile handling,
+and registration entry point.
+
+## Install
+
+```bash
+python -m pip install mxflashattn==1.5.0
+```
+
+Base requirements:
+
+- Python 3.10+
+- PyTorch 2.4+
+- PyYAML 6.x
+
+For the vLLM 0.17 dependency contract:
+
+```bash
+python -m pip install "mxflashattn[vllm]==1.5.0"
+```
+
+MetaX deployments normally provide their own compatible PyTorch, vLLM, and
+MXMACA wheels. vLLM 0.17.0 and 0.28.0 were validated on separate C500 software
+stacks; they are not interchangeable environments.
+
+## Python API
+
+```python
+import os
+import torch
+
+from mxflashattn import flash_attn_func, get_last_dispatch_info
+
+# Explicitly enable the correctness/reference path when no native provider exists.
+os.environ["MXFLASHATTN_ALLOW_FALLBACK"] = "1"
+
+q = torch.randn(1, 32, 8, 64, dtype=torch.float16)
+k = torch.randn(1, 128, 2, 64, dtype=q.dtype)
+v = torch.randn_like(k)
+
+out = flash_attn_func(q, k, v, causal=True)
+print(out.shape)
+print(get_last_dispatch_info())
+```
+
+The reference path is intended for API checks and correctness comparison. It
+is not a C500 performance backend. Without
+`MXFLASHATTN_ALLOW_FALLBACK=1`, a missing native provider raises an explicit
+error instead of silently selecting a slow path.
+
+Public entry points:
+
+```python
+from mxflashattn import (
+    flash_attn_func,
+    flash_attn_varlen_func,
+    flash_attn_with_kvcache,
+    get_last_dispatch_info,
+)
+```
+
+## C500 Native Setup
+
+Build native extensions in an environment matching the target MetaX PyTorch
+and MACA installation:
+
+```bash
+git clone https://github.com/leo1852098393-blip/MXFlashAttn.git
+cd MXFlashAttn
+python -m pip install --no-build-isolation -e .
+
+export MACA_PATH=/opt/maca
+bash csrc_native/build_m1.sh
+```
+
+Then enable the guarded M1 route and collect dispatch evidence:
+
+```bash
+export MXFA_M1_DECODE=1
+export MXFLASHATTN_DISPATCH_SAMPLE=1
+
+python integrations/vllm/smoke_generate.py \
+  --model /path/to/Qwen3-4B \
+  --suite --prompt-count 4 --max-tokens 16 \
+  --candidate --output artifacts/m1-smoke.json
+```
+
+A performance result is attributed to MXFlashAttn only when the associated
+`dispatch.jsonl` contains a successful `m1_split_decode` or
+`mma_split_decode` event.
+
+## Verified Scope
+
+| Area | v1.5.0 status |
+| --- | --- |
+| M1 decode | FP16 / BF16, head dimensions 64 / 128, GQA ratios 2 / 4 / 8 |
+| vLLM 0.17.0 | Real Qwen3-4B/8B generation, M1 dispatch, fallback, and long-decode measurements |
+| vLLM 0.28.0 | EngineCore plugin registration, real M1 dispatch, and a Qwen3-8B shape matrix |
+| MMA decode | FP16 single-token decode, exact calibrated profiles only |
+| Prefill | MetaX vendor production path; chunked prefill remains a correctness reference |
+| Backward | Explicit MetaX provider integration; no in-house backward kernel |
+
+In four vLLM 0.17.0 long-decode workloads, M1 delivered 2.67-3.54x the
+throughput of the project's historical v7 path and reached 79.2%-86.5% of
+MetaX vendor throughput. In one exact Qwen3-4B operator shape, MMA median
+latency was `43.520 us` versus `46.592 us` for vendor over 100 rounds.
+
+In the reported
+[vLLM 0.28.0 Qwen3-8B matrix](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/V1_5_028_MATRIX.md),
+long-KV samples at batch 16 or larger remained 28%-46% below MetaX vendor. No
+general M1-over-vendor crossover is claimed.
+
+These results are bound to their recorded device, software stack, model,
+shape, and baseline. The MMA comparison is operator-level, not an end-to-end
+request-speedup claim.
+
+## Runtime Boundaries
+
+- Prefill and unprofiled decode shapes retain the MetaX vendor path.
+- M1 requires its documented dtype, layout, head-dimension, and GQA contract.
+- MMA requires an exact profile match, including KV heads, page size, binary
+  variant, and shared-object identity.
+- Short or low-concurrency requests may not benefit from M1.
+- FP8 KV cache, multi-card execution, SGLang, GQ=7, and general TTFT gains are
+  not v1.5.0 support claims.
+- Unsupported API options fail explicitly or follow a documented fallback.
+
+## Development and Audit
+
+The project's own kernel implementations under `csrc_native/` were written by
+AI coding agents. The project owner set the project goals, verification rules,
+and release boundary, and made the final accept-or-reject decisions for code
+changes and public claims.
+
+Favorable measurements do not automatically become performance claims. The
+short-KV `+15.2%` difference was the same order as repeated-run variance; the
+long-KV `+22.7%` result was a single observation and does not establish a
+general crossover. Neither is used for a general performance claim.
+
+## Documentation
+
+- [GitHub repository](https://github.com/leo1852098393-blip/MXFlashAttn)
+- [GitLink mirror](https://gitlink.org.cn/Leo77/MXFlashAttn)
+- [API guide](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/API.md)
+- [Support matrix](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/support-matrix.md)
+- [Evidence index](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/EVIDENCE.md)
+- [Release validation](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/V1_5_RELEASE_VALIDATION.md)
+
+## License
+
+Apache License 2.0. MXMACA runtime, MetaX vendor packages, vLLM, model
+weights, and datasets remain subject to their respective licenses.

mxflashattn-1.5.0/README.md

@@ -0,0 +1,321 @@
+# MXFlashAttn
+
+> 面向 **MetaX C500 / MXMACA** 的 FlashAttention 兼容层与可观测
+> paged-decode 优化工程。
+
+中文说明 · [English](README.en.md)
+
+[![CI](https://github.com/leo1852098393-blip/MXFlashAttn/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/leo1852098393-blip/MXFlashAttn/actions/workflows/ci.yml)
+[![License](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
+[![Python](https://img.shields.io/badge/python-3.10%2B-3776AB.svg)](pyproject.toml)
+[![PyPI](https://img.shields.io/pypi/v/mxflashattn.svg)](https://pypi.org/project/mxflashattn/)
+
+**版本**：v1.5.0（2026-10-24） · **设备**：MetaX C500 64 GiB ·
+**已验证引擎**：vLLM 0.17.0 / 0.28.0 · **许可证**：Apache-2.0
+
+MXFlashAttn 把自研 Attention 优化接入真实 C500 推理链路：读取 vLLM paged
+KV cache，在 EngineCore worker 中完成后端注册，按输入 shape 选择 M1 或 MMA
+Split-KV decode，并为每次候选派发记录可审计证据。不满足条件的请求保留
+MetaX vendor 路径。
+
+## 已验证结果
+
+| 结论 | 实测范围 | 证据 |
+| --- | --- | --- |
+| M1 达到项目 v7 路径的 **2.67-3.54x**，达到 MetaX vendor 吞吐的 **79.2%-86.5%** | vLLM 0.17.0，Qwen3-4B/8B，4 组长 decode workload | [复测报告](docs/V1_5_017_RECHECK.md) |
+| MMA 中位延迟 `43.520 us`，vendor 为 `46.592 us`，该精确 shape 低约 **6.6%** | FP16，batch=1，KV=128，32 Q heads / 8 KV heads，GQA=4，head_dim=128，split=4 | [发布复核](docs/V1_5_RELEASE_VALIDATION.md) |
+| 自研路径已进入真实 EngineCore worker，并可区分候选命中、回退与未标定输入 | vLLM 0.17.0 / 0.28.0 的 C500 实例 | [证据索引](docs/EVIDENCE.md) |
+
+这里的 v7 是项目自有历史内核，不是 MetaX vendor。MMA 的 6.6% 是一个精确
+算子 shape 的结果，不是端到端请求加速结论。
+
+| 你想了解什么 | 从这里开始 |
+| --- | --- |
+| 在 C500 上安装和运行 | [快速开始](#快速开始) |
+| 判断项目的技术工作量 | [技术难点与工程演进](#技术难点与工程演进) |
+| 核对性能数字 | [实测结果](#实测结果) · [证据索引](docs/EVIDENCE.md) |
+| 查看支持和回退边界 | [支持范围与性能边界](#支持范围与性能边界) |
+
+## 为什么这项工程重要
+
+在国产 AI GPU 上，Attention 优化不是把一个独立 kernel 编译成功就结束了。
+要让它进入真实推理系统，还需要同时解决：
+
+1. vLLM paged KV cache 的布局与生命周期；
+2. 主进程和 EngineCore worker 之间的后端注册；
+3. dtype、GQA、head dimension、KV heads、page size 等输入守卫；
+4. Prefill、未标定 shape 和原生调用失败时的可靠回退；
+5. “注册成功”和“自研 kernel 确实执行”之间的证据差异；
+6. 算子延迟、端到端吞吐和 TTFT 等不同指标的归因边界。
+
+MXFlashAttn 围绕这条完整链路建设。项目的目标不是只展示一次可运行的算子，
+而是形成一套**可运行、可测量、可审计、可继续优化**的 C500 Attention
+工程基础。
+
+## 核心贡献
+
+| 能力 | 项目实现 |
+| --- | --- |
+| 自研 paged decode | M1 Split-KV kernel 直接读取 vLLM paged KV cache，覆盖 FP16 / BF16、head dimension 64 / 128、GQA 2 / 4 / 8 |
+| 张量核路径 | 自研 MMA Split-KV decode，通过设备探针恢复 fragment 映射，只对精确标定的 FP16 shape 启用 |
+| 真实引擎接线 | 通过 vLLM general plugin 进入 EngineCore worker；已在 vLLM 0.17.0 与 0.28.0 记录真实 dispatch |
+| 可观测运行时 | 每次候选派发记录 backend、operation、selection、fallback、split 等字段 |
+| 守卫与回退 | dtype、shape、layout、profile 或原生调用不满足条件时回到 MetaX vendor，不让实验路径处理未知输入 |
+| 可复现实验 | 基准脚本、脱敏数据、图表生成器、测量方法和负向结果随仓库公开 |
+
+## 技术难点与工程演进
+
+### 1. 从设备探针恢复 MMA fragment 契约
+
+项目不能依赖完整公开的 `__builtin_mxc_mma_16x16x16f16` fragment 布局，
+因此编写逐线程写入、读回和 NumPy 对照探针，确认 C500 上观察到的执行组为
+64 线程，并恢复 accumulator 的线程到矩阵元素映射。映射写错时可能得到有限
+但错误的数值，而不是直接崩溃，因此这一步同时是正确性门槛。
+
+### 2. 定位 Split-KV 的非确定性越界写
+
+早期 MMA Split-KV 结果会在相同输入下破坏不同 scratch slot。逐 slot 扫描
+发现损坏状态具有 `m=0, l=256` 指纹，最终定位到发布条件把 `lane < GQ`
+误写为 `lane < 16`：无效 query lane 会写穿到下一个 slot，并与合法 CTA
+发生竞态。修复后：
+
+| 校验 | 修复前 | 修复后 |
+| --- | ---: | ---: |
+| 正确的 per-split `(m, l)` slot | 4 / 8 | 8 / 8 |
+| 合并输出相对 FP32 参考的最大误差 | `2.99e-2` | `6.81e-5` |
+
+### 3. 把后端注册送进真正执行它的进程
+
+vLLM 会在独立解释器中创建 EngineCore。只在启动脚本注册后端，会出现父进程
+显示 `registered=true`、worker 仍走 vendor 的假阳性。v1.5.0 通过
+`vllm.general_plugins` entry point 在各进程重复注册，并在报告成功前检查
+最终解析到的实现类。
+
+### 4. 从历史 v7 到精确 profile 路由
+
+项目先以 v7 建立历史基线，再完成 M1 Split-KV paged decode、真实 worker
+接线和 dispatch 记录，最后加入 MMA 张量核与精确 shape profile。当前 MMA
+不会向邻近 shape 外推；batch、KV length、GQA、head dimension、KV heads、
+page size、二进制变体或共享库身份任一不匹配，都会回到 vendor。
+
+完整技术说明见 [技术亮点](docs/TECHNICAL_HIGHLIGHTS.md)。
+
+## 实测结果
+
+以下数据均来自 MetaX C500 64 GiB。不同 MACA、PyTorch 和 vLLM 软件栈分别
+报告，不做跨版本混算。
+
+### 真实 vLLM 0.17 长 decode
+
+![vLLM 0.17 C500 end-to-end throughput comparison](docs/results/readme-vllm017-e2e-2026-10-23.png)
+
+测试环境为 MACA 3.5.3.20、PyTorch `2.8.0+metax3.5.3.9`、vLLM 0.17.0；
+Qwen3-4B / 8B 每组生成 1024 token，关闭 prefix caching。四组 workload
+均记录到 `m1_split_decode`：
+
+- M1 吞吐为项目 v7 路径的 **2.67-3.54 倍**；
+- M1 达到 MetaX vendor 吞吐的 **79.2%-86.5%**；
+- M1 仍比 vendor 低 **13.5%-20.8%**。
+
+这组结果说明 M1 明显改善了项目自研 decode 路径，同时也量化了与 vendor
+之间仍需优化的空间。
+
+数据与方法：[vLLM 0.17 复测报告](docs/V1_5_017_RECHECK.md) ·
+[图表原始数据](docs/results/readme-data-2026-10-24.json)
+
+### C500 MMA 精确 shape
+
+![C500 MMA operator latency for one exact shape](docs/results/readme-mma-operator-2026-10-24.png)
+
+在 Qwen3-4B 的实际配置上，FP16、batch=1、KV=128、32 Q heads、
+8 KV heads、GQA=4、head_dim=128、page_size=16、split=4，100 轮算子
+计时中：
+
+- MetaX vendor 中位延迟：`46.592 us`；
+- MXFlashAttn MMA Split-KV 中位延迟：`43.520 us`；
+- 该精确 shape 下 MMA 中位延迟低约 **6.6%**。
+
+同一 profile 已在真实 vLLM 请求中记录 36 次 `mma_split_decode`，整次请求
+共有 396 个 attention 事件。这个结果证明了“算子标定 -> 精确 profile ->
+真实 worker 派发”的链路，不代表整个请求获得 6.6% 的端到端提升。
+
+数据与方法：[MMA Split-KV 技术记录](docs/V1_5_MMA_KV_SPLIT.md) ·
+[发布复核](docs/V1_5_RELEASE_VALIDATION.md) ·
+[公开 profile](docs/results/c500-mma-profile-qwen3-4b-hkv8-2026-10-24.json)
+
+## 运行路径
+
+![MXFlashAttn vLLM C500 dispatch and fallback path](docs/results/readme-runtime-routing.png)
+
+1. vLLM worker 将 Attention 请求交给 MXFlashAttn adapter。
+2. Adapter 检查 phase、dtype、layout、shape 和 profile。
+3. 精确 profile 命中时可选择 MMA；满足 M1 合约且显式启用时可选择 M1。
+4. Prefill、未标定 shape、不支持输入或原生执行失败时回到 MetaX vendor。
+5. 候选路径写入 dispatch 记录，性能数据只有在目标 kernel 确实派发后才归因。
+
+## 快速开始
+
+### Python API
+
+```bash
+python -m pip install mxflashattn==1.5.0
+```
+
+```python
+import os
+import torch
+
+from mxflashattn import flash_attn_func, get_last_dispatch_info
+
+# 无原生 provider 时，reference 路径必须显式开启。
+os.environ["MXFLASHATTN_ALLOW_FALLBACK"] = "1"
+
+q = torch.randn(1, 32, 8, 64, dtype=torch.float16)
+k = torch.randn(1, 128, 2, 64, dtype=q.dtype)
+v = torch.randn_like(k)
+
+out = flash_attn_func(q, k, v, causal=True)
+print(out.shape)
+print(get_last_dispatch_info())
+```
+
+Reference fallback 用于接口调试和正确性对照，不是 C500 性能路径。
+
+### C500 / vLLM
+
+原生扩展需要在与目标 MetaX PyTorch / MACA 匹配的 C500 环境中编译：
+
+```bash
+git clone https://github.com/leo1852098393-blip/MXFlashAttn.git
+cd MXFlashAttn
+python -m pip install --no-build-isolation -e .
+
+export MACA_PATH=/opt/maca
+bash csrc_native/build_m1.sh
+
+export MXFA_M1_DECODE=1
+export MXFLASHATTN_DISPATCH_SAMPLE=1
+python integrations/vllm/smoke_generate.py \
+  --model /path/to/Qwen3-4B \
+  --suite --prompt-count 4 --max-tokens 16 \
+  --candidate --output artifacts/m1-smoke.json
+```
+
+验收时同时检查结果 JSON 和同目录的 `dispatch.jsonl`。只有出现成功的
+`m1_split_decode` 或 `mma_split_decode` 事件，相关吞吐或延迟才能归因于
+MXFlashAttn 自研路径。MMA 的构建、profile 和精确匹配方式见
+[API 文档](docs/API.md)。
+
+## 支持范围与性能边界
+
+| 项目 | 当前能力 |
+| --- | --- |
+| Python API | `flash_attn_func`、`flash_attn_varlen_func`、`flash_attn_with_kvcache` |
+| M1 decode dtype | FP16 / BF16 |
+| M1 shape | head dimension 64 / 128；GQA 2 / 4 / 8 |
+| vLLM | 0.17.0 与 0.28.0 均完成 C500 worker 接线和 dispatch 验证 |
+| MMA decode | FP16 单 token decode；只对精确标定 shape 启用 |
+| Prefill | 生产路径使用 MetaX vendor；chunked prefill 是正确性研究原型 |
+| Backward | 可显式调用 MetaX backward provider；不是项目自研反向 kernel |
+| 不支持 shape | 不进入自研 kernel，回到 MetaX vendor 或明确报错 |
+
+更完整的 dtype、shape、软件栈和 fallback 说明见
+[支持矩阵](docs/support-matrix.md)与 [API 文档](docs/API.md)。
+
+### vLLM 0.28 形状矩阵
+
+![vLLM 0.28 C500 throughput delta versus MetaX vendor](docs/results/readme-vllm028-delta-2026-10-23.png)
+
+这组结果来自独立软件栈：MACA 3.8.2.6、PyTorch 2.10.0、vLLM 0.28.0、
+Qwen3-8B。它揭示了当前 M1 的适用边界：
+
+- 短 KV 序列中有 3 个 batch 点高于 vendor（最高 `+15.2%`），但这些差异
+  仍处于重复测量方差带内，不作为性能结论；
+- 中 KV、大 batch 样本低于 vendor 约 15%-19%；
+- 长 KV、batch >= 16 的样本低于 vendor 约 28%-46%；
+- batch=4 长 KV 的 `+22.7%` 是单个观测点，不作为通用 crossover。
+
+进一步的 NP 扫描把 batch=16、序列容量 1024 的一个 workload 收敛到
+vendor 的 `-2.2%`，但 batch=128 下所有已测 NP 仍落后 17.6%-24.2%。
+这说明 shape 标定可以减少调度损失，但大 batch 扩展性仍需要内核级访存、
+softmax / reduction 和 split / combine 优化。
+
+完整矩阵与测量伪影修复记录见
+[vLLM 0.28 测量报告](docs/V1_5_028_MATRIX.md)。
+
+## 项目结构
+
+```text
+mxflashattn/       Python API、参数校验、reference、dispatch 与 profile
+csrc_native/       M1、v7、MMA device kernel 及构建、正确性和基准工具
+csrc/mxmac/        ATen bring-up / 正确性对照路径，不是融合优化 kernel
+integrations/vllm/ vLLM backend、worker 注册、layout adapter 与 dispatch 记录
+benchmarks/        固定 workload 的基准与报告工具
+scripts/           profile、结果汇总、发布审计和 README 图表生成
+tests/             CPU/reference 回归与按环境启用的 C500 测试
+docs/results/      脱敏结果、图表及图表数据
+docs/              支持边界、实验方法和技术记录
+```
+
+## 可复现与审计
+
+`csrc_native/` 中的项目自有内核实现由 AI 编程代理完成。项目负责人确定项目
+目标、验证规则与发布边界，并对代码变更和公开结论作出最终的采纳或否决。
+
+- **归因规则**：只有 dispatch 日志出现成功的 `m1_split_decode` 或
+  `mma_split_decode` 事件，性能才计入本项目；注册成功本身不算证据。
+- **结论门槛**：对项目有利的测量也不会自动成为性能结论。短 KV 的 `+15.2%`
+  与重复测量方差同量级；长 KV 的 `+22.7%` 只是单个观测点，不能据此建立
+  通用 crossover。两者均未用于建立通用性能主张。
+- **复现优先**：缺陷须先产出稳定复现用例，修复后该用例转绿且优化路径仍在
+  派发，才算修复完成。
+- README 图表由
+  [`scripts/plot_readme_results.py`](scripts/plot_readme_results.py)
+  从
+  [`docs/results/readme-data-2026-10-24.json`](docs/results/readme-data-2026-10-24.json)
+  生成。
+- [证据索引](docs/EVIDENCE.md)把每项首页结论映射到设备、软件栈、模型、
+  workload、基线、公开数据和限制。
+- vLLM 0.17 与 0.28 使用不同软件栈，结果单独解释。
+- 原始模型权重、租用实例日志和凭据不进入公开仓库；公开摘要保留复现配置与
+  可审计结论。
+
+## 发展路线
+
+### 1. M1 内核与调度联合优化
+
+- 建立 `(batch, KV length, GQA, head_dim)` 的可复现 profile；
+- 优化 KV staging、softmax / reduction 和 split / combine 开销；
+- 以相同输入、独立进程、dispatch 命中、重复测量和 vendor 对照作为合入门槛。
+
+### 2. 扩展 MMA 的有效覆盖
+
+- 补充 BF16、长 KV、更多真实模型 shape 和误差压力测试；
+- 解决 CUDA Graph 下 scratch 生命周期与 capture 约束；
+- 只有在真实请求中取得稳定端到端收益后，才扩大自动 dispatch 范围。
+
+### 3. Prefill 与服务指标
+
+- 研究 C500 专用融合 Prefill kernel；
+- 建立可归因的 TTFT P50 / P95 / P99、吞吐与显存联合测量；
+- 将 Prefill、Decode 和调度开销分开报告。
+
+### 4. 生态扩展
+
+- 持续验证新的 vLLM / MACA 组合；
+- 在同一证据门槛下评估 FP8 KV cache、多卡和 SGLang；
+- 将 profile、测试协议和 dispatch 证据格式沉淀为 C500 Attention
+  优化基础设施。
+
+研究依据与实验映射见
+[研究笔记](docs/V1_5_RESEARCH_20261024.md)。
+
+## 参与贡献
+
+Issue 与 Pull Request 请附带最小复现、设备与软件栈版本、输入 shape，以及
+dispatch 记录。开发与测试约定见 [CONTRIBUTING.md](CONTRIBUTING.md)。
+
+## 许可证
+
+本项目采用 [Apache License 2.0](LICENSE)。MXMACA runtime、MetaX vendor
+wheel、vLLM、模型权重和数据集分别遵循各自许可证。

mxflashattn-1.5.0/README_PYPI.md

@@ -0,0 +1,166 @@
+# MXFlashAttn
+
+**MXFlashAttn 1.5.0** provides a FlashAttention-compatible Python API and an
+observable paged-decode integration for **MetaX C500 / MXMACA**.
+
+The package combines:
+
+- validated dense, varlen, and KV-cache forward entry points;
+- explicit validation, dispatch information, and opt-in reference fallback;
+- an M1 Split-KV decode path for C500 paged KV cache;
+- an exact-profile MMA Split-KV path for calibrated FP16 decode shapes;
+- vLLM EngineCore registration and per-dispatch evidence.
+
+Native C500 extensions are compiled inside the target MetaX/MACA environment.
+The PyPI artifact supplies the Python API, integration layer, profile handling,
+and registration entry point.
+
+## Install
+
+```bash
+python -m pip install mxflashattn==1.5.0
+```
+
+Base requirements:
+
+- Python 3.10+
+- PyTorch 2.4+
+- PyYAML 6.x
+
+For the vLLM 0.17 dependency contract:
+
+```bash
+python -m pip install "mxflashattn[vllm]==1.5.0"
+```
+
+MetaX deployments normally provide their own compatible PyTorch, vLLM, and
+MXMACA wheels. vLLM 0.17.0 and 0.28.0 were validated on separate C500 software
+stacks; they are not interchangeable environments.
+
+## Python API
+
+```python
+import os
+import torch
+
+from mxflashattn import flash_attn_func, get_last_dispatch_info
+
+# Explicitly enable the correctness/reference path when no native provider exists.
+os.environ["MXFLASHATTN_ALLOW_FALLBACK"] = "1"
+
+q = torch.randn(1, 32, 8, 64, dtype=torch.float16)
+k = torch.randn(1, 128, 2, 64, dtype=q.dtype)
+v = torch.randn_like(k)
+
+out = flash_attn_func(q, k, v, causal=True)
+print(out.shape)
+print(get_last_dispatch_info())
+```
+
+The reference path is intended for API checks and correctness comparison. It
+is not a C500 performance backend. Without
+`MXFLASHATTN_ALLOW_FALLBACK=1`, a missing native provider raises an explicit
+error instead of silently selecting a slow path.
+
+Public entry points:
+
+```python
+from mxflashattn import (
+    flash_attn_func,
+    flash_attn_varlen_func,
+    flash_attn_with_kvcache,
+    get_last_dispatch_info,
+)
+```
+
+## C500 Native Setup
+
+Build native extensions in an environment matching the target MetaX PyTorch
+and MACA installation:
+
+```bash
+git clone https://github.com/leo1852098393-blip/MXFlashAttn.git
+cd MXFlashAttn
+python -m pip install --no-build-isolation -e .
+
+export MACA_PATH=/opt/maca
+bash csrc_native/build_m1.sh
+```
+
+Then enable the guarded M1 route and collect dispatch evidence:
+
+```bash
+export MXFA_M1_DECODE=1
+export MXFLASHATTN_DISPATCH_SAMPLE=1
+
+python integrations/vllm/smoke_generate.py \
+  --model /path/to/Qwen3-4B \
+  --suite --prompt-count 4 --max-tokens 16 \
+  --candidate --output artifacts/m1-smoke.json
+```
+
+A performance result is attributed to MXFlashAttn only when the associated
+`dispatch.jsonl` contains a successful `m1_split_decode` or
+`mma_split_decode` event.
+
+## Verified Scope
+
+| Area | v1.5.0 status |
+| --- | --- |
+| M1 decode | FP16 / BF16, head dimensions 64 / 128, GQA ratios 2 / 4 / 8 |
+| vLLM 0.17.0 | Real Qwen3-4B/8B generation, M1 dispatch, fallback, and long-decode measurements |
+| vLLM 0.28.0 | EngineCore plugin registration, real M1 dispatch, and a Qwen3-8B shape matrix |
+| MMA decode | FP16 single-token decode, exact calibrated profiles only |
+| Prefill | MetaX vendor production path; chunked prefill remains a correctness reference |
+| Backward | Explicit MetaX provider integration; no in-house backward kernel |
+
+In four vLLM 0.17.0 long-decode workloads, M1 delivered 2.67-3.54x the
+throughput of the project's historical v7 path and reached 79.2%-86.5% of
+MetaX vendor throughput. In one exact Qwen3-4B operator shape, MMA median
+latency was `43.520 us` versus `46.592 us` for vendor over 100 rounds.
+
+In the reported
+[vLLM 0.28.0 Qwen3-8B matrix](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/V1_5_028_MATRIX.md),
+long-KV samples at batch 16 or larger remained 28%-46% below MetaX vendor. No
+general M1-over-vendor crossover is claimed.
+
+These results are bound to their recorded device, software stack, model,
+shape, and baseline. The MMA comparison is operator-level, not an end-to-end
+request-speedup claim.
+
+## Runtime Boundaries
+
+- Prefill and unprofiled decode shapes retain the MetaX vendor path.
+- M1 requires its documented dtype, layout, head-dimension, and GQA contract.
+- MMA requires an exact profile match, including KV heads, page size, binary
+  variant, and shared-object identity.
+- Short or low-concurrency requests may not benefit from M1.
+- FP8 KV cache, multi-card execution, SGLang, GQ=7, and general TTFT gains are
+  not v1.5.0 support claims.
+- Unsupported API options fail explicitly or follow a documented fallback.
+
+## Development and Audit
+
+The project's own kernel implementations under `csrc_native/` were written by
+AI coding agents. The project owner set the project goals, verification rules,
+and release boundary, and made the final accept-or-reject decisions for code
+changes and public claims.
+
+Favorable measurements do not automatically become performance claims. The
+short-KV `+15.2%` difference was the same order as repeated-run variance; the
+long-KV `+22.7%` result was a single observation and does not establish a
+general crossover. Neither is used for a general performance claim.
+
+## Documentation
+
+- [GitHub repository](https://github.com/leo1852098393-blip/MXFlashAttn)
+- [GitLink mirror](https://gitlink.org.cn/Leo77/MXFlashAttn)
+- [API guide](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/API.md)
+- [Support matrix](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/support-matrix.md)
+- [Evidence index](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/EVIDENCE.md)
+- [Release validation](https://github.com/leo1852098393-blip/MXFlashAttn/blob/main/docs/V1_5_RELEASE_VALIDATION.md)
+
+## License
+
+Apache License 2.0. MXMACA runtime, MetaX vendor packages, vLLM, model
+weights, and datasets remain subject to their respective licenses.