stream-cqsa 2.2.3__tar.gz → 2.2.4__tar.gz

{stream_cqsa-2.2.3/stream_cqsa.egg-info → stream_cqsa-2.2.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: stream-cqsa
-Version: 2.2.3
+Version: 2.2.4
 Summary: Stream-CQSA v2: exact out-of-memory recovery for attention (engine, planner, devkit, native kernels)
 Requires-Python: >=3.9
 License-File: LICENSE

{stream_cqsa-2.2.3 → stream_cqsa-2.2.4}/README.md

@@ -38,6 +38,12 @@ kept query–key pair is counted exactly once, so the result is exact.
 
 ## Install
 
+> **v2.2.4 (2026-11-01) fixes a wrong-result bug: upgrade.** In v2.1.0-v2.2.3, `attention(..., c=C)` with any
+> `C` other than 7 and no `interest_set=` returned a wrong result, fast and silently (the pinned `c` kept c=7's
+> quorum set, so most chunk pairs were never attended; relative error 0.7-1.8 against SDPA). The automatic path
+> (nothing pinned) was correct. v2.2.4 gives a pinned `c` its own set and the engine refuses any mismatched
+> pair. See the CHANGELOG.
+
 Two implementations of the kernels ship, and where you get them differs:
 
 | | kernels | where | install |
@@ -69,20 +75,20 @@ compiling: `CQSA_SKIP_EXT=1 pip install --no-build-isolation git+https://github.
 
 ### CUDA build (GitHub release page)
 
-The [release page](https://github.com/yiming-b/Stream-CQSA-v2/releases/tag/v2.2.3) carries, for
+The [release page](https://github.com/yiming-b/Stream-CQSA-v2/releases/tag/v2.2.4) carries, for
 every python 3.10/3.11/3.12 x torch 2.5.1/2.6.0 x CUDA 12.4 x GPU architecture (sm80 = A100 and
 other 8.x cards, sm90 = H100) cell, two wheels: `stream_cqsa` with the classic CUDA extensions and
 the companion `stream_cqsa_native` with the wave kernel. Install both; they replace the PyPI
 package with the same version plus the extensions:
 
 ```bash
-V=v2.2.3; T=1cu124torch2.6sm80; PY=cp311     # pick your torch / CUDA / GPU / python
-pip install https://github.com/yiming-b/Stream-CQSA-v2/releases/download/$V/stream_cqsa-2.2.3-$T-$PY-$PY-linux_x86_64.whl
-pip install https://github.com/yiming-b/Stream-CQSA-v2/releases/download/$V/stream_cqsa_native-2.2.3-$T-$PY-$PY-linux_x86_64.whl
+V=v2.2.4; T=1cu124torch2.6sm80; PY=cp311     # pick your torch / CUDA / GPU / python
+pip install https://github.com/yiming-b/Stream-CQSA-v2/releases/download/$V/stream_cqsa-2.2.4-$T-$PY-$PY-linux_x86_64.whl
+pip install https://github.com/yiming-b/Stream-CQSA-v2/releases/download/$V/stream_cqsa_native-2.2.4-$T-$PY-$PY-linux_x86_64.whl
 ```
 
 Also on the release page: one wheel with both extensions for torch 2.10 / CUDA 13 / python 3.11
-(`stream_cqsa-2.2.3-1cu130torch210sm8090-...whl`, sm80 + sm90), and the pure wheel and sdist that
+(`stream_cqsa-2.2.4-1cu130torch210sm8090-...whl`, sm80 + sm90), and the pure wheel and sdist that
 PyPI serves. The CUDA wheels are too large for PyPI (100 MB per file), which is why they live on
 GitHub, as PyTorch's own CUDA wheels do. They are built by `.github/workflows/wheels.yml` on every
 tag; publishing to PyPI is the `pypi` job, enabled by the repository variable `PYPI_PUBLISH=true`
@@ -222,6 +228,26 @@ Reproduce: `sbatch slurm/profile_sweep_<kernel>.slurm` for cuda / triton / wave_
 (one gpu-test job each, `--constraint="sxm&gpu80"` so that all four land on the same GPU model),
 then `python benchmarks/profile_sweep.py fit --out results/profile_sweep`.
 
+## Paper re-run (A100 80GB PCIe, `results/paper_rerun/`)
+
+The paper's single-GPU experiments re-run with the v2 engine (B=1 H=8 D=64, causal, fp16 / bf16, 262K-16.8M
+tokens, automatic depth with a floor of 1, every measurement in its own process). Baselines run on
+device-resident Q/K/V; Stream-CQSA streams Q/K/V from host memory (acc=GPU / acc=CPU = where the fp32
+accumulator lives). Native CUDA kernel, fp16 (time / peak device memory; ratio = Stream-CQSA's faster arm
+over FlashAttention-2):
+
+| N | FA-2 fwd | Stream-CQSA fwd | ratio | FA-2 fwd+bwd | Stream-CQSA fwd+bwd | ratio |
+|---|---|---|---|---|---|---|
+| 1.0M | 7.7 s / 5.0 GiB | 11.2 s / 6.5 GiB | 1.45x | 24.9 s / 10.1 GiB | 52.1 s / 13.6 GiB | 2.10x |
+| 4.2M | 128.4 s / 20.1 GiB | 153.2 s / 25.9 GiB | 1.19x | 405.1 s / 40.3 GiB | 799.2 s / 46.5 GiB | 1.97x |
+| 8.4M | 534.2 s / 40.3 GiB | 595.6 s / 51.8 GiB | 1.12x | not run (> 79.3 GiB) | 3189.8 s / 44.8 GiB (acc=CPU) | only Stream-CQSA |
+| 16.8M | not run (> 79.3 GiB) | 2536.2 s / 32.0 GiB (acc=CPU) | only Stream-CQSA | not run | 14353.0 s / 38.4 GiB (acc=CPU) | only Stream-CQSA |
+
+Full tables (both dtypes, SDPA mem-efficient, accuracy against float64, the Triton kernel set as its own
+table, not-run reasons): `results/paper_rerun/tables_native.md`, `tables_triton.md`; regenerate with
+`python results/paper_rerun/make_tables.py`. The forward+backward cost (~2x) is the v11 backward kernel
+this release ships; the Triton backward is already faster (732 s at 4.2M).
+
 ## License
 
 BSD-3-Clause (see `LICENSE`); vendored FlashAttention-2 and CUTLASS notices in `third_party/`.

{stream_cqsa-2.2.3 → stream_cqsa-2.2.4}/stream_cqsa/__init__.py

@@ -10,7 +10,7 @@ in this package (stream_cqsa_forward, stream_cqsa_attn, native_wave, distributed
 engine layer for kernel and systems developers; attention_oom_safe / stream_cqsa_auto / auto_attention
 are legacy entry points kept for old code.
 """
-__version__ = "2.2.3"
+__version__ = "2.2.4"
 
 from .stable_stream import (
     stream_cqsa_forward,

{stream_cqsa-2.2.3 → stream_cqsa-2.2.4}/stream_cqsa/api.py

@@ -404,6 +404,19 @@ def _run(q, k, v, p, *, is_causal, scale, direction, on_cuda, out_el, overrides,
                 q, k, v = (_pinned_copy(t) for t in (q, k, v))            # token-major: the engine streams it as is
             on_cuda = False
     kw.update(overrides)
+    # v2.2.4: a pinned c= carries its own quorum set. Before, `c` was overridden and `interest_set` kept the
+    # plan's (c=7's (0, 1, 3) whenever the plan was monolithic or c=7), which is not a perfect difference set
+    # for any other c: most chunk pairs were never attended and every pinned c != 7 returned a wrong result,
+    # fast and silently (relative error 0.7-1.8 against SDPA at c = 31..133).
+    if "c" in overrides and "interest_set" not in overrides:
+        from .autoconfig import QUORUM_SETS
+        try:
+            kw["interest_set"] = QUORUM_SETS[int(overrides["c"])]
+        except (KeyError, TypeError, ValueError):
+            raise ValueError(f"c={overrides['c']!r} is not a known quorum size; pass interest_set= with its perfect "
+                             f"difference set (known: {sorted(QUORUM_SETS)})") from None
+    from .stable_stream import check_quorum_set
+    check_quorum_set(int(kw.get("c", 7)), kw.get("interest_set", (0, 1, 3)))
     bwd_itr = kw.pop("bwd_itr", "auto")          # depth of the backward: "auto" (planned on its own), "fwd", or an int
     if not kw.get("stream_from_host"):
         kw["shared_chunks"] = False

{stream_cqsa-2.2.3 → stream_cqsa-2.2.4}/stream_cqsa/stable_stream.py

@@ -385,6 +385,38 @@ def max_depth_for(N: int, c: int = 7) -> int:
     return k
 
 
+def check_quorum_set(c: int, interest_set: Sequence[int]) -> None:
+    """
+    Refuse a ``(c, interest_set)`` pair that is not a perfect difference set.
+
+    The decomposition is exact only when every pair of chunks ``(i, j)`` lands in
+    exactly one subproblem. With an interest set that does not match ``c`` the
+    differences it covers are a subset of the residues, so most chunk pairs are
+    never gathered and the engine returns a result with most of the attention
+    SKIPPED -- silently, fast, and with a plausible-looking output. That is what
+    ``attention(..., c=73)`` did in v2.1.0-v2.2.3: it pinned ``c`` and kept the
+    plan's ``(0, 1, 3)``. The engine entry points therefore validate the pair
+    before any task is built (v2.2.4).
+    """
+    c = int(c)
+    iset = tuple(int(x) for x in interest_set)
+    if len(set(iset)) == len(iset):
+        d = sorted((a - b) % c for a in iset for b in iset if a != b)
+        if d == list(range(1, c)):
+            return
+    from .autoconfig import QUORUM_SETS        # lazy: autoconfig imports this module
+    covered = {(a - b) % c for i, a in enumerate(iset) for j, b in enumerate(iset) if i != j}
+    missing = (c - 1) - len(covered & set(range(1, c)))
+    known = QUORUM_SETS.get(c)
+    hint = (f"the quorum set for c={c} is interest_set={known}" if known is not None
+            else f"the known quorum sets are c in {sorted(QUORUM_SETS)} (stream_cqsa.autoconfig.QUORUM_SETS)")
+    raise ValueError(
+        f"(c={c}, interest_set={iset}) is not a perfect difference set: {missing} of the {c - 1} non-zero "
+        f"residues mod {c} are not a difference of two members, so that many chunk-pair classes would never be "
+        f"attended and the result would be wrong. {hint}; a pinned c= takes its own set."
+    )
+
+
 def refine_task(
     task: "SubproblemTask",
     N: int,
@@ -414,6 +446,7 @@ def refine_task(
     the hybrid schedules of the paper: branches that fit stay coarse, and only
     those that failed are split.
     """
+    check_quorum_set(c, interest_set)
     kids: list[SubproblemTask] = []
     align = int(seg_align) if seg_align else 1
     for j in range(int(c)):
@@ -476,6 +509,7 @@ def build_tasks(
     members, which is what makes the off-diagonal chunk pairs covered exactly
     once. ``c = l^2 - l + 1`` for ``l = len(interest_set)``.
     """
+    check_quorum_set(c, interest_set)
     tasks: list[SubproblemTask] = []
     for path_idx, path in enumerate(all_paths(itr, c)):
         align = int(seg_align) if seg_align else 1

{stream_cqsa-2.2.3 → stream_cqsa-2.2.4/stream_cqsa.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: stream-cqsa
-Version: 2.2.3
+Version: 2.2.4
 Summary: Stream-CQSA v2: exact out-of-memory recovery for attention (engine, planner, devkit, native kernels)
 Requires-Python: >=3.9
 License-File: LICENSE

{stream_cqsa-2.2.3 → stream_cqsa-2.2.4}/stream_cqsa.egg-info/SOURCES.txt

@@ -1899,6 +1899,7 @@ tests/test_devkit_autoconfig.py
 tests/test_escalation_autograd.py
 tests/test_evict_inputs.py
 tests/test_place_and_verbose.py
+tests/test_quorum_sizes.py
 tests/test_rect_ooc.py
 tests/test_reference.py
 tests/test_stable_stream.py

stream_cqsa-2.2.4/tests/test_quorum_sizes.py

@@ -0,0 +1,101 @@
+"""
+v2.2.4 regression: every quorum size pinned through ``stream_cqsa.attention(..., c=)`` computes exact attention.
+
+In v2.1.0-v2.2.3 ``attention(..., itr=1, c=73)`` pinned ``c`` but kept the plan's interest set ``(0, 1, 3)``
+(the c=7 set), so the engine ran 73 subproblems of 3 chunks each, most chunk pairs were never gathered, and the
+result had a norm-relative error of about 1.3 against flash attention, in a third of the time. Every c other
+than 7 was wrong (c=31: 0.72, c=57: 1.1, c=91: 1.4, c=133: 1.8); c=7 was correct only because its set is the
+default. Now a pinned ``c`` takes ``QUORUM_SETS[c]`` and the task builders refuse any pair that is not a perfect
+difference set (``stable_stream.check_quorum_set``).
+
+Rows: every c in ``QUORUM_SETS`` at N=65536, itr=1, fp16, causal, B=1 H=4 D=64, classic engine
+(``kernel="cuda"``), forward and backward against torch SDPA on the same inputs. The call is pinned (``itr=1``)
+so the monolithic path cannot be what is compared, and the info is asserted to report ``c`` subproblems with
+``QUORUM_SETS[c]``. Shared GPU: correctness only, < 2 GiB per row.
+"""
+import pytest
+import torch
+import torch.nn.functional as F
+
+cuda = pytest.mark.skipif(not torch.cuda.is_available(), reason="needs a GPU")
+B, H, D, N = 1, 4, 64, 65536
+# norm-relative error vs SDPA fp16 on the same inputs: 1.3e-4..2.4e-4 (out) / 3.5e-4 (grads) for every c
+# (A100, v11 module); the v2.2.3 failure is ~1e0
+TOL_OUT, TOL_GRAD = 1e-3, 1.5e-3
+
+
+def _quorum_sets():
+    from stream_cqsa.autoconfig import QUORUM_SETS
+    return QUORUM_SETS
+
+
+def _rel(a, b):
+    return ((a.double() - b.double()).norm() / b.double().norm()).item()
+
+
+def _inputs():
+    g = torch.Generator(device="cuda").manual_seed(N)
+    q, k, v = (torch.randn(B, H, N, D, device="cuda", dtype=torch.float16, generator=g) for _ in range(3))
+    dout = torch.randn(B, H, N, D, device="cuda", dtype=torch.float16, generator=g)
+    return q, k, v, dout
+
+
+def _reference(q, k, v, dout):
+    qr, kr, vr = (t.clone().requires_grad_(True) for t in (q, k, v))
+    out = F.scaled_dot_product_attention(qr, kr, vr, is_causal=True)
+    grads = torch.autograd.grad(out, (qr, kr, vr), dout)
+    return out.detach(), [g_.detach() for g_ in grads]
+
+
+@cuda
+@pytest.mark.parametrize("c", sorted(_quorum_sets()), ids=lambda c: f"c{c}")
+def test_pinned_quorum_set_is_exact(c):
+    from stream_cqsa import attention
+    QUORUM_SETS = _quorum_sets()
+    q, k, v, dout = _inputs()
+    ref, ref_grads = _reference(q, k, v, dout)
+    qs, ks, vs = (t.clone().requires_grad_(True) for t in (q, k, v))
+    out, info = attention(qs, ks, vs, is_causal=True, kernel="cuda", itr=1, c=c, bwd_itr=1, return_info=True)
+    # the decomposition must have run (an unpinned call that fits would be SDPA against itself)
+    assert info["engine"] == "classic" and info.get("itr") == 1 and info.get("c") == c, info
+    assert tuple(info.get("interest_set", QUORUM_SETS[c])) == QUORUM_SETS[c], info
+    grads = torch.autograd.grad(out, (qs, ks, vs), dout)
+    e_out = _rel(out, ref)
+    e_grad = max(_rel(g, r) for g, r in zip(grads, ref_grads))
+    assert e_out < TOL_OUT, f"c={c}: output rel error {e_out:.2e} (v2.2.3 bug: ~1e0)"
+    assert e_grad < TOL_GRAD, f"c={c}: gradient rel error {e_grad:.2e} (v2.2.3 bug: ~1e0)"
+
+
+@cuda
+def test_pinned_c_without_interest_set_reports_its_own_set():
+    """The forward info of a pinned c=73 reports 73 subproblems on (0, 1, 3, 7, 15, 31, 36, 54, 63)."""
+    from stream_cqsa import attention
+    q, k, v, _ = _inputs()
+    with torch.no_grad():
+        out, info = attention(q, k, v, is_causal=True, kernel="cuda", itr=1, c=73, return_info=True)
+    assert info["n_subproblems"] == 73 and tuple(info["interest_set"]) == _quorum_sets()[73], info
+
+
+def test_task_builders_refuse_a_mismatched_pair():
+    from stream_cqsa.stable_stream import build_tasks, check_quorum_set
+    for c, iset in ((73, (0, 1, 3)), (7, (0, 1, 2)), (7, (0, 1, 1)), (13, (0, 1, 3))):
+        with pytest.raises(ValueError, match="perfect difference set"):
+            check_quorum_set(c, iset)
+        with pytest.raises(ValueError, match="perfect difference set"):
+            build_tasks(8192, 1, B=1, H=1, D=64, itemsize=2, c=c, interest_set=iset)
+
+
+def test_every_quorum_set_passes_the_guard():
+    from stream_cqsa.stable_stream import check_quorum_set
+    for c, iset in _quorum_sets().items():
+        check_quorum_set(c, iset)
+
+
+def test_attention_rejects_a_mismatched_or_unknown_pin():
+    from stream_cqsa import attention
+    q, k, v = (torch.randn(1, 2, 8192, 64, dtype=torch.float16) for _ in range(3))
+    hw = {"cuda:0": "40GiB", "host": "64GiB"}
+    with pytest.raises(ValueError, match="perfect difference set"):
+        attention(q, k, v, is_causal=True, hardware=hw, itr=1, c=73, interest_set=(0, 1, 3))
+    with pytest.raises(ValueError, match="not a known quorum size"):
+        attention(q, k, v, is_causal=True, hardware=hw, itr=1, c=11)