nvmeof 2026.7.21__tar.gz

nvmeof-2026.7.21/LICENSE

@@ -0,0 +1,28 @@
+BSD 3-Clause License
+
+Copyright (c) 2026, Tristan Rice
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+3. Neither the name of the copyright holder nor the names of its
+   contributors may be used to endorse or promote products derived from
+   this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

nvmeof-2026.7.21/PKG-INFO

@@ -0,0 +1,142 @@
+Metadata-Version: 2.4
+Name: nvmeof
+Version: 2026.7.21
+Summary: NVMe over Fabrics RDMA initiator with direct GPU memory support
+Author: Tristan Rice
+License-Expression: BSD-3-Clause
+Project-URL: Documentation, https://d4l3k.github.io/rdma4py/nvmeof/
+Project-URL: Homepage, https://github.com/d4l3k/rdma4py
+Project-URL: Repository, https://github.com/d4l3k/rdma4py
+Keywords: nvme,nvmeof,rdma,ibverbs,gpudirect,gpu
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: System :: Hardware
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: ibverbs>=0.1.0
+Provides-Extra: test
+Requires-Dist: pytest>=7; extra == "test"
+Requires-Dist: numpy; extra == "test"
+Provides-Extra: gpu
+Requires-Dist: torch; extra == "gpu"
+Dynamic: license-file
+
+# nvmeof
+
+`nvmeof` is a userspace NVMe over Fabrics RDMA initiator for Python. It is a
+separate package layered on
+[`ibverbs`](https://github.com/d4l3k/rdma4py/tree/main/ibverbs) because NVMe controller and
+namespace policy do not belong in low-level verbs bindings.
+
+The data path uses NVMe/RDMA keyed SGL descriptors that point at an existing
+ibverbs MR. The target transfers blocks directly to or from that MR, including
+CUDA memory registered by `ibverbs.cuda`; there is no host staging buffer in
+the NVMe-to-RDMA-to-GPU path.
+
+## Why this does not bundle SPDK
+
+SPDK is a strong choice for a managed native storage service, but it includes
+an environment abstraction, hugepage/driver setup, native plugins, and a large
+dependency graph. Embedding it would not produce a self-contained, portable
+`pip`/manylinux wheel. This package instead implements the small NVMe/RDMA host
+protocol directly over the existing portable `ibverbs` ABI3 extension.
+
+`nvmeof` itself is a pure-Python `py3-none-any` wheel. Its `ibverbs` dependency
+is an ABI3 manylinux wheel that `dlopen`s the host's `libibverbs.so.1` and,
+only when NVMe/RDMA is used, `librdmacm.so.1`.
+
+## Requirements
+
+- Linux, an RDMA NIC, and a reachable NVMe/RDMA target.
+- `libibverbs.so.1`, `librdmacm.so.1`, and the matching RDMA provider.
+- For GPU I/O: CUDA, an HCA with GPUDirect RDMA, and dma-buf support or the
+  `nvidia_peermem` module.
+- The target must support keyed SGL data block descriptors. Separate namespace
+  metadata and DH-HMAC-CHAP are not currently supported.
+
+On Debian/Ubuntu the runtime packages are typically:
+
+```bash
+sudo apt-get install libibverbs1 librdmacm1 ibverbs-providers
+python -m pip install nvmeof
+```
+
+## Host memory
+
+```python
+import nvmeof
+
+with nvmeof.Controller.connect(
+    "192.0.2.20",
+    "nqn.2026-07.io.example:storage",
+    source="192.0.2.21",
+) as controller:
+    namespace = controller.namespace(1)
+    with controller.allocate(128 * 4096) as buffer:
+        namespace.read(buffer, slba=0, blocks=128)
+        payload = buffer.read()
+```
+
+`controller.register(array_or_cpu_tensor)` registers an existing contiguous
+host allocation instead.
+
+## Direct GPU memory
+
+```python
+import os
+os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"
+
+import torch
+import nvmeof
+
+tensor = torch.empty(128 * 4096, dtype=torch.uint8, device="cuda:0")
+
+with nvmeof.Controller.connect(
+    "192.0.2.20", "nqn.2026-07.io.example:storage"
+) as controller:
+    namespace = controller.namespace(1)
+    with controller.register_gpu(tensor) as gpu_mr:
+        # Target NVMe -> target RDMA WRITE -> local GPU memory.
+        # Namespace.read flushes completed GPUDirect writes before returning.
+        with torch.cuda.device(tensor.device):
+            namespace.read(gpu_mr, slba=0, blocks=128)
+
+        # Local GPU memory -> target RDMA READ -> target NVMe.
+        # Namespace.write synchronizes the current CUDA context first.
+        with torch.cuda.device(tensor.device):
+            namespace.write(gpu_mr, slba=1024, blocks=128)
+            namespace.flush()
+```
+
+The CUDA context that owns the tensor must be current while issuing GPU I/O.
+The registered MR must be closed before its controller.
+
+`source` is optional. Set it to an address on the intended initiator HCA when
+the host has multiple RDMA NICs. This makes an NVMe -> target NIC -> initiator
+NIC -> GPU route explicit and prevents route selection from silently choosing
+an HCA with poor GPU PCIe locality.
+
+Reads and writes larger than the target's MDTS are split into multiple NVMe
+commands while retaining direct placement in the same registered buffer. The
+target used for the repository benchmarks advertises a 1 MiB MDTS; 4, 16, and
+64 MiB GPU tensors were verified end to end.
+
+## Scope
+
+The first implementation provides dynamic controller connection, controller
+enablement, Identify Controller/Namespace, one I/O queue, synchronous and
+low-level asynchronous command submission, block read/write splitting, and
+flush. It deliberately does not claim filesystem semantics, multipath,
+reconnect, keep-alive, protection information, authentication, or target
+functionality. KATO is therefore negotiated as zero.
+
+Real-target tests are opt-in through `NVME4PY_TARGET` and
+`NVME4PY_SUBSYSTEM_NQN`. Set `NVME4PY_SOURCE` to bind the initiator to a
+specific HCA address. The GPU round-trip test also requires
+`NVME4PY_DESTRUCTIVE=1` and must only be used with a disposable namespace.
+
+Measured multi-QP bandwidth, latency, QP scaling, and PCIe locality results
+are in [`BENCHMARKS.md`](BENCHMARKS.md).

nvmeof-2026.7.21/README.md

@@ -0,0 +1,116 @@
+# nvmeof
+
+`nvmeof` is a userspace NVMe over Fabrics RDMA initiator for Python. It is a
+separate package layered on
+[`ibverbs`](https://github.com/d4l3k/rdma4py/tree/main/ibverbs) because NVMe controller and
+namespace policy do not belong in low-level verbs bindings.
+
+The data path uses NVMe/RDMA keyed SGL descriptors that point at an existing
+ibverbs MR. The target transfers blocks directly to or from that MR, including
+CUDA memory registered by `ibverbs.cuda`; there is no host staging buffer in
+the NVMe-to-RDMA-to-GPU path.
+
+## Why this does not bundle SPDK
+
+SPDK is a strong choice for a managed native storage service, but it includes
+an environment abstraction, hugepage/driver setup, native plugins, and a large
+dependency graph. Embedding it would not produce a self-contained, portable
+`pip`/manylinux wheel. This package instead implements the small NVMe/RDMA host
+protocol directly over the existing portable `ibverbs` ABI3 extension.
+
+`nvmeof` itself is a pure-Python `py3-none-any` wheel. Its `ibverbs` dependency
+is an ABI3 manylinux wheel that `dlopen`s the host's `libibverbs.so.1` and,
+only when NVMe/RDMA is used, `librdmacm.so.1`.
+
+## Requirements
+
+- Linux, an RDMA NIC, and a reachable NVMe/RDMA target.
+- `libibverbs.so.1`, `librdmacm.so.1`, and the matching RDMA provider.
+- For GPU I/O: CUDA, an HCA with GPUDirect RDMA, and dma-buf support or the
+  `nvidia_peermem` module.
+- The target must support keyed SGL data block descriptors. Separate namespace
+  metadata and DH-HMAC-CHAP are not currently supported.
+
+On Debian/Ubuntu the runtime packages are typically:
+
+```bash
+sudo apt-get install libibverbs1 librdmacm1 ibverbs-providers
+python -m pip install nvmeof
+```
+
+## Host memory
+
+```python
+import nvmeof
+
+with nvmeof.Controller.connect(
+    "192.0.2.20",
+    "nqn.2026-07.io.example:storage",
+    source="192.0.2.21",
+) as controller:
+    namespace = controller.namespace(1)
+    with controller.allocate(128 * 4096) as buffer:
+        namespace.read(buffer, slba=0, blocks=128)
+        payload = buffer.read()
+```
+
+`controller.register(array_or_cpu_tensor)` registers an existing contiguous
+host allocation instead.
+
+## Direct GPU memory
+
+```python
+import os
+os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"
+
+import torch
+import nvmeof
+
+tensor = torch.empty(128 * 4096, dtype=torch.uint8, device="cuda:0")
+
+with nvmeof.Controller.connect(
+    "192.0.2.20", "nqn.2026-07.io.example:storage"
+) as controller:
+    namespace = controller.namespace(1)
+    with controller.register_gpu(tensor) as gpu_mr:
+        # Target NVMe -> target RDMA WRITE -> local GPU memory.
+        # Namespace.read flushes completed GPUDirect writes before returning.
+        with torch.cuda.device(tensor.device):
+            namespace.read(gpu_mr, slba=0, blocks=128)
+
+        # Local GPU memory -> target RDMA READ -> target NVMe.
+        # Namespace.write synchronizes the current CUDA context first.
+        with torch.cuda.device(tensor.device):
+            namespace.write(gpu_mr, slba=1024, blocks=128)
+            namespace.flush()
+```
+
+The CUDA context that owns the tensor must be current while issuing GPU I/O.
+The registered MR must be closed before its controller.
+
+`source` is optional. Set it to an address on the intended initiator HCA when
+the host has multiple RDMA NICs. This makes an NVMe -> target NIC -> initiator
+NIC -> GPU route explicit and prevents route selection from silently choosing
+an HCA with poor GPU PCIe locality.
+
+Reads and writes larger than the target's MDTS are split into multiple NVMe
+commands while retaining direct placement in the same registered buffer. The
+target used for the repository benchmarks advertises a 1 MiB MDTS; 4, 16, and
+64 MiB GPU tensors were verified end to end.
+
+## Scope
+
+The first implementation provides dynamic controller connection, controller
+enablement, Identify Controller/Namespace, one I/O queue, synchronous and
+low-level asynchronous command submission, block read/write splitting, and
+flush. It deliberately does not claim filesystem semantics, multipath,
+reconnect, keep-alive, protection information, authentication, or target
+functionality. KATO is therefore negotiated as zero.
+
+Real-target tests are opt-in through `NVME4PY_TARGET` and
+`NVME4PY_SUBSYSTEM_NQN`. Set `NVME4PY_SOURCE` to bind the initiator to a
+specific HCA address. The GPU round-trip test also requires
+`NVME4PY_DESTRUCTIVE=1` and must only be used with a disposable namespace.
+
+Measured multi-QP bandwidth, latency, QP scaling, and PCIe locality results
+are in [`BENCHMARKS.md`](BENCHMARKS.md).

nvmeof-2026.7.21/pyproject.toml

@@ -0,0 +1,47 @@
+[build-system]
+requires = ["setuptools>=77", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "nvmeof"
+dynamic = ["version"]
+description = "NVMe over Fabrics RDMA initiator with direct GPU memory support"
+readme = "README.md"
+requires-python = ">=3.9"
+license = "BSD-3-Clause"
+license-files = ["LICENSE"]
+authors = [{ name = "Tristan Rice" }]
+keywords = ["nvme", "nvmeof", "rdma", "ibverbs", "gpudirect", "gpu"]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Operating System :: POSIX :: Linux",
+    "Programming Language :: Python :: 3",
+    "Topic :: System :: Hardware",
+]
+dependencies = ["ibverbs>=0.1.0"]
+
+[project.optional-dependencies]
+test = ["pytest>=7", "numpy"]
+gpu = ["torch"]
+
+[tool.setuptools.dynamic]
+version = { attr = "nvmeof._version.__version__" }
+
+[tool.setuptools]
+package-dir = { "" = "src" }
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.pytest.ini_options]
+markers = [
+    "integration: exercises a real NVMe/RDMA target",
+    "gpu: requires CUDA and GPUDirect RDMA",
+]
+testpaths = ["tests"]
+
+[project.urls]
+Documentation = "https://d4l3k.github.io/rdma4py/nvmeof/"
+Homepage = "https://github.com/d4l3k/rdma4py"
+Repository = "https://github.com/d4l3k/rdma4py"

nvmeof-2026.7.21/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

nvmeof-2026.7.21/src/nvmeof/__init__.py

@@ -0,0 +1,27 @@
+"""Userspace NVMe over Fabrics RDMA with direct host and GPU I/O."""
+
+from ._version import __version__
+from .controller import Controller, Namespace
+from .protocol import (
+    Completion,
+    ControllerInfo,
+    NamespaceInfo,
+    NVME_RDMA_PORT,
+    NVMeStatusError,
+)
+from .rdma import HostBuffer, QueueFullError, RDMAQueue, Request
+
+__all__ = [
+    "__version__",
+    "Completion",
+    "Controller",
+    "ControllerInfo",
+    "HostBuffer",
+    "Namespace",
+    "NamespaceInfo",
+    "NVMeStatusError",
+    "NVME_RDMA_PORT",
+    "QueueFullError",
+    "RDMAQueue",
+    "Request",
+]

nvmeof-2026.7.21/src/nvmeof/_version.py

@@ -0,0 +1 @@
+__version__ = "2026.7.21"

nvmeof-2026.7.21/src/nvmeof/controller.py

@@ -0,0 +1,314 @@
+"""High-level NVMe-oF controller and namespace APIs."""
+
+from __future__ import annotations
+
+import time
+import uuid
+
+import ibverbs as ib
+
+from . import protocol as p
+from .rdma import HostBuffer, RDMAQueue
+
+_DATA_ACCESS = (
+    ib.AccessFlags.LOCAL_WRITE
+    | ib.AccessFlags.REMOTE_WRITE
+    | ib.AccessFlags.REMOTE_READ
+)
+
+
+def _is_gpu_memory(buffer) -> bool:
+    tensor = getattr(buffer, "tensor", None)
+    return tensor is not None and bool(getattr(tensor, "is_cuda", True))
+
+
+class Controller:
+    """A userspace NVMe-oF/RDMA controller with one direct I/O queue."""
+
+    def __init__(
+        self,
+        host: str,
+        subsystem_nqn: str,
+        *,
+        port=p.NVME_RDMA_PORT,
+        host_id=None,
+        host_nqn=None,
+        queue_depth=128,
+        keep_alive_ms=0,
+        timeout=30.0,
+        source=None,
+    ):
+        self.host = host
+        self.port = int(port)
+        self.source = source
+        self.subsystem_nqn = subsystem_nqn
+        self.host_id = uuid.UUID(str(host_id)) if host_id else uuid.uuid4()
+        self.host_nqn = host_nqn or (
+            "nqn.2014-08.org.nvmexpress:uuid:%s" % self.host_id
+        )
+        self.keep_alive_ms = int(keep_alive_ms)
+        if self.keep_alive_ms != 0:
+            raise NotImplementedError(
+                "non-zero keep-alive requires a background command worker"
+            )
+        self.timeout = float(timeout)
+        self.admin = None
+        self.io = None
+        self.controller_id = None
+        self.capabilities = None
+        self.info = None
+        self.max_transfer_bytes = None
+        try:
+            self._connect(int(queue_depth))
+        except Exception:
+            try:
+                self.close()
+            except OSError:
+                pass
+            raise
+
+    @classmethod
+    def connect(cls, host: str, subsystem_nqn: str, **kwargs) -> "Controller":
+        return cls(host, subsystem_nqn, **kwargs)
+
+    def _connect(self, requested_depth: int) -> None:
+        if requested_depth < 2 or requested_depth > p.MAX_QUEUE_DEPTH:
+            raise ValueError("queue_depth must be between 2 and 256")
+        self.admin = RDMAQueue(
+            self.host,
+            self.port,
+            qid=0,
+            depth=p.ADMIN_QUEUE_DEPTH,
+            source=self.source,
+        )
+        with HostBuffer(self.admin.pd, 1024) as data:
+            data.write(
+                p.connect_data(self.host_id, self.subsystem_nqn, self.host_nqn, 0xFFFF)
+            )
+            completion = self.admin.execute(
+                p.fabrics_connect(0, p.ADMIN_QUEUE_DEPTH, data, self.keep_alive_ms),
+                data_owner=data,
+                timeout=self.timeout,
+            )
+        self.controller_id = completion.result & 0xFFFF
+        auth_required = completion.result & ((1 << 17) | (1 << 18))
+        if auth_required:
+            raise NotImplementedError("NVMe DH-HMAC-CHAP authentication is required")
+
+        self.capabilities = self._property_get(p.REG_CAP, 8)
+        mpsmin = (self.capabilities >> 48) & 0xF
+        self.controller_page_size = 1 << (12 + mpsmin)
+        css = (self.capabilities >> 37) & 0xFF
+        cc = p.CC_ENABLE | p.CC_IOSQES | p.CC_IOCQES | (mpsmin << p.CC_MPS_SHIFT)
+        if css & (1 << 6):
+            cc |= p.CC_CSS_CSI
+        self._property_set(p.REG_CC, cc)
+        self._wait_ready()
+
+        with HostBuffer(self.admin.pd, 4096) as identify:
+            self.admin.execute(
+                p.identify(identify, controller=True),
+                data_owner=identify,
+                timeout=self.timeout,
+            )
+            self.info = p.ControllerInfo.from_bytes(identify.read())
+        if self.info.controller_id != self.controller_id:
+            raise RuntimeError("Connect and Identify returned different controller IDs")
+        if not (self.info.sgls & (1 << 2)):
+            raise RuntimeError("NVMe/RDMA target does not advertise keyed SGL support")
+        if self.info.icdoff:
+            raise RuntimeError("non-zero NVMe in-capsule data offsets are unsupported")
+
+        queue_limit = (self.capabilities & 0xFFFF) + 1
+        if self.info.max_commands:
+            queue_limit = min(queue_limit, self.info.max_commands)
+        depth = min(requested_depth, queue_limit, p.MAX_QUEUE_DEPTH)
+        queue_count = (0 << 16) | 0  # Request one submission and one completion queue.
+        completion = self.admin.execute(
+            p.set_features(p.FEAT_NUMBER_OF_QUEUES, queue_count),
+            timeout=self.timeout,
+        )
+        if min(completion.result & 0xFFFF, completion.result >> 16) + 1 < 1:
+            raise RuntimeError("NVMe target did not allocate an I/O queue")
+
+        self.io = RDMAQueue(
+            self.host,
+            self.port,
+            qid=1,
+            depth=depth,
+            controller_id=self.controller_id,
+            source=self.source,
+        )
+        with HostBuffer(self.io.pd, 1024) as data:
+            data.write(
+                p.connect_data(
+                    self.host_id,
+                    self.subsystem_nqn,
+                    self.host_nqn,
+                    self.controller_id,
+                )
+            )
+            self.io.execute(
+                p.fabrics_connect(1, depth, data),
+                data_owner=data,
+                timeout=self.timeout,
+            )
+        mdts = p.MAX_KEYED_SGL_LENGTH
+        if self.info.mdts:
+            mdts = min(mdts, self.controller_page_size << self.info.mdts)
+        self.max_transfer_bytes = mdts
+
+    def _property_get(self, offset: int, size: int) -> int:
+        return self.admin.execute(
+            p.property_get(offset, size), timeout=self.timeout
+        ).result
+
+    def _property_set(self, offset: int, value: int, size: int = 4) -> None:
+        self.admin.execute(p.property_set(offset, value, size), timeout=self.timeout)
+
+    def _wait_ready(self) -> None:
+        timeout = ((self.capabilities >> 24) & 0xFF) + 1
+        deadline = time.monotonic() + max(0.5, timeout * 0.5)
+        while True:
+            csts = self._property_get(p.REG_CSTS, 4)
+            if csts & p.CSTS_FATAL:
+                raise RuntimeError("NVMe controller entered fatal status")
+            if csts & p.CSTS_READY:
+                return
+            if time.monotonic() >= deadline:
+                raise TimeoutError("NVMe controller did not become ready")
+            time.sleep(0.01)
+
+    def identify_namespace(self, nsid: int) -> "Namespace":
+        nsid = int(nsid)
+        if nsid <= 0 or nsid > self.info.namespace_count:
+            raise ValueError("namespace ID is outside the controller's range")
+        with HostBuffer(self.admin.pd, 4096) as identify:
+            self.admin.execute(
+                p.identify(identify, nsid=nsid),
+                data_owner=identify,
+                timeout=self.timeout,
+            )
+            info = p.NamespaceInfo.from_bytes(nsid, identify.read())
+        if info.metadata_size:
+            raise NotImplementedError(
+                "namespaces with separate metadata are unsupported"
+            )
+        return Namespace(self, info)
+
+    namespace = identify_namespace
+
+    def allocate(self, length: int) -> HostBuffer:
+        """Allocate host memory registered for direct NVMe/RDMA I/O."""
+        return HostBuffer(self.io.pd, length, _DATA_ACCESS)
+
+    def register(self, tensor):
+        """Register contiguous host tensor/array memory for NVMe/RDMA I/O."""
+        return ib.reg_tensor(self.io.pd, tensor, _DATA_ACCESS)
+
+    def register_gpu(self, tensor):
+        """Register CUDA tensor memory for direct NVMe/RDMA I/O."""
+        import ibverbs.cuda as ibcuda
+
+        return ibcuda.register_tensor(self.io.pd, tensor, _DATA_ACCESS)
+
+    def close(self) -> None:
+        error = None
+        if self.io is not None:
+            try:
+                self.io.close()
+                self.io = None
+            except OSError as exc:
+                error = exc
+        if self.admin is not None:
+            try:
+                self.admin.close()
+                self.admin = None
+            except OSError as exc:
+                if error is None:
+                    error = exc
+        if error is not None:
+            raise error
+
+    def __enter__(self):
+        return self
+
+    def __exit__(self, *exc):
+        self.close()
+        return False
+
+
+class Namespace:
+    """An NVM namespace accessed through the controller's direct I/O queue."""
+
+    def __init__(self, controller: Controller, info: p.NamespaceInfo):
+        self.controller = controller
+        self.info = info
+        self.nsid = info.nsid
+        self.lba_size = info.lba_size
+
+    def _transfer(
+        self, opcode: int, buffer, slba: int, blocks=None, buffer_offset: int = 0
+    ) -> None:
+        slba = int(slba)
+        buffer_offset = int(buffer_offset)
+        available = int(buffer.length) - buffer_offset
+        if blocks is None:
+            if available % self.lba_size:
+                raise ValueError("buffer length is not a whole number of LBAs")
+            blocks = available // self.lba_size
+        blocks = int(blocks)
+        if blocks <= 0 or slba < 0 or slba + blocks > self.info.size_lbas:
+            raise ValueError("I/O range is outside the namespace")
+        total = blocks * self.lba_size
+        if buffer_offset < 0 or total > available:
+            raise ValueError("I/O range is outside the registered buffer")
+
+        gpu = _is_gpu_memory(buffer)
+        if gpu and opcode == p.OPC_WRITE:
+            import ibverbs.cuda as ibcuda
+
+            ibcuda.synchronize()
+
+        max_blocks = min(65536, self.controller.max_transfer_bytes // self.lba_size)
+        if max_blocks == 0:
+            raise RuntimeError(
+                "namespace LBA size exceeds the controller transfer limit"
+            )
+        remaining = blocks
+        offset = buffer_offset
+        current_lba = slba
+        while remaining:
+            count = min(remaining, max_blocks)
+            command = p.rw_command(
+                opcode,
+                self.nsid,
+                current_lba,
+                count,
+                buffer,
+                lba_size=self.lba_size,
+                buffer_offset=offset,
+            )
+            self.controller.io.execute(
+                command, data_owner=buffer, timeout=self.controller.timeout
+            )
+            current_lba += count
+            offset += count * self.lba_size
+            remaining -= count
+
+        if gpu and opcode == p.OPC_READ:
+            import ibverbs.cuda as ibcuda
+
+            ibcuda.flush_gpudirect_writes()
+
+    def read(self, buffer, slba: int, blocks=None, *, buffer_offset: int = 0) -> None:
+        """Read namespace LBAs directly into a registered host/GPU buffer."""
+        self._transfer(p.OPC_READ, buffer, slba, blocks, buffer_offset)
+
+    def write(self, buffer, slba: int, blocks=None, *, buffer_offset: int = 0) -> None:
+        """Write namespace LBAs directly from a registered host/GPU buffer."""
+        self._transfer(p.OPC_WRITE, buffer, slba, blocks, buffer_offset)
+
+    def flush(self) -> None:
+        command = p._command(p.OPC_FLUSH, self.nsid)
+        self.controller.io.execute(bytes(command), timeout=self.controller.timeout)