deltagraphar 0.1.0__tar.gz

deltagraphar-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Nishank Mahore
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

deltagraphar-0.1.0/PKG-INFO

@@ -0,0 +1,204 @@
+Metadata-Version: 2.4
+Name: deltagraphar
+Version: 0.1.0
+Summary: A mutable, versioned property-graph store built on the GraphAr physical layout with ACID semantics delegated to LakeFS.
+Author-email: Nishank Mahore <nishankmahore@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/nishankmahore/DeltaGraphAr
+Project-URL: Repository, https://github.com/nishankmahore/DeltaGraphAr
+Project-URL: Bug Tracker, https://github.com/nishankmahore/DeltaGraphAr/issues
+Keywords: graph,graphar,lakefs,parquet,versioning,property-graph
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Database
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pyarrow>=14.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: lakefs>=0.7
+Requires-Dist: lakefs-spec>=0.12
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: hypothesis>=6.100; extra == "dev"
+Requires-Dist: pandas>=2.0; extra == "dev"
+Dynamic: license-file
+
+# DeltaGraphAr
+
+A mutable, versioned property-graph store built on the [GraphAr](https://graphar.apache.org) physical layout (chunked Parquet + YAML metadata) with ACID semantics delegated to [LakeFS](https://lakefs.io).
+
+Pure-Python reference implementation. Suitable for graph datasets that evolve over time and need repeatable reads at arbitrary historical snapshots.
+
+## What it does
+
+- Stores vertices and edges as chunked Parquet files following the GraphAr layout spec.
+- Appends edges to an unordered "delta" region; CSR-ordered adjacency is built on demand via `compact()`.
+- Every mutating operation produces a versioned commit. Any commit ref can be used as a `ref=` argument to read historical state.
+- Vertices are identified by arbitrary string logical IDs; the ID map translates to contiguous physical chunk-aligned integers for storage.
+- LakeFS backend delegates branching, tagging, and atomic commits to a running LakeFS instance. The local backend (copy-on-commit) requires no external dependencies.
+
+## Install
+
+```bash
+pip install deltagraphar
+```
+
+Requires Python ≥ 3.10.
+
+For development (includes pytest, hypothesis, pandas):
+
+```bash
+git clone https://github.com/nishankmahore/DeltaGraphAr.git
+cd DeltaGraphAr
+pip install -e ".[dev]"
+```
+
+## Quickstart
+
+```bash
+python examples/quickstart.py
+```
+
+Or with LakeFS (requires `docker compose up` first):
+
+```bash
+docker compose up -d
+python examples/ldbc_snb_tiny_loader.py
+```
+
+## API
+
+```python
+from deltagraphar.versioning.local_backend import LocalBackend
+from deltagraphar.store.graphstore import GraphStore
+from deltagraphar.format.schema import GraphInfo, VertexInfo, EdgeInfo
+
+b = LocalBackend("/path/to/repo")
+vi = VertexInfo(label="person", chunk_size=65_536)
+ei = EdgeInfo("person", "knows", "person", chunk_size=1_048_576, src_chunk_size=65_536)
+gi = GraphInfo(name="social", prefix="", vertex_infos=[vi], edge_infos=[ei])
+
+gs = GraphStore.create(b, gi)
+gs.add_vertices("person", [{"id": "alice"}, {"id": "bob"}])
+gs.add_edges(("person", "knows", "person"), [{"src": "alice", "dst": "bob"}])
+gs.compact(("person", "knows", "person"))
+
+neighbors = gs.out_neighbors("person", "alice", ("person", "knows", "person"))
+# → ["bob"]
+
+# Time travel
+ref = gs.snapshots()[1].ref
+old_neighbors = gs.out_neighbors("person", "alice", ("person", "knows", "person"), ref=ref)
+```
+
+## CLI
+
+```bash
+deltagraphar log --repo /path/to/repo
+deltagraphar neighbors --repo /path/to/repo --label person --vertex alice --etype person,knows,person
+deltagraphar compact --repo /path/to/repo --etype person,knows,person
+deltagraphar tag --repo /path/to/repo v1
+```
+
+## Schema evolution
+
+Add a new property group to existing vertices without rewriting existing data:
+
+```python
+from deltagraphar.format.schema import PropertyGroup, Property
+
+pg = PropertyGroup([Property("score", "float64")], prefix="person_score")
+gs.add_property_group("vertex:person", pg, {"alice": 0.9, "bob": 0.7})
+```
+
+## Tests
+
+```bash
+pytest
+```
+
+51 tests, 2 skipped (LakeFS integration — requires `docker compose up`).
+
+## Benchmarks
+
+```bash
+python benchmarks/bench_v1.py --rows 10000 --queries 1000
+```
+
+## Architecture
+
+```
+GraphStore
+├── IDMap          — logical ↔ physical vertex ID, chunk-aligned Parquet
+├── compaction.py  — delta→CSR merge, offset sweep, property reorder
+└── VersioningBackend (ABC)
+    ├── LocalBackend   — copy-on-commit snapshots, no external deps
+    └── LakeFSBackend  — atomic commits, branching, tagging via LakeFS API
+
+Physical layout (GraphAr spec)
+  vertex/<label>/<pg_prefix>/chunk<k>         — vertex property tables
+  vertex/<label>/__vid_map__/chunk<k>         — ID map
+  edge/<src>_<et>_<dst>/ordered_by_source/    — CSR adj list + offsets
+  edge/<src>_<et>_<dst>/unordered_by_source/  — delta (append-only per vchunk)
+```
+
+## Data storage layout
+
+Data is stored as chunked Parquet files under a local repo directory. Using the movie graph as an example (`repo_dir = "/tmp/movies_repo"`):
+
+```
+/tmp/movies_repo/
+├── work/                                          ← current HEAD (mutable working copy)
+│   ├── movies.graph.yml                           ← graph manifest
+│   ├── Person.vertex.yml                          ← vertex schema
+│   ├── Movie.vertex.yml
+│   ├── vertex/
+│   │   ├── Person/
+│   │   │   ├── person_name/
+│   │   │   │   └── chunk0                        ← name column (Parquet)
+│   │   │   └── __vid_map__/
+│   │   │       └── chunk0                        ← logical↔physical ID map
+│   │   └── Movie/
+│   │       └── movie_props/
+│   │           └── chunk0                        ← title, released columns (Parquet)
+│   └── edge/
+│       └── Person_ACTED_IN_Movie/
+│           ├── Person_ACTED_IN_Movie.edge.yml     ← edge schema
+│           ├── ordered_by_source/                 ← CSR (written after compact)
+│           │   ├── adj_list/
+│           │   │   └── part0/chunk0              ← sorted src/dst pairs (Parquet)
+│           │   └── offset/
+│           │       └── part0/chunk0              ← CSR offset array (Parquet)
+│           └── unordered_by_source/               ← delta (append-only, pre-compact)
+│               └── adj_list/
+│                   └── part0/chunk0              ← unsorted src/dst pairs (Parquet)
+└── snapshots/
+    ├── <sha1ref>/                                 ← immutable copy-on-commit snapshot
+    ├── <sha1ref>/
+    └── ...                                        ← one directory per commit
+```
+
+To persist data across runs, replace `tempfile.TemporaryDirectory()` with a fixed path:
+
+```python
+repo_dir = "/tmp/movies_repo"
+b = LocalBackend(repo_dir)
+```
+
+To inspect any chunk file directly:
+
+```python
+import pyarrow.parquet as pq
+pq.read_table("/tmp/movies_repo/work/vertex/Person/person_name/chunk0").to_pandas()
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE)

deltagraphar-0.1.0/README.md

@@ -0,0 +1,172 @@
+# DeltaGraphAr
+
+A mutable, versioned property-graph store built on the [GraphAr](https://graphar.apache.org) physical layout (chunked Parquet + YAML metadata) with ACID semantics delegated to [LakeFS](https://lakefs.io).
+
+Pure-Python reference implementation. Suitable for graph datasets that evolve over time and need repeatable reads at arbitrary historical snapshots.
+
+## What it does
+
+- Stores vertices and edges as chunked Parquet files following the GraphAr layout spec.
+- Appends edges to an unordered "delta" region; CSR-ordered adjacency is built on demand via `compact()`.
+- Every mutating operation produces a versioned commit. Any commit ref can be used as a `ref=` argument to read historical state.
+- Vertices are identified by arbitrary string logical IDs; the ID map translates to contiguous physical chunk-aligned integers for storage.
+- LakeFS backend delegates branching, tagging, and atomic commits to a running LakeFS instance. The local backend (copy-on-commit) requires no external dependencies.
+
+## Install
+
+```bash
+pip install deltagraphar
+```
+
+Requires Python ≥ 3.10.
+
+For development (includes pytest, hypothesis, pandas):
+
+```bash
+git clone https://github.com/nishankmahore/DeltaGraphAr.git
+cd DeltaGraphAr
+pip install -e ".[dev]"
+```
+
+## Quickstart
+
+```bash
+python examples/quickstart.py
+```
+
+Or with LakeFS (requires `docker compose up` first):
+
+```bash
+docker compose up -d
+python examples/ldbc_snb_tiny_loader.py
+```
+
+## API
+
+```python
+from deltagraphar.versioning.local_backend import LocalBackend
+from deltagraphar.store.graphstore import GraphStore
+from deltagraphar.format.schema import GraphInfo, VertexInfo, EdgeInfo
+
+b = LocalBackend("/path/to/repo")
+vi = VertexInfo(label="person", chunk_size=65_536)
+ei = EdgeInfo("person", "knows", "person", chunk_size=1_048_576, src_chunk_size=65_536)
+gi = GraphInfo(name="social", prefix="", vertex_infos=[vi], edge_infos=[ei])
+
+gs = GraphStore.create(b, gi)
+gs.add_vertices("person", [{"id": "alice"}, {"id": "bob"}])
+gs.add_edges(("person", "knows", "person"), [{"src": "alice", "dst": "bob"}])
+gs.compact(("person", "knows", "person"))
+
+neighbors = gs.out_neighbors("person", "alice", ("person", "knows", "person"))
+# → ["bob"]
+
+# Time travel
+ref = gs.snapshots()[1].ref
+old_neighbors = gs.out_neighbors("person", "alice", ("person", "knows", "person"), ref=ref)
+```
+
+## CLI
+
+```bash
+deltagraphar log --repo /path/to/repo
+deltagraphar neighbors --repo /path/to/repo --label person --vertex alice --etype person,knows,person
+deltagraphar compact --repo /path/to/repo --etype person,knows,person
+deltagraphar tag --repo /path/to/repo v1
+```
+
+## Schema evolution
+
+Add a new property group to existing vertices without rewriting existing data:
+
+```python
+from deltagraphar.format.schema import PropertyGroup, Property
+
+pg = PropertyGroup([Property("score", "float64")], prefix="person_score")
+gs.add_property_group("vertex:person", pg, {"alice": 0.9, "bob": 0.7})
+```
+
+## Tests
+
+```bash
+pytest
+```
+
+51 tests, 2 skipped (LakeFS integration — requires `docker compose up`).
+
+## Benchmarks
+
+```bash
+python benchmarks/bench_v1.py --rows 10000 --queries 1000
+```
+
+## Architecture
+
+```
+GraphStore
+├── IDMap          — logical ↔ physical vertex ID, chunk-aligned Parquet
+├── compaction.py  — delta→CSR merge, offset sweep, property reorder
+└── VersioningBackend (ABC)
+    ├── LocalBackend   — copy-on-commit snapshots, no external deps
+    └── LakeFSBackend  — atomic commits, branching, tagging via LakeFS API
+
+Physical layout (GraphAr spec)
+  vertex/<label>/<pg_prefix>/chunk<k>         — vertex property tables
+  vertex/<label>/__vid_map__/chunk<k>         — ID map
+  edge/<src>_<et>_<dst>/ordered_by_source/    — CSR adj list + offsets
+  edge/<src>_<et>_<dst>/unordered_by_source/  — delta (append-only per vchunk)
+```
+
+## Data storage layout
+
+Data is stored as chunked Parquet files under a local repo directory. Using the movie graph as an example (`repo_dir = "/tmp/movies_repo"`):
+
+```
+/tmp/movies_repo/
+├── work/                                          ← current HEAD (mutable working copy)
+│   ├── movies.graph.yml                           ← graph manifest
+│   ├── Person.vertex.yml                          ← vertex schema
+│   ├── Movie.vertex.yml
+│   ├── vertex/
+│   │   ├── Person/
+│   │   │   ├── person_name/
+│   │   │   │   └── chunk0                        ← name column (Parquet)
+│   │   │   └── __vid_map__/
+│   │   │       └── chunk0                        ← logical↔physical ID map
+│   │   └── Movie/
+│   │       └── movie_props/
+│   │           └── chunk0                        ← title, released columns (Parquet)
+│   └── edge/
+│       └── Person_ACTED_IN_Movie/
+│           ├── Person_ACTED_IN_Movie.edge.yml     ← edge schema
+│           ├── ordered_by_source/                 ← CSR (written after compact)
+│           │   ├── adj_list/
+│           │   │   └── part0/chunk0              ← sorted src/dst pairs (Parquet)
+│           │   └── offset/
+│           │       └── part0/chunk0              ← CSR offset array (Parquet)
+│           └── unordered_by_source/               ← delta (append-only, pre-compact)
+│               └── adj_list/
+│                   └── part0/chunk0              ← unsorted src/dst pairs (Parquet)
+└── snapshots/
+    ├── <sha1ref>/                                 ← immutable copy-on-commit snapshot
+    ├── <sha1ref>/
+    └── ...                                        ← one directory per commit
+```
+
+To persist data across runs, replace `tempfile.TemporaryDirectory()` with a fixed path:
+
+```python
+repo_dir = "/tmp/movies_repo"
+b = LocalBackend(repo_dir)
+```
+
+To inspect any chunk file directly:
+
+```python
+import pyarrow.parquet as pq
+pq.read_table("/tmp/movies_repo/work/vertex/Person/person_name/chunk0").to_pandas()
+```
+
+## License
+
+MIT — see [LICENSE](LICENSE)

deltagraphar-0.1.0/deltagraphar/cli.py

@@ -0,0 +1,140 @@
+"""Command-line interface for DeltaGraphAr."""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+
+
+def _get_backend(args):
+    from deltagraphar.versioning.local_backend import LocalBackend
+    return LocalBackend(args.repo)
+
+
+def cmd_log(args):
+    b = _get_backend(args)
+    commits = b.log()
+    if not commits:
+        print("(no commits)")
+        return
+    for c in commits:
+        from datetime import datetime, timezone
+        dt = datetime.fromtimestamp(c.timestamp, tz=timezone.utc).strftime("%Y-%m-%d %H:%M:%S UTC")
+        meta = f"  {c.metadata}" if c.metadata else ""
+        print(f"{c.ref[:8]}  {dt}  {c.message}{meta}")
+
+
+def cmd_tag(args):
+    b = _get_backend(args)
+    log = b.log()
+    if not log:
+        print("error: no commits to tag", file=sys.stderr)
+        sys.exit(1)
+    ref = log[-1].ref
+    b.tag(args.name, ref)
+    print(f"tagged {ref[:8]} as {args.name!r}")
+
+
+def cmd_neighbors(args):
+    from deltagraphar.versioning.local_backend import LocalBackend
+    from deltagraphar.store.graphstore import GraphStore
+    from deltagraphar.format.reader import read_yaml
+    from deltagraphar.format.paths import vertex_yaml_path, edge_yaml_path
+    from deltagraphar.format.schema import GraphInfo, VertexInfo, EdgeInfo
+
+    b = LocalBackend(args.repo)
+    ref = args.ref or None
+
+    etype = tuple(args.etype.split(","))
+    if len(etype) != 3:
+        print("error: --etype must be 'src,edge,dst'", file=sys.stderr)
+        sys.exit(1)
+
+    src_label, et, dst_label = etype
+    vi_data = read_yaml(b, vertex_yaml_path(args.label), ref=ref)
+    ei_data = read_yaml(b, edge_yaml_path(src_label, et, dst_label), ref=ref)
+
+    vi = VertexInfo(label=vi_data["label"], chunk_size=vi_data["chunk_size"])
+    ei = EdgeInfo(
+        src_type=ei_data["src_type"],
+        edge_type=ei_data["edge_type"],
+        dst_type=ei_data["dst_type"],
+        chunk_size=ei_data["chunk_size"],
+        src_chunk_size=ei_data["src_chunk_size"],
+    )
+    gi = GraphInfo(name="graph", prefix="", vertex_infos=[vi], edge_infos=[ei])
+    gs = GraphStore(b, gi, vertex_chunk_size=vi_data["chunk_size"])
+    nbrs = gs.out_neighbors(args.label, args.vertex, etype, ref=ref)
+    print(json.dumps(nbrs))
+
+
+def cmd_compact(args):
+    from deltagraphar.versioning.local_backend import LocalBackend
+    from deltagraphar.store.graphstore import GraphStore
+    from deltagraphar.format.reader import read_yaml
+    from deltagraphar.format.paths import vertex_yaml_path, edge_yaml_path
+    from deltagraphar.format.schema import GraphInfo, VertexInfo, EdgeInfo
+
+    b = LocalBackend(args.repo)
+    etype = tuple(args.etype.split(","))
+    if len(etype) != 3:
+        print("error: --etype must be 'src,edge,dst'", file=sys.stderr)
+        sys.exit(1)
+
+    src_label, et, dst_label = etype
+    vi_data = read_yaml(b, vertex_yaml_path(src_label))
+    ei_data = read_yaml(b, edge_yaml_path(src_label, et, dst_label))
+
+    vi = VertexInfo(label=vi_data["label"], chunk_size=vi_data["chunk_size"])
+    ei = EdgeInfo(
+        src_type=ei_data["src_type"],
+        edge_type=ei_data["edge_type"],
+        dst_type=ei_data["dst_type"],
+        chunk_size=ei_data["chunk_size"],
+        src_chunk_size=ei_data["src_chunk_size"],
+    )
+    gi = GraphInfo(name="graph", prefix="", vertex_infos=[vi], edge_infos=[ei])
+    gs = GraphStore(b, gi, vertex_chunk_size=vi_data["chunk_size"])
+
+    vchunks = [int(v) for v in args.vchunks.split(",")] if args.vchunks else None
+    ref = gs.compact(etype, vchunks=vchunks)
+    print(f"compacted → {ref[:8]}")
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        prog="deltagraphar",
+        description="DeltaGraphAr — versioned property-graph store",
+    )
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    p_log = sub.add_parser("log", help="Show commit history")
+    p_log.add_argument("--repo", required=True, help="Path to local repo")
+    p_log.set_defaults(func=cmd_log)
+
+    p_tag = sub.add_parser("tag", help="Tag the latest commit")
+    p_tag.add_argument("--repo", required=True)
+    p_tag.add_argument("name", help="Tag name")
+    p_tag.set_defaults(func=cmd_tag)
+
+    p_nbr = sub.add_parser("neighbors", help="List out-neighbors of a vertex at a ref")
+    p_nbr.add_argument("--repo", required=True)
+    p_nbr.add_argument("--label", required=True, help="Vertex label")
+    p_nbr.add_argument("--vertex", required=True, help="Logical vertex ID")
+    p_nbr.add_argument("--etype", required=True, help="Edge type as 'src,edge,dst'")
+    p_nbr.add_argument("--ref", default=None, help="Commit ref or tag (default: HEAD)")
+    p_nbr.set_defaults(func=cmd_neighbors)
+
+    p_compact = sub.add_parser("compact", help="Compact delta into ordered CSR")
+    p_compact.add_argument("--repo", required=True)
+    p_compact.add_argument("--etype", required=True, help="Edge type as 'src,edge,dst'")
+    p_compact.add_argument("--vchunks", default=None,
+                           help="Comma-separated vchunk indices (default: auto-discover)")
+    p_compact.set_defaults(func=cmd_compact)
+
+    args = parser.parse_args()
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main()

deltagraphar-0.1.0/deltagraphar/format/paths.py

@@ -0,0 +1,35 @@
+def vertex_chunk_path(label: str, pg_prefix: str, chunk_idx: int) -> str:
+    return f"vertex/{label}/{pg_prefix}/chunk{chunk_idx}"
+
+
+def vid_map_chunk_path(label: str, chunk_idx: int) -> str:
+    return f"vertex/{label}/__vid_map__/chunk{chunk_idx}"
+
+
+def adj_list_chunk_path(
+    src: str, etype: str, dst: str, adj_type: str, vchunk: int, chunk_idx: int
+) -> str:
+    return f"edge/{src}_{etype}_{dst}/{adj_type}/adj_list/part{vchunk}/chunk{chunk_idx}"
+
+
+def offset_chunk_path(src: str, etype: str, dst: str, vchunk: int) -> str:
+    return f"edge/{src}_{etype}_{dst}/ordered_by_source/offset/part{vchunk}/chunk0"
+
+
+def edge_prop_chunk_path(
+    src: str, etype: str, dst: str,
+    adj_type: str, pg_prefix: str, vchunk: int, chunk_idx: int,
+) -> str:
+    return f"edge/{src}_{etype}_{dst}/{adj_type}/{pg_prefix}/part{vchunk}/chunk{chunk_idx}"
+
+
+def graph_yaml_path(name: str) -> str:
+    return f"{name}.graph.yml"
+
+
+def vertex_yaml_path(label: str) -> str:
+    return f"{label}.vertex.yml"
+
+
+def edge_yaml_path(src: str, etype: str, dst: str) -> str:
+    return f"{src}_{etype}_{dst}.edge.yml"

deltagraphar-0.1.0/deltagraphar/format/reader.py

@@ -0,0 +1,58 @@
+import io
+import yaml
+import pyarrow as pa
+import pyarrow.parquet as pq
+import pyarrow.compute as pc
+
+from deltagraphar.format.paths import offset_chunk_path
+
+
+def read_table(backend, path: str, ref=None) -> pa.Table:
+    data = backend.read_file(path, ref=ref)
+    return pq.read_table(io.BytesIO(data))
+
+
+def read_yaml(backend, path: str, ref=None) -> dict:
+    return yaml.safe_load(backend.read_file(path, ref=ref))
+
+
+def read_adj_list(
+    backend, src: str, etype: str, dst: str, adj_type: str, vchunk: int, ref=None
+) -> pa.Table:
+    """Concatenate all chunk files for one (adj_type, vchunk) partition."""
+    prefix = f"edge/{src}_{etype}_{dst}/{adj_type}/adj_list/part{vchunk}"
+    paths = sorted(backend.list(prefix, ref=ref))
+    if not paths:
+        return pa.table({
+            "src_physical": pa.array([], type=pa.int64()),
+            "dst_physical": pa.array([], type=pa.int64()),
+        })
+    return pa.concat_tables([read_table(backend, p, ref=ref) for p in paths])
+
+
+def read_offsets(backend, src: str, etype: str, dst: str, vchunk: int, ref=None) -> list[int]:
+    """Read the CSR offset array for a vchunk; returns [] if not yet compacted."""
+    path = offset_chunk_path(src, etype, dst, vchunk)
+    try:
+        tbl = read_table(backend, path, ref=ref)
+        return tbl["offset"].to_pylist()
+    except FileNotFoundError:
+        return []
+
+
+def scan_delta(
+    backend, src: str, etype: str, dst: str, vchunk: int, src_physical: int, ref=None
+) -> list[int]:
+    """Return dst_physical list from unordered delta where src_physical matches."""
+    delta = read_adj_list(backend, src, etype, dst, "unordered_by_source", vchunk, ref=ref)
+    if len(delta) == 0:
+        return []
+    mask = pc.equal(delta["src_physical"], src_physical)
+    return delta.filter(mask)["dst_physical"].to_pylist()
+
+
+def count_rows(
+    backend, src: str, etype: str, dst: str, adj_type: str, vchunk: int, ref=None
+) -> int:
+    tbl = read_adj_list(backend, src, etype, dst, adj_type, vchunk, ref=ref)
+    return len(tbl)