hyperstreamdb 0.5.2__tar.gz → 0.5.4__tar.gz

hyperstreamdb-0.5.4/.cargo/audit.toml

@@ -0,0 +1,11 @@
+[advisories]
+ignore = [
+    "RUSTSEC-2026-0104",
+    "RUSTSEC-2026-0099",
+    "RUSTSEC-2026-0098",
+    "RUSTSEC-2026-0097",
+    "RUSTSEC-2024-0437",
+    "RUSTSEC-2026-0177",
+    "RUSTSEC-2026-0176",
+    "RUSTSEC-2023-0071"
+]

hyperstreamdb-0.5.4/ADMIN_CLI.md

@@ -0,0 +1,59 @@
+# HyperStreamDB Admin CLI
+
+`hyperstreamdb-admin` is the official administrative tooling for HyperStreamDB. It provides cluster operators with safe, easy-to-use commands to manage storage, optimize performance, and clean up historical data.
+
+## Installation / Building
+If you are compiling from source, the admin CLI is built alongside the main engine:
+```bash
+cargo build --release --bin hyperstreamdb-admin
+```
+
+The compiled binary will be located at `target/release/hyperstreamdb-admin`.
+
+## Commands
+
+### 1. `compact`
+Over time, continuous ingestion can lead to fragmentation—creating hundreds or thousands of small data files (often called the "small files problem"). The `compact` command merges these small files into larger, optimized files. This significantly improves query performance by reducing metadata overhead and maximizing the efficiency of vectorized scans.
+
+**Usage:**
+```bash
+hyperstreamdb-admin compact --uri <TABLE_URI> [OPTIONS]
+```
+
+**Options:**
+- `--uri`, `-u`: **(Required)** The storage URI of the table you want to compact. (e.g., `file:///tmp/hdb`, `s3://my-bucket/table`)
+- `--target-file-size`: **(Optional)** The ideal size for the compacted data files. The CLI accepts human-readable string values like `64MB`, `1GB`, `512K`. The default is `64MB`.
+
+**Example:**
+```bash
+hyperstreamdb-admin compact --uri "s3://production-data/events_table" --target-file-size 128MB
+```
+
+### 2. `vacuum`
+HyperStreamDB relies on Multi-Version Concurrency Control (MVCC) and copy-on-write semantics. As data is updated, deleted, or compacted, old files are unlinked from the active manifest but remain on disk to allow time-travel queries and safe rollback.
+
+The `vacuum` command permanently deletes these unreferenced historical files to reclaim physical storage space. 
+
+> [!WARNING]
+> Running vacuum is irreversible. You will not be able to time-travel to versions older than the retention limit once vacuum completes.
+
+**Usage:**
+```bash
+hyperstreamdb-admin vacuum --uri <TABLE_URI> [OPTIONS]
+```
+
+**Options:**
+- `--uri`, `-u`: **(Required)** The storage URI of the table you want to vacuum.
+- `--retain-versions`: **(Optional)** The number of historical manifest versions to keep. Any unreferenced data file not tied to the kept versions will be permanently deleted. Default is `10`.
+
+**Example:**
+```bash
+hyperstreamdb-admin vacuum --uri "s3://production-data/events_table" --retain-versions 5
+```
+
+## E2E Testing
+The CLI is strictly tested for regressions. You can run the administrative automated bash test locally using:
+```bash
+./tests/verify_admin_cli.sh
+```
+This test spins up a mock dataset, runs a compaction pass, and successfully vacuums the resulting orphaned files.

{hyperstreamdb-0.5.2 → hyperstreamdb-0.5.4}/CHANGELOG.md

@@ -9,6 +9,38 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ---
 
+## [0.5.3] - 2026-06-21
+
+### Added
+- Unified catalog table loading logic (`load_from_catalog`) across Glue, Hive, Nessie, and REST catalog wrappers in python bindings.
+- Explicit commit-time flushing and background synchronization in integration tests (`test_turboquant_integration.py` and `test_wikipedia.py`).
+
+### Changed
+- Fixed remote path routing in `flush_async` to correctly construct relative remote paths for staged files instead of using local staging paths.
+- Upload data files synchronously relative to commit operation to avoid read-after-write race conditions in remote catalogs.
+- Fixed a missing file upload call in the background indexing task for vector indexes.
+
+### Fixed
+- Clippy compiler warnings and errors in HNSW/PQ modules.
+
+---
+
+## [0.5.2] - 2026-06-15
+
+### Changed
+- Bumped version to 0.5.2.
+- Resolved write buffer schema alignment & list array indexing bugs.
+
+---
+
+## [0.5.1] - 2026-06-07
+
+### Changed
+- Optimized vector search.
+- Fixed WAL tests.
+
+---
+
 ## [0.5.0] - 2026-06-03
 
 ### Added
@@ -292,7 +324,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ---
 
-[Unreleased]: https://github.com/rla3rd/hyperstreamdb/compare/v0.5.0...HEAD
+[Unreleased]: https://github.com/rla3rd/hyperstreamdb/compare/v0.5.3...HEAD
+[0.5.3]: https://github.com/rla3rd/hyperstreamdb/compare/v0.5.2...v0.5.3
+[0.5.2]: https://github.com/rla3rd/hyperstreamdb/compare/v0.5.1...v0.5.2
+[0.5.1]: https://github.com/rla3rd/hyperstreamdb/compare/v0.5.0...v0.5.1
 [0.5.0]: https://github.com/rla3rd/hyperstreamdb/compare/v0.4.1...v0.5.0
 [0.4.0]: https://github.com/rla3rd/hyperstreamdb/compare/v0.3.3...v0.4.0
 [0.3.3]: https://github.com/rla3rd/hyperstreamdb/compare/v0.3.2...v0.3.3

{hyperstreamdb-0.5.2 → hyperstreamdb-0.5.4}/CONTRIBUTING.md

@@ -12,7 +12,7 @@ Before contributing, please read [STEERING.md](STEERING.md) for our development
 
 ### Prerequisites
 
-- **Rust** (stable, 1.75+) — install via [rustup](https://rustup.rs/)
+- **Rust** (stable, 1.80+) — install via [rustup](https://rustup.rs/)
 - **Python** (3.10–3.14) — for binding development and testing
 - **Docker** — for integration tests against MinIO and Nessie
 - **Cargo** and **maturin** — for building Python wheels