lithos 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 7741f8dffd9dd5d4d0f710b18fca0a7b1a0d574e893d60c9f5735c36a93954d6
+  data.tar.gz: 6d4eb49a96a6cb44a97587f997905cc77c4e93bd4c33f64579db48754180b7d8
+SHA512:
+  metadata.gz: 6d24765cfbfdc9d6394eacd84081eeba1891f9f8423d2c3665cb7270e417e0cfe4d3e147b79e97607cc1ed6c56d96894805c13a88db2957de37fe350401f60bb
+  data.tar.gz: 4af63db49e563dc16b3411f44a238b86174336b532f1caa6ab478c8e7d68b8282e3ec8d291eeba821177bdda0d0ab654eeaca2a61b4eaf4d735797d40d244883

data/CHANGELOG.md

@@ -0,0 +1,28 @@
+# Changelog
+
+All notable changes to this project are documented here. The format is based on
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
+adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-05-30
+
+### Added
+- `Lithos.open(path, sync:, memtable_bytes:)` with block (auto-close) and
+  non-block forms; `#close`, `#closed?`, `#path`.
+- `#put`/`#[]=`, `#get`/`#[]`, `#delete`, `#key?` (+ `include?`/`has_key?`),
+  `#fetch` (with default/block, raises `KeyError`).
+- Ordered iteration: `#each`, `#each_key`, `#scan(gte:/gt:/lte:/lt:)`,
+  `#range(from, to)`; `Enumerable` mixed in; `#size`/`#length`, `#empty?`, `#to_h`.
+- `#flush` (seal the memtable into an SSTable) and `#compact` (merge SSTables,
+  dropping shadowed keys and tombstones).
+- LSM engine: CRC32 write-ahead log (durable per write in sync mode), sorted
+  memtable, immutable memory-mapped SSTables with bloom filters + sparse index,
+  atomically-published MANIFEST catalog, crash recovery via WAL replay (torn /
+  bad-CRC tail discarded), and an exclusive single-writer directory lock.
+- Arbitrary binary keys and values (embedded NULs); lexicographic unsigned-byte
+  key ordering.
+
+[Unreleased]: https://github.com/main-path/lithos/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/main-path/lithos/releases/tag/v0.1.0

data/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ned
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,101 @@
+# lithos
+
+**A small embedded, ordered, crash-safe key-value store for Ruby — no external database.**
+
+`lithos` is a from-scratch storage engine: a **log-structured merge (LSM) tree**,
+the same design behind LevelDB/RocksDB, written as a native Ruby extension with
+**zero external dependency**. You get:
+
+- **Durable writes** — every `put`/`delete` is appended to a CRC-checked
+  write-ahead log and `fsync`'d before it returns (in the default `sync:` mode).
+- **Crash recovery** — on open, the WAL is replayed; a torn or corrupted tail is
+  detected and discarded.
+- **Ordered keys** — keys are stored sorted (unsigned-byte order), so you get
+  `each`, `each_key`, and range `scan`s for free.
+- **Binary safe** — keys and values are arbitrary byte strings (embedded NULs OK).
+
+The niche it fills: every other ordered + crash-safe Ruby KV store is a binding
+to an external C/C++ library (LMDB/LevelDB/RocksDB) that's a pain to build on a
+native-MSVC Windows Ruby. `lithos` is self-contained and builds with
+[`vcvars`](https://rubygems.org/gems/vcvars) — no prebuilt database needed.
+
+## Requirements
+
+- **Windows** with a native **MSVC (mswin)** Ruby. Not supported on MinGW/UCRT.
+- Visual Studio 2017+ / Build Tools with the **Desktop development with C++** workload.
+
+## Install
+
+```sh
+gem install lithos
+```
+
+## Usage
+
+```ruby
+require "lithos"
+
+Lithos.open("data/mydb") do |db|
+  db["alpha"] = "one"
+  db.put("beta", "two")
+  db.put("\x00raw\xff", "binary ok")     # arbitrary binary keys + values
+
+  db["alpha"]            # => "one"
+  db["missing"]          # => nil
+  db.key?("beta")        # => true
+  db.fetch("nope", "default")            # => "default" (or raises KeyError)
+  db.delete("beta")      # => true (existed)
+
+  # ordered iteration (ascending unsigned-byte key order)
+  db.each      { |k, v| puts "#{k}=#{v}" }
+  db.each_key  { |k|    puts k }
+
+  # ordered range scans
+  db.scan(gte: "a", lt: "m") { |k, v| ... }   # gt/lt exclusive, gte/lte inclusive
+  db.range("a", "m")         { |k, v| ... }    # half-open [a, m)
+  db.scan(gte: "a").to_a                       # Enumerator without a block
+
+  db.size                # live key count
+  db.flush               # seal the memtable into an SSTable
+  db.compact             # merge SSTables, dropping shadowed keys + tombstones
+end                      # closed automatically (flush + fsync)
+```
+
+Open without a block when you want to manage the lifetime yourself:
+
+```ruby
+db = Lithos.open("data/mydb", sync: true)
+db["k"] = "v"
+db.close
+```
+
+Data persists across reopens, and survives a crash (process kill, power loss at
+the OS-flush boundary): reopening replays the WAL and discards any partial tail.
+
+## How it works
+
+```
+put/delete ──> WAL (append + CRC + fsync) ──> memtable (sorted std::map)
+                                                  │  (size threshold)
+                                                  ▼
+                                          SSTable (immutable, mmap'd:
+                                          sorted data + bloom + sparse index)
+get ──> memtable ──> SSTables newest→oldest (bloom-filtered)
+compact ──> k-way merge of all SSTables ──> one SSTable (drops tombstones)
+```
+
+The set of live SSTables + the active WAL is recorded in a `MANIFEST` that is
+rewritten atomically (temp file + `FlushFileBuffers` + atomic rename), so the
+on-disk catalog is never observed half-updated.
+
+## Notes
+
+- **Durability vs speed:** `sync: true` (default) `fsync`s every write. For bulk
+  loads, `Lithos.open(path, sync: false)` skips per-write `fsync` (call `#flush`
+  or `#close` to make data durable) — faster, but a crash can lose recent writes.
+- **Single writer:** a store takes an exclusive directory lock on open; it is not
+  safe to share one store across threads (give each thread its own, or use a Mutex).
+
+## License
+
+[MIT](LICENSE.txt).

data/ext/lithos/extconf.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+#
+# extconf.rb for the lithos LSM key-value store extension.
+
+require "mkmf"
+
+unless RbConfig::CONFIG["target_os"] =~ /mswin/
+  abort <<~MSG
+    lithos requires a native Windows MSVC (mswin) Ruby — its storage engine uses
+    Win32 file APIs (memory-mapped SSTables, FlushFileBuffers, atomic rename) and
+    is built with cl.exe. Your Ruby is "#{RbConfig['arch']}".
+  MSG
+end
+
+# C++ engine (std::map/std::string/std::vector/std::priority_queue) — needs C++ exceptions.
+$CXXFLAGS << " -EHsc"
+
+# kernel32 (file APIs / mapping) is linked by default; no extra import libs needed.
+
+create_makefile("lithos/lithos")