sparkey 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1d02b7853fe52790fde3dd206f8af7febc4c01f8
-  data.tar.gz: 34f41f81cbf31b608fa6faed0ba9b252c1e4b1b8
+  metadata.gz: a41aea2172a4710c84b4992da760068abbdeb439
+  data.tar.gz: 2a325fa579cfca5516681541784f60a9872cd173
 SHA512:
-  metadata.gz: 2719e8fd06e6f6d108f5eab7a856a37fafddfafa712b8daff293315293d458c56709a26e6891b68ffdbefd122affad22b8d6e8a79b498e4bbcece8222591c965
-  data.tar.gz: 81397ce7bf58162b23ad076d2334a1acb63d8064f9cf6bc12da2e71eaa4b3d5ab8fda1c10b84d237437904557c54bd3fee7d529f769694bcb486a647ec0e06a0
+  metadata.gz: c6de97f658c7c89cff32e17078cd8a551d926a98aed2261b2ad56b269a1f7532b2c49c9635e5ea189d526d693839664c2a02ccf90252a6daad499fc2ad55df78
+  data.tar.gz: 2ffeb03f7a7e2dbbf9641f877ce224a04ec36cf2e9cf1eede3bda466a2529f3ad8f0feb48e3a6bb21588962d35ec73e9dff5334bbc340529c1f535548c5425d3

data/CHANGELOG.md

@@ -0,0 +1,43 @@
+# Sparkey 1.1.0 (November 18, 2013)
+* `Sparkey` API
+  * Added `::build_log_filename` to build a log file name from a hash file name.
+* `Sparkey::Store` API
+  * Added `#each_from_log` to explicitly iterate using the log file. Will repeatedly yield the key, value, and the type of log entry to the block.
+  * Added `#each_from_hash` to explicitly iterate using the hash file. Will repeatedly yield the key and value to the block.
+  * Fixed `#get` to return nil if the key is not found.
+  * Fixed `#flush` to also re-open the `Sparkey::LogReader` to ensure it does not receive stale entries if used for iteration.
+  * Changed `#flush` to no longer accept a block.
+* `Sparkey::HashReader` API
+  * Renamed `#size` to `#entry_count`.
+  * Added `#collision_count`.
+* `Sparkey::LogIterator` API
+  * Added `#new?` to check if the iterator is in a new state.
+  * Added `#invalid?` to check if the iterator is in an invalid state.
+  * Added `#closed?` to check if the iterator is in a closed state.
+  * Added `#entry_put?` to check if the iterator log entry type is a put entry.
+  * Added `#entry_delete?` to check if the iterator log entry type is a delete entry.
+  * Added `#skip` to skip over a specified number of entries in a log file.
+  * Added `#reset` to reset the iterator back to the beginning.
+  * Added `#get_key_chunk` to enable retrieving chunks of the key without
+    having to store the entire key in memory. Will repeatedly yield to the
+    block the key chunk in the requested block size in bytes until the key has
+    been fully consumed. Block sizes must be at least 8 bytes.
+  * Added `#get_value_chunk` to enable retrieving chunks of the value without
+    having to store the entire value in memory. Will repeatedly yield to the
+    block the value chunk in the requested block size in bytes until the value has
+    been fully consumed. Block sizes must be at least 8 bytes.
+  * Fixed `#get_key` to read the maximum key length when consuming the key.
+  * Fixed `#get_value` to read the maximum value length when consuming the value.
+* `Sparkey::HashIterator` API
+  * Added API to specifically deal with iterating through a hash reader.
+  * Supports all existing `Sparkey::LogIterator` API methods.
+  * Implement `#next` using the `sparkey_logiter_hashnext` function.
+
+# Sparkey 1.0.0 (November 16, 2013)
+* `Sparkey::Store` API
+* `Sparkey::LogReader` API
+* `Sparkey::LogWriter` API
+* `Sparkey::LogIterator` API
+* `Sparkey::HashReader` API
+* `Sparkey::HashWriter` API
+* `Sparkey::Native` API

data/README.md

@@ -1,9 +1,12 @@
 # Sparkey
 
-TODO: Write a gem description
+Ruby [FFI](https://github.com/ffi/ffi) bindings to Spotify's [Sparkey](https://github.com/spotify/sparkey) key-value store.
 
 ## Installation
 
+### Requirements
+* `libsparkey` ([Github](https://github.com/spotify/sparkey))
+
 Add this line to your application's Gemfile:
 
     gem 'sparkey'
@@ -16,14 +19,73 @@ Or install it yourself as:
 
     $ gem install sparkey
 
+## Design
+This gem aims to expose all of Sparkey's native functionality to Ruby via [FFI](https://github.com/ffi/ffi) bindings.
+
+Additionally, it provides higher-level abstractions around the native functionality to provide a more idiomatic Ruby interface.
+
+## Low Level
+`Sparkey::Native` is intended to expose the raw C functions from Sparkey to Ruby via FFI and nothing more.
+
+Use this interface if you are adding Ruby classes and methods to expose new Sparkey functionality. A solid understanding of the [FFI](https://github.com/ffi/ffi) API will be required.
+
+`Sparkey::Native` should wrap all available functions from [sparkey.h](https://github.com/spotify/sparkey/blob/master/src/sparkey.h). If you find one missing please submit a Pull Request.
+
+## Mid Level
+This gem exposes Ruby-ish versions of most of Sparkey's public data structures and their related functions.
+
+Use these interfaces if you need more control over the specific behavior of Sparkey than the `Sparkey::Store` API provides.
+
+## High Level
+The `Sparkey::Store` API provides a very small interface for using Sparkey as a generic key-value store.
+
+Use the `Sparkey::Store` API if you only need a fast persistent key-value store and don't want to delve into Sparkey specifics.
+
 ## Usage
+```ruby
+  require "sparkey"
+
+  # Creates or overwrites the Sparkey file.
+  sparkey = Sparkey.create("sparkey_store")
+
+  # Opens an existing Sparkey file.
+  # sparkey = Sparkey.open("sparkey_store")
+
+  sparkey.put("first", "Hello")
+  sparkey.put("second", "World")
+  sparkey.put("third", "Goodbye")
+  sparkey.flush
+
+  puts sparkey.size
+  #=> 3
+
+  puts sparkey.get("second")
+  #=> "World"
+
+  sparkey.put("fourth", "Again")
+  sparkey.delete("second")
+  sparkey.delete("third")
+  sparkey.flush
+
+  puts sparkey.size
+  #=> 2
+
+  collector = Hash.new
+  sparkey.each do |key, value|
+    collector[key] = value
+  end
+
+  sparkey.close
 
-TODO: Write usage instructions here
+  puts collector
+  #=> { "first" => "Hello", "fourth" => "Again" }
+```
 
 ## Contributing
 
 1. Fork it
 2. Create your feature branch (`git checkout -b my-new-feature`)
 3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
+4. Ensure all tests pass (`rake test`)
+5. Push to the branch (`git push origin my-new-feature`)
+6. Create new Pull Request

data/lib/sparkey.rb

@@ -8,6 +8,10 @@ module Sparkey
   def self.open(filename)
     Store.open(filename)
   end
+
+  def self.build_log_filename(hash_file)
+    Sparkey::Native.create_log_filename(hash_file)
+  end
 end
 
 require "sparkey/native"
@@ -18,3 +22,4 @@ require "sparkey/log_reader"
 require "sparkey/log_iterator"
 require "sparkey/hash_writer"
 require "sparkey/hash_reader"
+require "sparkey/hash_iterator"

data/lib/sparkey/hash_iterator.rb

@@ -0,0 +1,11 @@
+class Sparkey::HashIterator < Sparkey::LogIterator
+  def initialize(hash_reader)
+    @hash_reader = hash_reader
+
+    super @hash_reader.log_reader
+  end
+
+  def next
+    handle_status Sparkey::Native.logiter_hashnext(@log_iter_ptr, @hash_reader.ptr)
+  end
+end

data/lib/sparkey/hash_reader.rb

@@ -8,32 +8,34 @@ class Sparkey::HashReader
 
     handle_status Sparkey::Native.hash_open(ptr, hash_filename, log_filename)
 
-    @hash_reader_ptr = ptr.get_pointer(0)
+    @hash_reader_ptr = ptr.read_pointer
   end
 
   def close
-    ptr = FFI::MemoryPointer.new(:pointer)
-    ptr.put_pointer(0, @hash_reader_ptr)
+    ptr = FFI::MemoryPointer.new(:pointer).write_pointer(@hash_reader_ptr)
 
     Sparkey::Native.hash_close(ptr)
   end
 
   def seek(key)
-    iterator = Sparkey::LogIterator.new(log_reader)
+    iterator = Sparkey::HashIterator.new(self)
 
-    key_length = key.size
-    key_ptr = FFI::MemoryPointer.new(:uint8, key_length)
-    key_ptr.put_bytes(0, key)
+    key_length = key.bytesize
+    key_ptr = FFI::MemoryPointer.new(:uint8, key_length).write_bytes(key)
 
     handle_status Sparkey::Native.hash_get(@hash_reader_ptr, key_ptr, key_length, iterator.ptr)
 
     iterator
   end
 
-  def size
+  def entry_count
     Sparkey::Native.hash_numentries(@hash_reader_ptr)
   end
 
+  def collision_count
+    Sparkey::Native.hash_numcollisions(@hash_reader_ptr)
+  end
+
   def log_reader
     reader_ptr = Sparkey::Native.hash_getreader(@hash_reader_ptr)
 

data/lib/sparkey/log_iterator.rb

@@ -1,23 +1,26 @@
 class Sparkey::LogIterator
   include Sparkey::Errors
 
-  def initialize(log_reader, hash_reader = nil)
+  def initialize(log_reader)
     @log_reader = log_reader
-    @hash_reader = hash_reader
 
     ptr = FFI::MemoryPointer.new(:pointer)
 
     handle_status Sparkey::Native.logiter_create(ptr, @log_reader.ptr)
 
-    @log_iter_ptr = ptr.get_pointer(0)
+    @log_iter_ptr = ptr.read_pointer
   end
 
   def next
     handle_status Sparkey::Native.logiter_next(@log_iter_ptr, @log_reader.ptr)
   end
 
-  def hash_next
-    handle_status Sparkey::Native.logiter_hashnext(@log_iter_ptr, @hash_reader.ptr)
+  def skip(count)
+    handle_status Sparkey::Native.logiter_skip(@log_iter_ptr, @log_reader.ptr, count)
+  end
+
+  def reset
+    handle_status Sparkey::Native.logiter_reset(@log_iter_ptr, @log_reader.ptr)
   end
 
   def state
@@ -29,17 +32,37 @@ class Sparkey::LogIterator
   end
 
   def <=>(iterator)
-    ptr = FFI::MemoryPointer.new(:int, 1)
+    ptr = FFI::MemoryPointer.new(:int)
 
-    handle_status Sparkey::Native.logiter_keycmp(@log_iter_ptr, interator.ptr, @log_reader.ptr, ptr)
+    handle_status Sparkey::Native.logiter_keycmp(@log_iter_ptr, iterator.ptr, @log_reader.ptr, ptr)
 
     ptr.read_int
   end
 
+  def new?
+    state == :iter_new
+  end
+
   def active?
     state == :iter_active
   end
 
+  def invalid?
+    state == :iter_invalid
+  end
+
+  def closed?
+    state == :iter_closed
+  end
+
+  def entry_put?
+    type == :entry_put
+  end
+
+  def entry_delete?
+    type == :entry_delete
+  end
+
   def key_length
     Sparkey::Native.logiter_keylen(@log_iter_ptr)
   end
@@ -49,28 +72,57 @@ class Sparkey::LogIterator
   end
 
   def get_key
-    wanted_key_length = key_length
-    key_ptr = FFI::MemoryPointer.new(:uint8, wanted_key_length)
-    actual_key_length_ptr = FFI::MemoryPointer.new(:uint64, 1)
+    max_key_length = @log_reader.max_key_length
+    buffer_ptr = FFI::MemoryPointer.new(:uint8, max_key_length)
+    buffer_length_ptr = FFI::MemoryPointer.new(:uint64)
 
-    handle_status Sparkey::Native.logiter_fill_key(@log_iter_ptr, @log_reader.ptr, wanted_key_length, key_ptr, actual_key_length_ptr)
+    handle_status Sparkey::Native.logiter_fill_key(@log_iter_ptr, @log_reader.ptr, max_key_length, buffer_ptr, buffer_length_ptr)
 
-    key_ptr.read_bytes(actual_key_length_ptr.read_uint64)
+    buffer_ptr.read_bytes(buffer_length_ptr.read_uint64)
+  end
+
+  def get_key_chunk(chunk_size = 1024)
+    buffer = FFI::Buffer.alloc_out(:uint8, chunk_size)
+    buffer_length_ptr = FFI::MemoryPointer.new(:uint64)
+
+    loop do
+      handle_status Sparkey::Native.logiter_keychunk(@log_iter_ptr, @log_reader.ptr, chunk_size, buffer, buffer_length_ptr)
+
+      buffer_length = buffer_length_ptr.read_uint64
+
+      break if buffer_length.zero?
+
+      yield buffer.read_pointer.read_bytes(buffer_length)
+    end
   end
 
   def get_value
-    wanted_value_length = value_length
-    value_ptr = FFI::MemoryPointer.new(:uint8, wanted_value_length)
-    actual_value_length_ptr = FFI::MemoryPointer.new(:uint64, 1)
+    max_value_length = @log_reader.max_value_length
+    buffer_ptr = FFI::MemoryPointer.new(:uint8, max_value_length)
+    buffer_length_ptr = FFI::MemoryPointer.new(:uint64)
+
+    handle_status Sparkey::Native.logiter_fill_value(@log_iter_ptr, @log_reader.ptr, max_value_length, buffer_ptr, buffer_length_ptr)
 
-    handle_status Sparkey::Native.logiter_fill_value(@log_iter_ptr, @log_reader.ptr, wanted_value_length, value_ptr, actual_value_length_ptr)
+    buffer_ptr.read_bytes(buffer_length_ptr.read_uint64)
+  end
+
+  def get_value_chunk(chunk_size = 1024)
+    buffer = FFI::Buffer.alloc_out(:uint8, chunk_size)
+    buffer_length_ptr = FFI::MemoryPointer.new(:uint64)
+
+    loop do
+      handle_status Sparkey::Native.logiter_valuechunk(@log_iter_ptr, @log_reader.ptr, chunk_size, buffer, buffer_length_ptr)
 
-    value_ptr.read_bytes(actual_value_length_ptr.read_uint64)
+      buffer_length = buffer_length_ptr.read_uint64
+
+      break if buffer_length.zero?
+
+      yield buffer.read_pointer.read_bytes(buffer_length)
+    end
   end
 
   def close
-    ptr = FFI::MemoryPointer.new(:pointer)
-    ptr.put_pointer(0, @log_iter_ptr)
+    ptr = FFI::MemoryPointer.new(:pointer).write_pointer(@log_iter_ptr)
 
     Sparkey::Native.logiter_close(ptr)
   end

data/lib/sparkey/log_reader.rb

@@ -7,12 +7,11 @@ class Sparkey::LogReader
 
     handle_status Sparkey::Native.logreader_open(ptr, log_filename)
 
-    @log_reader_ptr = ptr.get_pointer(0)
+    @log_reader_ptr = ptr.read_pointer
   end
 
   def close
-    ptr = FFI::MemoryPointer.new(:pointer)
-    ptr.put_pointer(0, @log_reader_ptr)
+    ptr = FFI::MemoryPointer.new(:pointer).write_pointer(@log_reader_ptr)
 
     Sparkey::Native.logreader_close(ptr)
   end

data/lib/sparkey/log_writer.rb

@@ -7,7 +7,7 @@ class Sparkey::LogWriter
 
     handle_status Sparkey::Native.logwriter_create(ptr, log_filename, compression, block_size)
 
-    @log_writer_ptr = ptr.get_pointer(0)
+    @log_writer_ptr = ptr.read_pointer
   end
 
   def open(filename)
@@ -15,25 +15,22 @@ class Sparkey::LogWriter
     log_filename = "#{filename}.spl"
 
     handle_status Sparkey::Native.logwriter_append(ptr, log_filename)
-    @log_writer_ptr = ptr.get_pointer(0)
+    @log_writer_ptr = ptr.read_pointer
   end
 
   def put(key, value)
     key_length = key.bytesize
-    key_ptr = FFI::MemoryPointer.new(:uint8, key_length)
-    key_ptr.put_bytes(0, key)
+    key_ptr = FFI::MemoryPointer.new(:uint8, key_length).write_bytes(key)
 
     value_length = value.bytesize
-    value_ptr = FFI::MemoryPointer.new(:uint8, value_length)
-    value_ptr.put_bytes(0, value)
+    value_ptr = FFI::MemoryPointer.new(:uint8, value_length).write_bytes(value)
 
     handle_status Sparkey::Native.logwriter_put(@log_writer_ptr, key_length, key_ptr, value_length, value_ptr)
   end
 
   def delete(key)
     key_length = key.bytesize
-    key_ptr = FFI::MemoryPointer.new(:uint8, key_length)
-    key_ptr.put_bytes(0, key)
+    key_ptr = FFI::MemoryPointer.new(:uint8, key_length).write_bytes(key)
 
     handle_status Sparkey::Native.logwriter_delete(@log_writer_ptr, key_length, key_ptr)
   end
@@ -43,8 +40,7 @@ class Sparkey::LogWriter
   end
 
   def close
-    ptr = FFI::MemoryPointer.new(:pointer)
-    ptr.put_pointer(0, @log_writer_ptr)
+    ptr = FFI::MemoryPointer.new(:pointer).write_pointer(@log_writer_ptr)
 
     Sparkey::Native.logwriter_close(ptr)
   end

data/lib/sparkey/store.rb

@@ -49,41 +49,31 @@ class Sparkey::Store
   end
 
   def size
-    hash_reader.size
+    hash_reader.entry_count
   end
 
   def get(key)
     iterator = hash_reader.seek(key)
 
+    return unless iterator.active?
+
     iterator.get_value
   end
 
-  def each_from_hash
-    log_reader = hash_reader.log_reader
-    iterator = Sparkey::LogIterator.new(log_reader, hash_reader)
-
-    loop do
-      iterator.hash_next
-
-      break unless iterator.active?
+  def each_from_hash(&block)
+    iterator = Sparkey::HashIterator.new(hash_reader)
+    typeless_block = ->(k, v, _){ block.call(k, v) }
 
-      yield iterator.get_key, iterator.get_value
-    end
+    each_with_iterator(iterator, &typeless_block)
 
     iterator.close
   end
   alias_method :each, :each_from_hash
 
-  def each_from_log
+  def each_from_log(&block)
     iterator = Sparkey::LogIterator.new(log_reader)
 
-    loop do
-      iterator.next
-
-      break unless iterator.active?
-
-      yield iterator.get_key, iterator.get_value
-    end
+    each_with_iterator(iterator, &block)
 
     iterator.close
   end
@@ -97,12 +87,22 @@ class Sparkey::Store
   end
 
   def flush
-    yield self if block_given?
-
     log_writer.flush
 
-    # Reset the hash headers
+    # Reset to flush cached headers
+    log_reader.open(filename)
     hash_writer.create(filename)
     hash_reader.open(filename)
   end
+
+  private
+  def each_with_iterator(iterator)
+    loop do
+      iterator.next
+
+      break unless iterator.active?
+
+      yield iterator.get_key, iterator.get_value, iterator.type
+    end
+  end
 end

data/sparkey.gemspec

@@ -4,7 +4,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 Gem::Specification.new do |spec|
   spec.name          = "sparkey"
-  spec.version       = "1.0.0"
+  spec.version       = "1.1.0"
   spec.authors       = ["Adam Tanner"]
   spec.email         = ["adam@adamtanner.org"]
   spec.description   = %{ Ruby FFI bindings for Spotify's Sparkey }
@@ -20,4 +20,5 @@ Gem::Specification.new do |spec|
   spec.add_dependency "ffi"
   spec.add_development_dependency "bundler", "~> 1.3"
   spec.add_development_dependency "rake"
+  spec.add_development_dependency "m"
 end

data/spec/sparkey/store_spec.rb

@@ -0,0 +1,88 @@
+require "minitest/autorun"
+require "sparkey"
+require "sparkey/testing"
+
+describe Sparkey::Store do
+  include Sparkey::Testing
+
+  before { @filename = random_filename }
+  after  { delete(@filename) }
+
+  it "creates a Sparkey log file" do
+    sparkey = Sparkey::Store.create(@filename, :compression_snappy, 100)
+
+    File.exists?("#{@filename}.spl").must_equal(true)
+  end
+
+  it "sets values to keys" do
+    sparkey = Sparkey.create(@filename)
+
+    sparkey.put("first", "Michael")
+    sparkey.flush
+
+    sparkey.get("first").must_equal("Michael")
+  end
+
+  it "deletes keys" do
+    sparkey = Sparkey.create(@filename)
+    sparkey.put("first", "Michael")
+    sparkey.put("middle", "Adam")
+    sparkey.flush
+
+    sparkey.delete("first")
+    sparkey.flush
+
+    sparkey.get("first").must_be_nil
+  end
+
+  it "has the size" do
+    sparkey = Sparkey.create(@filename)
+    sparkey.put("middle", "Adam")
+    sparkey.put("last", "Tanner")
+    sparkey.flush
+
+    sparkey.size.must_equal(2)
+  end
+
+  it "supports iterating through the log file" do
+    sparkey = Sparkey.create(@filename)
+    sparkey.put("first", "Michael")
+    sparkey.put("middle", "Adam")
+    sparkey.put("last", "Tanner")
+    sparkey.delete("middle")
+    sparkey.flush
+
+    collector = []
+
+    sparkey.each_from_log do |key, value, type|
+      collector << [key, value, type]
+    end
+
+    collector.must_equal([
+      ["first",  "Michael", :entry_put],
+      ["middle", "Adam",    :entry_put],
+      ["last",   "Tanner",  :entry_put],
+      ["middle", "",        :entry_delete]
+    ])
+  end
+
+  it "supports iterating through the hash file" do
+    sparkey = Sparkey.create(@filename)
+    sparkey.put("first", "Michael")
+    sparkey.put("middle", "Adam")
+    sparkey.put("last", "Tanner")
+    sparkey.delete("middle")
+    sparkey.flush
+
+    collector = []
+
+    sparkey.each_from_hash do |key, value|
+      collector << [key, value]
+    end
+
+    collector.must_equal([
+      ["first", "Michael"],
+      ["last",  "Tanner"]
+    ])
+  end
+end

data/spec/sparkey_spec.rb

@@ -8,31 +8,210 @@ describe Sparkey do
   before { @filename = random_filename }
   after  { delete(@filename) }
 
-  it "functions as a key value store" do
-    sparkey = Sparkey::Store.create(@filename, :compression_snappy, 1000)
-    sparkey.put("first", "Michael")
-    sparkey.put("second", "Adam")
-    sparkey.put("third", "Tanner")
-    sparkey.close
+  it "assigns values to keys" do
+    log_writer = Sparkey::LogWriter.new
+    log_writer.create(@filename, :compression_none, 100)
 
-    sparkey = Sparkey::Store.open(@filename)
+    log_writer.put("first", "Michael")
+    log_writer.flush
 
-    sparkey.size.must_equal 3
+    hash_writer = Sparkey::HashWriter.new
+    hash_writer.create(@filename)
 
-    sparkey.get("first").must_equal("Michael")
-    sparkey.delete("second")
-    sparkey.flush
+    hash_reader = Sparkey::HashReader.new
+    hash_reader.open(@filename)
+    iterator = hash_reader.seek("first")
 
-    sparkey.size.must_equal 2
+    iterator.get_value.must_equal("Michael")
 
-    hash = Hash.new
-    sparkey.each do |key, value|
-      hash[key] = value
+    iterator.close
+    hash_reader.close
+    log_writer.close
+  end
+
+  it "deletes keys" do
+    log_writer = Sparkey::LogWriter.new
+    log_writer.create(@filename, :compression_none, 100)
+    log_writer.put("first", "Michael")
+    log_writer.flush
+
+    log_writer.delete("first")
+    log_writer.flush
+
+    hash_writer = Sparkey::HashWriter.new
+    hash_writer.create(@filename)
+
+    hash_reader = Sparkey::HashReader.new
+    hash_reader.open(@filename)
+    iterator = hash_reader.seek("first")
+
+    iterator.must_be(:invalid?)
+    hash_reader.entry_count.must_equal(0)
+
+    iterator.close
+    hash_reader.close
+    log_writer.close
+  end
+
+  it "has the max key length and max value length" do
+    log_writer = Sparkey::LogWriter.new
+    log_writer.create(@filename, :compression_none, 100)
+    log_writer.put("middle", "Adam")
+    log_writer.put("last", "Tanner")
+    log_writer.flush
+
+    log_reader = Sparkey::LogReader.new
+    log_reader.open(@filename)
+
+    log_reader.max_key_length.must_equal(6)
+    log_reader.max_value_length.must_equal(6)
+
+    log_reader.close
+    log_writer.close
+  end
+
+  it "builds a log filename from a hash filename" do
+    Sparkey.build_log_filename("sparkey.spi").must_equal("sparkey.spl")
+  end
+
+  it "iterates over the log file" do
+    log_writer = Sparkey::LogWriter.new
+    log_writer.create(@filename, :compression_none, 100)
+
+    log_writer.put("first", "Michael")
+    log_writer.put("middle initial", "A.")
+    log_writer.put("last", "Tanner")
+    log_writer.delete("middle initial")
+    log_writer.flush
+
+    log_reader = Sparkey::LogReader.new
+    log_reader.open(@filename)
+
+    log_iterator = Sparkey::LogIterator.new(log_reader)
+
+    log_iterator.must_be(:new?)
+
+    log_iterator.next
+
+    log_iterator.must_be(:active?)
+    log_iterator.must_be(:entry_put?)
+
+    log_iterator.key_length.must_equal(5)
+    log_iterator.value_length.must_equal(7)
+
+    log_iterator.get_key.must_equal("first")
+    log_iterator.get_value.must_equal("Michael")
+
+    log_iterator.next
+
+    key, iterations = "", 0
+    log_iterator.get_key_chunk(8) do |chunk|
+      iterations += 1
+      key << chunk
+    end
+
+    key.must_equal("middle initial")
+    iterations.must_equal(2)
+
+    value, iterations = "", 0
+    log_iterator.get_value_chunk(8) do |chunk|
+      iterations += 1
+      value << chunk
     end
 
-    hash.must_equal(
-      "first" => "Michael",
-      "third" => "Tanner"
-    )
+    value.must_equal("A.")
+    iterations.must_equal(1)
+
+    log_iterator.skip(4)
+    log_iterator.must_be(:entry_delete?)
+
+    log_iterator.next
+    log_iterator.must_be(:closed?)
+
+    log_iterator.close
+    log_reader.close
+    log_writer.close
+  end
+
+  it "iterates over the hash file" do
+    log_writer = Sparkey::LogWriter.new
+    log_writer.create(@filename, :compression_none, 100)
+
+    log_writer.put("salutation", "Mr.")
+    log_writer.put("first", "Michael")
+    log_writer.put("middle", "Adam")
+    log_writer.put("last", "Tanner")
+    log_writer.delete("first")
+    log_writer.flush
+
+    hash_writer = Sparkey::HashWriter.new
+    hash_writer.create(@filename)
+
+    hash_reader = Sparkey::HashReader.new
+    hash_reader.open(@filename)
+
+    hash_reader.entry_count.must_equal(3)
+    hash_reader.collision_count.must_equal(0)
+
+    hash_iterator = Sparkey::HashIterator.new(hash_reader)
+
+    hash_iterator.must_be(:new?)
+
+    hash_iterator.next
+    hash_iterator.must_be(:active?)
+
+    hash_iterator.get_key.must_equal("salutation")
+    hash_iterator.get_value.must_equal("Mr.")
+
+    hash_iterator.next
+    hash_iterator.get_key.must_equal("middle")
+    hash_iterator.get_value.must_equal("Adam")
+
+    seek_iterator = hash_reader.seek("last")
+    seek_iterator.get_value.must_equal("Tanner")
+
+    seek_iterator.next
+    seek_iterator.must_be(:closed?)
+
+    invalid_iterator = hash_reader.seek("first")
+    invalid_iterator.must_be(:invalid?)
+
+    invalid_iterator.close
+    seek_iterator.close
+    hash_iterator.close
+    hash_reader.close
+    log_writer.close
+  end
+
+  it "compares iterators" do
+    log_writer = Sparkey::LogWriter.new
+    log_writer.create(@filename, :compression_none, 100)
+
+    log_writer.put("first", "Michael")
+    log_writer.put("middle", "Adam")
+    log_writer.put("last", "Tanner")
+    log_writer.flush
+
+    log_reader = Sparkey::LogReader.new
+    log_reader.open(@filename)
+
+    first_iterator = Sparkey::LogIterator.new(log_reader)
+    second_iterator = Sparkey::LogIterator.new(log_reader)
+
+    first_iterator.next
+    second_iterator.next
+
+    comparison = first_iterator <=> second_iterator
+    comparison.must_equal(0)
+
+    first_iterator.next
+    comparison = first_iterator <=> second_iterator
+    comparison.must_equal(1)
+
+    first_iterator.reset
+    first_iterator.next
+    second_iterator.next
+    comparison = first_iterator <=> second_iterator
+    comparison.must_equal(-1)
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: sparkey
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Adam Tanner
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-11-17 00:00:00.000000000 Z
+date: 2013-11-19 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: ffi
@@ -52,6 +52,20 @@ dependencies:
     - - '>='
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: m
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - '>='
+      - !ruby/object:Gem::Version
+        version: '0'
 description: ' Ruby FFI bindings for Spotify''s Sparkey '
 email:
 - adam@adamtanner.org
@@ -60,12 +74,14 @@ extensions: []
 extra_rdoc_files: []
 files:
 - .gitignore
+- CHANGELOG.md
 - Gemfile
 - LICENSE
 - README.md
 - Rakefile
 - lib/sparkey.rb
 - lib/sparkey/errors.rb
+- lib/sparkey/hash_iterator.rb
 - lib/sparkey/hash_reader.rb
 - lib/sparkey/hash_writer.rb
 - lib/sparkey/log_iterator.rb
@@ -75,6 +91,7 @@ files:
 - lib/sparkey/store.rb
 - lib/sparkey/testing.rb
 - sparkey.gemspec
+- spec/sparkey/store_spec.rb
 - spec/sparkey_spec.rb
 homepage: https://github.com/adamtanner/sparkey
 licenses:
@@ -101,4 +118,5 @@ signing_key:
 specification_version: 4
 summary: Ruby FFI bindings for Spotify's Sparkey
 test_files:
+- spec/sparkey/store_spec.rb
 - spec/sparkey_spec.rb