carton_db 1.1.3 → 1.1.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b0569eb04a7231087ac8ba262cc56330a1c61912
-  data.tar.gz: aaae82f45cfdeb14e4dea5110e9a09c3b062ab5e
+  metadata.gz: ce6b3966ce215081d2036a69277b65cd3958d0ff
+  data.tar.gz: 62f46c6218f8a938a60d6a7b67a0d7314b07f43e
 SHA512:
-  metadata.gz: e07e734b21b74e9988654549060f615897fb9b5e57763a61d392bf58ef0ff46845ba43118c805766f9177a117d0c4f7d49d1fa52e13e8e2042f410dd385b18a3
-  data.tar.gz: 5c818326762c092a868e9abc642ce1baa3a1b9a2fbe38a17c7932f0ebbbde99a1e0a8d72cd7479a98e61a5c97cc1516968f92e9c655cb0e1758333f02a778b2d
+  metadata.gz: 61c8a47573f2f015934f797309381c250cf7cd96ab00a2a244c6c0ae287c3b75bc90a36c2643b76ecd071ac6103c0c39403d7d7a3e0ded63f4e6836a0244be83
+  data.tar.gz: d8ceaa939dcc182dced7fc4e41e896419eac34625598c68a6ddd7b2007b66ca7a12e1be3136aa0cdb1309c115b177620888df457735045af3e9ad426d56c1c67

data/README.md

@@ -7,28 +7,17 @@ The primary goals of this library are simplicity of implementation
 and reliable, predictable behavior when used as intended, along
 with documentation making it reasonably clear what is intended.
 
+Secondarily, this library is optimized for fast appending to
+existing entries.
+
 ## Uses
 
-Uses for this gem seem pretty limited, but you might have a
-purpose for it that the author has not thought about.
-
-This is a formalization of a solution that was created to solve
-a specific problem. The problem was adding a feature to a
-Ruby program running on Heroku to collect data into a map of
-sets of elements that would be too large to be effectively
-handled in memory. The application didn't already have any use
-for a relational database server, and I didn't want to add one
-just for this requirement. A Redis db with sufficient capacity
-would have been expensive, and solutions such as SQLite are
-specifically not supported by Heroku so people don't mistakenly
-expect the data to be preserved. Ruby's `PStore`, `DBM` and
-`SDMB` each proved to be too unpredicatable and flakey to be a
-practical solution.
-
-Although this tool was initially developed to store transient
-data for use within a single process invocation and then
-discarded, it is also well suited for long term data storage on a
-system that retains filesystem contents.
+This library is useful in some of the same situations in which
+one might consider using the `PStore`, `DBM`, or `SMDB` classes
+provided as part of Ruby's standard library, but you either need
+something more solid and trustworthy or you need something that
+supports fast appending of elements to lists or sets within
+entries.
 
 ## Installation
 
@@ -53,13 +42,13 @@ filesystem containing the files that store the data.
 
 A database is accessed through an instance of a database class.
 
-An instance of a database class maintains no internal state
-between calls to its methods except for the database name and the
-expectation of a directory with that name existing in the
-filesystem.
+An instance of a database class assumes nothing about the state
+of the database between calls to its methods, and only expects
+that the database exists and is valid when a call is made that
+reads or writes data.
 
-Only instances of classes maintain any internal state. No global
-internal state is maintained.
+Only instances of database classes maintain any internal state.
+No global internal state is maintained.
 
 An empty directory is a valid empty database.
 
@@ -76,11 +65,14 @@ will be raised if it doesn't.
 
 The database structure is designed to effectively handle up to
 several million elements with any entry containing up to around
-50 thousand characters (elements ⨉ chars in an entry's content).
-
-The speed of database operations is good, but this is not a high
-performance database management system. See the code
-documentation in the classes for more details about the
+50 thousand characters (in all of the entry's elements combined).
+
+The speed of database operations is good, and it is particularly
+optimized for appending to new or existing entries. It was not
+designed or optimized to be a "high performance" database
+management system though, and it has not been benchmarked against
+other systems for specific performance comparison. See the inline
+code documentation of the classes for details about the
 performance of each kind of database operation.
 
 ## Usage
@@ -89,7 +81,8 @@ The primary kind of database provided by this gem is the one
 implemented by `CartonDB::ListMapDb`. It is a map of lists where
 each entry has a string for a key and a list of of 0 or more
 string elements as content. Other kinds of database are
-implemented on top of that and share the same storage format.
+implemented as specializations of that and share the same storage
+format.
 
 The name of the database is the path of a directory in the
 filesystem that either already exists or shall be created as

data/lib/carton_db.rb

@@ -9,4 +9,8 @@ require "carton_db/simple_map_db"
 require "carton_db/set_map_db"
 
 module CartonDb
+  Error = Class.new(StandardError)
+  UnescapingError = Class.new(Error)
+  InvalidEscapeSequence = Class.new(UnescapingError)
+  IncompleteEscapeSequence = Class.new(UnescapingError)
 end

data/lib/carton_db/escaping.rb

@@ -44,18 +44,36 @@ module CartonDb
 
     UNESCAPING_MAP = ESCAPING_MAP.invert.freeze
 
-    def self.escape(value)
-      value.gsub(
-        /[\x00-\x1F\x7F\\]/,
-        ESCAPING_MAP
-      )
-    end
+    class << self
+      def escape(value)
+        value.gsub(
+          /[\x00-\x1F\x7F\\]/,
+          ESCAPING_MAP
+        )
+      end
+
+      def unescape(esc)
+        esc.gsub( /\\(?:\\|x[01][0-9A-F]|x7F|[^x\\]|$)/ ) { |match|
+          UNESCAPING_MAP.fetch match do
+            incomplete_sequence! match if match == "\\"
+            invalid_sequence! match
+          end
+        }
+      end
+
+      private
+
+      def incomplete_sequence!(sequence)
+        message =
+          "Escaped text contains incomplete escape sequence %s" % sequence
+        raise CartonDb::IncompleteEscapeSequence, message
+      end
 
-    def self.unescape(esc)
-      esc.gsub(
-        /\\(?:\\|x[01][0-9A-F]|x7F|[abtnvfr])/,
-        UNESCAPING_MAP
-      )
+      def invalid_sequence!(sequence)
+        message =
+          "Escaped text contains invalid escape sequence %s" % sequence
+        raise CartonDb::InvalidEscapeSequence, message
+      end
     end
 
   end

data/lib/carton_db/list_map_db.rb

@@ -217,10 +217,8 @@ module CartonDb
       segment = segment_containing(key_d)
       return if segment.empty?
 
-      segment.replace do |replacement|
-        replacement.open_overwrite do |repl_io|
-          segment.copy_entries_except key_d, repl_io
-        end
+      segment.replace do |repl_io|
+        segment.copy_entries_except key_d, repl_io
       end
     end
 
@@ -325,13 +323,11 @@ module CartonDb
     attr_accessor :name
 
     def replace_entry_in_file(segment, key_d, content)
-      segment.replace do |replacement|
-        replacement.open_overwrite do |repl_io|
-          segment.copy_entries_except key_d, repl_io
-          element_count = 0
-          count = write_key_elements(key_d, content, repl_io)
-          repl_io << "#{key_d.escaped}\n" if count.zero?
-        end
+      segment.replace do |repl_io|
+        segment.copy_entries_except key_d, repl_io
+        element_count = 0
+        count = write_key_elements(key_d, content, repl_io)
+        repl_io << "#{key_d.escaped}\n" if count.zero?
       end
     end
 

data/lib/carton_db/list_map_db/segment.rb

@@ -67,11 +67,7 @@ module CartonDb
       end
 
       def touch_d(key_d, optimization)
-        if optimization == :small && content?
-          each_entry_element_line do |kd, _ed, _line|
-            return if kd == key_d
-          end
-        end
+        return if optimization == :small && key_d?(key_d)
 
         open_append do |io|
           io << key_d.escaped << "\n"
@@ -102,8 +98,7 @@ module CartonDb
 
       def collect_content(key_d, collection_class)
         result = nil
-        each_entry_element_line do |kd, ed, _line|
-          next unless kd == key_d
+        each_element_for_d key_d do |ed|
           result ||= collection_class.new
           result << ed.plain unless ed.placeholder?
         end
@@ -111,12 +106,7 @@ module CartonDb
       end
 
       def each_entry
-        entries = nil
-        each_entry_element_line do |key_d, elem_d, _line|
-          entries ||= {}
-          content = entries[key_d] ||= []
-          content << elem_d.plain unless elem_d.placeholder?
-        end
+        entries = key_d_contents_map
         return unless entries
         entries.each do |key_d, content|
           yield key_d.plain, content
@@ -131,13 +121,10 @@ module CartonDb
       end
 
       def each_first_element
-        first_entries = nil
-        each_entry_element_line do |key_d, elem_d, _line|
-          first_entries ||= {}
-          first_entries[key_d] ||= elem_d.plain
-        end
-        return unless first_entries
-        first_entries.each do |key_d, element|
+        first_element_map = key_d_first_element_map
+        return unless first_element_map
+
+        first_element_map.each do |key_d, element|
           yield key_d.plain, element
         end
       end
@@ -154,11 +141,12 @@ module CartonDb
       end
 
       def replace
-        replacement = self.class.new(
-          segment_group, "#{segment_filename}.txt"
-        )
+        repl_name = "#{segment_filename}.temp"
+        replacement = self.class.new(segment_group, repl_name)
         begin
-          yield replacement
+          replacement.open_overwrite do |io|
+            yield io
+          end
         rescue StandardError
           File.unlink replacement.filename
           raise
@@ -188,12 +176,7 @@ module CartonDb
         end
       end
 
-      def open_append
-        touch_dir
-        File.open filename, 'a', **FILE_ENCODING_OPTS do |io|
-          yield io
-        end
-      end
+      protected
 
       def open_overwrite
         touch_dir
@@ -216,6 +199,13 @@ module CartonDb
         end
       end
 
+      def open_append
+        touch_dir
+        File.open filename, 'a', **FILE_ENCODING_OPTS do |io|
+          yield io
+        end
+      end
+
       def touch_dir
         dir = File.dirname(filename)
         return if File.directory?(dir)
@@ -230,6 +220,25 @@ module CartonDb
         end
       end
 
+      def key_d_contents_map
+        entries = nil
+        each_entry_element_line do |key_d, elem_d, _line|
+          entries ||= {}
+          content = entries[key_d] ||= []
+          content << elem_d.plain unless elem_d.placeholder?
+        end
+        entries
+      end
+
+      def key_d_first_element_map
+        result = nil
+        each_entry_element_line do |key_d, elem_d, _line|
+          result ||= {}
+          result[key_d] ||= elem_d.plain
+        end
+        return result
+      end
+
     end
 
   end

data/lib/carton_db/version.rb

@@ -2,5 +2,5 @@
 # frozen_string_literal: true
 
 module CartonDb
-  VERSION = "1.1.3"
+  VERSION = "1.1.4"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: carton_db
 version: !ruby/object:Gem::Version
-  version: 1.1.3
+  version: 1.1.4
 platform: ruby
 authors:
 - Steve Jorgensen
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2017-05-09 00:00:00.000000000 Z
+date: 2017-05-17 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler