safedb 0.01.0001

data/lib/keytools/key.64.rb

@@ -0,0 +1,326 @@
+#!/usr/bin/ruby
+
+module SafeDb
+
+  # First use the class methods to source keys, then use a key's instance
+  # methods to access its properties and in concert with other symmetrical
+  # information, you can use the keys to lock (encrypt) or unlock (decrypt)
+  # other keys and objects.
+  #
+  # == Sourcing and Deriving Keys
+  #
+  # Keys can be
+  #
+  # - sourced from a secure random byte generating function
+  # - sourced from ciphertext and another (decryption) key
+  # - generated by passing a secret through key derivation functions
+  # - regenerated from a secret and previously stored salts
+  # - sourced from the current unique workstation shell environment
+  # - sourced from an environment variable containing ciphertext
+  #
+  #
+  # Keys need to be viewed (represented) in multiple ways and the essence
+  # of the key viewer is to input keys {as_bits}, {as_bytes} and {as_base64}
+  # and then output the same key (in as far as is possible) - as bits, as
+  # bytes and as base64.
+  #
+  # == Key | To and From Behaviour
+  #
+  # Use the <b>From</b> methods to create Keys from a variety of resources
+  # such as
+  #
+  # - a base64 encoded string
+  # - a binary byte string
+  # - a string of one and zero bits
+  # - a hexadecimal representation
+  #
+  # Once you have instantiated the key, you will then be able to convert it
+  # (within reason due to bit, byte and base64 lengths) to any of the above
+  # key representations.
+  #
+  # == Key | Bits Bytes and Base64
+  #
+  # The shoe doesn't always fit when its on the other foot and this is best
+  # illustratd with a table that maps bits to 8 bit bytes and 6 bit Base64
+  # characters.
+  #
+  #   | --------- | -------- | ------------ | ------------------------------- |
+  #   | Fit?      | Bits     | Bytes        | (and) Base64                    |
+  #   | --------- | -------- | ------------ | ------------------------------- |
+  #   | Perfect   | 168 Bits | is 21 bytes  | 28 Chars - bcrypt chops to this |
+  #   | Perfect   | 216 Bits | is 27 bytes  | 36 Chars -                      |
+  #   | Perfect   | 264 Bits | is 33 bytes  | 44 Chars - holder 4 256bit keys |
+  #   | Perfect   | 384 Bits | is 48 bytes  | 64 Chars - 216 + 168 equals 384 |
+  #   | --------- | -------- | ------------ | ------------------------------- |
+  #   | Imperfect | 128 Bits | 16 precisely | 22 Chars - 21 + 2 remain bits   |
+  #   | Imperfect | 186 Bits | 23 remain 2  | 31 Characers precisely          |
+  #   | Imperfect | 256 Bits | 32 precisely | 43 Chars - 42 + 4 remain bits   |
+  #   | --------- | -------- | ------------ | ------------------------------- |
+  #
+  # Yes, the shoe doesn't always fit when it's on the other foot.
+  #
+  # == Schoolboy Error
+  #
+  # <b>The strategy is so simple, we call it a schoolboy error.</b>
+  #
+  # If we want to use a key with n bits and either n % 6 or n % 8 (or both)
+  # are not zero - <b>we instantiate a Key</b> with the lowest common
+  # denominator of 6 and 8 that exceeds n.
+  #
+  # So when we request a byte, or base64 representation the viewer will
+  # truncate (not round down) to the desired length.
+  #
+  #
+  # == YACHT 64 | Yet Another Character Table
+  #
+  # This binary key class is a dab hand at converting base64 strings
+  # into their 6-bit binary string equivalents.
+  #
+  # It can convert non-alphanumeric characters within either Base64 or
+  # Radix64 into the SafeDb YACHT64 standard which has a forward slash
+  # but neither a plus sign nor a period character.
+  #
+  # <b>The Big4 Character Sets | Base64 | UrlSafe64 | Radix64 | YACHT64</b>
+  #
+  # Base64 and Radix64 (from OpenBSD) differ in both the order of characters
+  # and their choice of the two non-alphanumeric characters. Base64 can also
+  # contain line breaks and equal signs for padding. UrlSafe base64 has different
+  # choices for the two non alphanum characters in keeping with URL standards.
+  #
+  # The character sets for each of the four 64 fomats are as follows.
+  #
+  # - Base-64 is <b>A to Z</b> then <b>a to z</b> then <b>0 to 9</b> then <b>+</b> then <b>/</b>
+  # - Radix64 is <b>.</b> then <b>/</b> then <b>0 to 9</b> then <b>A to Z</b> then <b>a to z</b>
+  # - UrlSafeBase64 is Base64 but chars 63/64 are an <b>underscore (_)</b> and <b>hyphen (-)</b>
+  # - UrlSafeBase64 <b>does not have line breaks and carriage returns</b> (unlike Base64)
+  # - <b>SafeDb 64 (YACHT64)</b> uses the same 62 characters plus an @ sign and a forward slash
+  # - The 64 <b>SafeDb 64</b> characters are <b>obfuscated into a random order</b>
+  #
+  # == 4 Non-AlphaNumerics | Base64 | Radix64 | YACHT64
+  #
+  # The behaviour here is happy to convert base64 strings produced by either
+  # Radix64 or Base64 or UrlSafe Base64. Howeverr it aware of the
+  # <b>non alpha-numeric characters</b> and converts them before processing
+  # with the modus operandi that says
+  #
+  # - ignore the forward slash  in <b>YACHT64, Base64 and Radix64</b>
+  # - convert the <b>plus (+)</b> in Base64 to the <b>@ symbol</b> in YACHT64
+  # - convert the <b>period (.)</b> in <b>Radix64</b> to the @ symbol in YACHT64
+  # - convert <b>hyphen (-)</b> in <b>Url Safe Base64</b> into a fwd slash
+  # - convert <b>underscore (_)</b> in <b>Url Safe Base64</b> to an @ sign
+  # - <b>delete the (=) equals</b> padding character used by Base64
+  #
+  # Neither the OpenBSD backed Radix64 nor the SafeDb (YACHT64) entertain the
+  # concept of padding.
+  #
+  # == Mapping Each Character to 6 Binary Bits
+  #
+  # We need 6 binary bits to represent a base64 character (and 4
+  # bits for hexadecimal). Here is an example mapping between
+  # a base 64 character, an integer and the six bit binary.
+  #
+  #    Character   Integer  Binary (6 Bit)
+  #
+  #       a           0        000000
+  #       b           1        000001
+  #       c           2        000010
+  #
+  #       y           25       011001
+  #       z           26       011010
+  #       A           27       011011
+  #       B           28       011100
+  #
+  #       8           60       111100
+  #       9           61       111101
+  #       /           62       111110
+  #       +           63       111111
+  #
+  class Key64
+
+    # YACHT stands for <b>Yet Another Character Table</b> and it
+    # can map binary sequences onto 64 well chosen characters.
+    #
+    # The 64 character sets are all similar in that they hold 64
+    # characters and they define two non alphanumeric characters
+    # because the 26 lowercase, 26 uppercase and 10 digits only
+    # adds up to an <b><em>agonisingly close</em></b> 62 characters.
+    #
+    YACHT64_CHARACTER_SET = [
+      "a", "9", "W", "B", "f", "K", "O", "z",
+      "3", "s", "1", "5", "c", "n", "E", "J",
+      "L", "A", "l", "6", "I", "w", "o", "g",
+      "k", "N", "t", "Y", "S", "%", "T", "b",
+      "V", "R", "H", "0", "@", "Z", "8", "F",
+      "G", "j", "u", "m", "M", "h", "4", "p",
+      "q", "d", "7", "v", "e", "2", "U", "X",
+      "r", "C", "y", "Q", "D", "x", "P", "i"
+    ]
+
+
+    # Radix64 strings can contain period characters in their midst.
+    PERIOD = "."
+
+    # Radix64 strings can contain forward slashes in their midst.
+    FORWARD_SLASH = "/"
+
+    # YACHT64 strings can contain at symbols in their midst.
+    AT_SYMBOL = "@"
+
+    # YACHT64 strings can contain percent signs in their midst.
+    PERCENT_SIGN = "%"
+
+
+    # Convert the parameter string of ones and zeroes into an
+    # internal base64 character set known as YACHT for yet another
+    # character table.
+    #
+    # @param bit_string [String]
+    #    a string of ones and zeroes that can be sliced into
+    #    six character chunks with each chunk then being mapped
+    #    to a YACHT64 character.
+    #
+    # @return [String]
+    #    printable characters from a set of 62 alpha-numerics
+    #    plus an @ symbol and a percent % sign.
+    #
+    # @raise ArgumentError
+    #    If the bit string is nil.
+    #    Or if the bit string length is not a multiple of six.
+    #    Or if it contains any character that is not a 1 or 0.
+    def self.from_bits bit_string
+
+      nil_err_msg = "The parameter bit string cannot be nil."
+      raise ArgumentError, nil_err_msg if bit_string.nil?
+
+      bit_size_msg = "The bit string length is not a multiple of #{SIX}."
+      raise ArgumentError, bit_size_msg unless bit_string.length % SIX == 0
+
+      num_unknowns = bit_string.delete("10").length
+      unknowns_msg = "The bit string has #{num_unknowns} characters that are not 1 or 0."
+      raise ArgumentError, unknowns_msg if num_unknowns > 0
+
+      characters64 = ""
+      char_count = bit_string.length / SIX
+      for n in 0 .. (char_count-1)
+        six_bit_chunk = bit_string[ (n*SIX), SIX ]
+        six_bit_index = six_bit_chunk.to_i(2)
+        characters64 += Key64.character(six_bit_index)
+      end
+      
+      code_size_msg = "Length is #{characters64.length} but #{char_count} is expected."
+      raise RuntimeError, code_size_msg unless characters64.length == char_count
+
+      return characters64
+
+    end
+
+
+    # Convert the parameter characters based on an internal base64
+    # character set (known as YACHT) into a <b>bit string</b> of ones
+    # and zeroes.
+    #
+    # @param char64_string [String]
+    #    The base64 character sequence which which will be used to
+    #    derive the returned bit string. Naturally this character
+    #    sequencee cannot be nil, nor can it contain any characters
+    #    that are not present in {Key64::YACHT64_CHARACTER_SET}.
+    #
+    # @return [String]
+    #    a string of ones and zeroes that have been strung out
+    #    from each YACHT64 character. The returned string length of
+    #    ones and zeroes will be exactly 6 times the length of the
+    #    input parameter.
+    #
+    # @raise [ArgumentError]
+    #    If a nil or zero length character string is received.
+    #    Or if the character sequence contains a character not present
+    #    in the {Key64::YACHT64_CHARACTER_SET}.
+    #    
+    # @raise [RuntimeError]
+    #    if the conversion does not result in 6 bits for every character
+    #    in the parameter string.
+    def self.to_bits char64_string
+
+      bit_string = ""
+      char64_string.each_char do |the_char|
+
+        yacht64_index = YACHT64_CHARACTER_SET.index(the_char)
+        assert_yacht64_index( the_char, yacht64_index )
+        bit_string += "%06d" % [ yacht64_index.to_s(2) ]
+
+      end
+
+      assert_bit_lengths char64_string, bit_string
+      return bit_string
+
+    end
+
+
+    # Convert a string of Radix64 characters into a bit representation which
+    # will be 6 times longer than the input parameter. This method first
+    # converts the string into the internal YACHT64 format and then converts
+    # that to a bit string using the {Key64.to_bits} method.
+    #
+    # @param radix64_string [String]
+    #    the radix64 string to convert into bits. This string will be a subset
+    #    of the usual 62 character suspects together with period and forward
+    #    slash characters.
+    #
+    #    This parameter should not contain newlines nor carriage returns.
+    #
+    # @return [String]
+    #    a string of ones and zeroes that represent the bits converted from the
+    #    radix64 input. The return value will be exactly 6 times the number of
+    #    input characters.
+    def self.from_radix64_to_bits radix64_string
+
+      yacht64_chars = radix64_string.gsub( PERIOD, AT_SYMBOL ).gsub( FORWARD_SLASH, PERCENT_SIGN )
+      out_bitstring = to_bits( yacht64_chars )
+      assert_bit_lengths( radix64_string, out_bitstring )
+      return out_bitstring
+
+    end
+
+
+
+    private
+
+
+
+    SIX = 6
+
+    def self.character char_index
+
+      index_oob_msg = "The character index must be between 0 and 63 inclusive."
+      index_is_oob = char_index < 0 || char_index > 63
+      raise ArgumentError, index_oob_msg if index_is_oob
+      return YACHT64_CHARACTER_SET[ char_index ]
+
+    end
+
+    def self.assert_bit_lengths( in_string, out_string )
+
+      in_length = in_string.length
+      out_length = out_string.length
+      good_ratio = out_length == in_length * SIX
+      size_msg = "Bit string length [#{out_length}] not 6 times more than [#{in_length}]."
+      raise RuntimeError, size_msg unless good_ratio
+
+    end
+
+    def self.assert_yacht64_index the_char, yacht64_index
+
+      nil_msg = "Character [ #{the_char} ] not in YACHT character set."
+      raise ArgumentError, nil_msg if yacht64_index.nil?
+
+      index_msg = "Index of character [ #{the_char} ] not within expected bounds."
+      all_good = ( yacht64_index >= 0 ) && ( yacht64_index <= 63 )
+      raise ArgumentError, index_msg unless all_good
+
+    end
+
+
+  end
+
+
+end

data/lib/keytools/key.algo.rb

@@ -0,0 +1,109 @@
+#!/usr/bin/ruby
+# coding: utf-8
+
+module SafeDb
+
+  # Algorithms that are quality catalysts in the derivation and entropy spread
+  # of keys, identifiers and base64 character numbers.
+  class KeyAlgo
+
+
+    # Cherry pick a given number of characters from the character pool
+    # so that a good spread is achieved. This picker is the anti-pattern
+    # of just axing the first 5 characters from a 100 character string
+    # essentially wasting over 90% of the available entropy.
+    #
+    # This is the <b>algorithem to cherry pick</b> a spread of characters
+    # from the pool in the second parameter.
+    #
+    # - if the character pool length is a multiple of num_chars all is good otherwise
+    # - constrict to the <b>highest multiple of the pick size below</b> the pool length
+    # - divide that number by num_chars to get the first offset and character spacing
+    # - if spacing is 3, the first character is the 3rd, the second the 6th and so on
+    # - then return the cherry picked characters
+    #
+    # @param pick_size [FixNum] the number of characters to cherry pick
+    # @param char_pool [String] a pool of characters to cherry pick from
+    # @return [String]
+    #    a string whose length is the one indicated by the first parameter
+    #    and whose characters contain a predictable, repeatable spread from
+    #    the character pool parameter
+    def self.cherry_picker( pick_size, char_pool )
+
+      hmb_limit = highest_multiple_below( pick_size, char_pool.length )
+      jump_size = hmb_limit / pick_size
+      read_point = jump_size
+      picked_chars = ""
+      loop do 
+        picked_chars += char_pool[ read_point - 1 ]
+        read_point += jump_size
+        break if read_point > hmb_limit
+      end
+      
+      err_msg = "Expected cherry pick size to be #{pick_size} but it was #{picked_chars.length}."
+      raise RuntimeError, err_msg unless picked_chars.length == pick_size
+
+      return picked_chars
+
+    end
+    
+
+    # Affectionately known as <b>a hmb</b>, this method returns the
+    # <b>highest multiple</b> of the first parameter that is below
+    # <b>(either less than or equal to)</b> the second parameter.
+    #
+    #      - -------- - ------- - ----------------- -
+    #      |  Small   |   Big   | Highest Multiple  |
+    #      |  Number  |  Number |  Below Big Num    |
+    #      | -------- - ------- - ----------------- |
+    #      |    5     |   25    |      25           |
+    #      |    3     |   20    |      18           |
+    #      |    8     |   63    |      56           |
+    #      |    1     |    1    |       1           |
+    #      |    26    |   28    |      26           |
+    #      |    1     |    7    |       7           |
+    #      |    16    |   16    |      16           |
+    #      | -------- - ------- - ----------------- |
+    #      |    10    |    8    |     ERROR         |
+    #      |   -4     |   17    |     ERROR         |
+    #      |    4     |  -17    |     ERROR         |
+    #      |    0     |   32    |     ERROR         |
+    #      |    29    |   0     |     ERROR         |
+    #      |   -4     |   0     |     ERROR         |
+    #      | -------- - ------- - ----------------- |
+    #      - -------- - ------- - ----------------- -
+    #
+    # Zeroes and negative numbers cannot be entertained, nor can the
+    # small number be larger than the big one.
+    #
+    # @param small_num [FixNum]
+    #    the highest multiple of this number below the one in the
+    #    next parameter is what will be returned.
+    #
+    # @param big_num [FixNum]
+    #    returns either this number or the nearest below it that is
+    #    a multiple of the number in the first parameter.
+    #
+    # @raise [ArgumentError]
+    #    if the first parameter is greater than the second
+    #    if either or both parameters are zero or negative
+    def self.highest_multiple_below small_num, big_num
+
+      arg_issue = (small_num > big_num) || small_num < 1 || big_num < 1
+      err_msg = "Invalid args #{small_num} and #{big_num} to HMB function."
+      raise ArgumentError, err_msg if arg_issue
+
+      for index in 0 .. ( big_num - 1 )
+        invex = big_num - index # an [invex] is an inverted index
+        return invex if invex % small_num == 0
+      end
+      
+      raise ArgumentError, "Could not find a multiple of #{small_num} lower than #{big_num}"
+
+    end
+
+
+  end
+
+
+end