opensecret 0.0.962 → 0.0.988

data/lib/keytools/doc.conversion.to.ones.and.zeroes.ruby

@@ -0,0 +1,179 @@
+# coding: utf-8
+
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+
+Conversion from String to binary 
+Easiest way, using unpack and tell Ruby that it is binary
+"Hello".unpack("B*")
+=> ["0100100001100101011011000110110001101111"]
+
+
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+# =====> Look you convert to ones and zeroes like this - see below too.
+
+
+	-100.to_s(2)                         # => "-1100100"
+	-100.to_s(2)                         # => "-1100100"
+	-100.to_s(2)                         # => "-1100100"
+	-100.to_s(2)                         # => "-1100100"
+	-100.to_s(2)                         # => "-1100100"
+	-100.to_s(2)                         # => "-1100100"
+	-100.to_s(2)                         # => "-1100100"
+	-100.to_s(2)                         # => "-1100100"
+
+
+
+#### -----> See important padding article
+#### -----> See important padding article
+#### -----> See important padding article
+#### -----> See important padding article
+#### -----> See important padding article
+#### -----> See important padding article
+
+## --> https://stackoverflow.com/questions/4080988/why-does-base64-encoding-require-padding-if-the-input-length-is-not-divisible-by?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
+
+
+
+
+2.6. Converting Between Numeric Bases
+Problem
+You want to convert numbers from one base to another.
+
+Solution
+You can convert specific binary, octal, or hexadecimal numbers to decimal by representing them with the 0b, 0o, or 0x prefixes:
+
+	0b100                                # => 4
+	0o100                                # => 64
+	0x100                                # => 256
+You can also convert between decimal numbers and string representations of those numbers in any base from 2 to 36. Simply pass the base into String#to_i or Integer#to_s.
+
+Here are some conversions between string representations of numbers in various bases, and the corresponding decimal numbers:
+
+	"1045".to_i(10)                      # => 1045
+	"-1001001".to_i(2)                   # => -73
+	"abc".to_i(16)                       # => 2748
+	"abc".to_i(20)                       # => 4232
+	"number".to_i(36)                    # => 1442151747
+	"zz1z".to_i(36)                      # => 1678391
+	"abcdef".to_i(16)                    # => 11259375
+	"AbCdEf".to_i(16)                    # => 11259375
+Here are some reverse conversions of decimal numbers to the strings that represent those numbers in various bases:
+
+	42.to_s(10)                          # => "42"
+	-100.to_s(2)                         # => "-1100100"
+	255.to_s(16)                         # => "ff"
+	1442151747.to_s(36)                  # => "number"
+Some invalid conversions:
+
+	"6".to_i(2)                          # => 0
+	"0".to_i(1)                          # ArgumentError: illegal radix 1
+	40.to_s(37)                          # ArgumentError: illegal radix 37
+Discussion
+String#to_i can parse and Integer#to_s can create a string representation in every common integer base: from binary (the familiar base 2, which uses only the digits 0 and 1) to hexatridecimal (base 36). Hexatridecimal uses the digits 0–9 and the letters a–z; it's sometimes used to generate alphanumeric
+
+
+
+
+
+
+Convert String to Binary in Ruby
+As for the sake of today 11/11/11,
+Everything is in binary mode.
+Why not do a conversion between String and binary :)
+
+Conversion from String to binary 
+Easiest way, using unpack and tell Ruby that it is binary
+"Hello".unpack("B*")
+=> ["0100100001100101011011000110110001101111"]
+
+You can specify length of each binary by replace * with total length of the word (default binary has 8 bit per character)
+"Hello".unpack("B40")
+=> ["0100100001100101011011000110110001101111"]
+
+The output is in array format, removing an array can be done by adding [0] to select the first element from array.
+"Hello".unpack("B*")[0]
+=> "0100100001100101011011000110110001101111"
+
+Other method is to break down the string into characters first and get ASCII number of each character, then convert them to binary.
+Here is a way for breaking a String into ASCII character code.
+‎"Hello".unpack("C*")
+=> [72, 101, 108, 108, 111]
+
+Then using print formatting with %b to convert each ASCII code (int) to binary
+"Hello".unpack("C*").each{|c| print "%08b" % c}
+
+Another way to do using split('')
+"Hello".split('').each{|c| print "%08b" % c[0] }
+=> 0100100001100101011011000110110001101111
+
+Or you could use scan(/./)
+"Hello".scan(/./).each{|c| print "%08b" % c[0] }
+=> 0100100001100101011011000110110001101111
+
+Actually there is another method for converting number into binary, it is using ".to_str(2)" but this method doesn't put leading-zeros in the result, as you can see:
+"Hello".scan(/./).each{|c| print "%08s " % c[0].to_s(2) }
+=> 1001000 1100101 1101100 1101100 1101111
+
+Conversion from binary back to String
+Same as above, we can use the same method to convert binary back to String 
+a="0100100001100101011011000110110001101111"
+[a].pack("B*")
+=> "Hello"
+
+Or split it by 8 digit each, then convert each 8 digit to int , then to character 
+a="0100100001100101011011000110110001101111"
+(0..a.length).step(8).each {|i| print a[i,8].to_i(2).chr}
+=> Hello
+
+
+
+P.S.
+After reading .pack and .unpack function API, I found that Ruby has a built-in Base64 encoder/decoder.
+To use it just use pack() or unpack() with "m" directive
+Base64 Encoding 
+["Hello"].pack("m*").chomp
+=> "SGVsbG8="
+Base64 Decoding 
+"SGVsbG8=".unpack("m*")[0]
+=> "Hello"

data/lib/keytools/doc.rsa.radix.binary-mapping.ruby

@@ -0,0 +1,190 @@
+
+For Starters....
+I am often torn when programming between using libraries and following the DIY approach. On one hand you can accomplish a lot in a short time by including libraries that have the functionality you need and just bridging the gaps, but I am often left with the feeling of some loss-of-control and bloat. Obviously some library usage is inevitable, but I do like to take a minimalist approach to including external libraries because the management burden of updating libraries, the effort to make various libraries play together and because moving library-heavy applications between platforms can be... interesting. Most of all I like the learning that takes place when building it yourself. It was for this reason that I decided to see if I could effectively utilize the OpenSSL library that is typically part of a Ruby installation to create my own higher-level encryption/decryption class. Before we dive in I would like to layout the details of the application.
+
+Password Management App
+
+I have for years maintained a PGP encrypted file for my account passwords. I had the desire to centralize this and make it more web accessible. In order to do this I wanted assurance that the data was not only encrypted on the wire via SSL, but also at rest. I looked at various Ruby PGP/GPG libraries, but all of the ones I surveyed were wrappers around the GPG binary itself which didn't make it very web host friendly ( I planned on hosting it on Heroku with a Couchdb backend on Cloudant). That was when I decided to look at OpenSSL for file encryption since it is typically part of a Ruby installation and was available on Heroku. I also had the need for public-key encryption so a password store could be shared between two or more people (I share some accounts with my partner).
+
+Take One... RSA
+I thought I could simply use RSA encryption to meet all my needs and at first it worked quite well. Creating keys was straight forward and encryption/decryption worked well. One caveat is that the string passed to the FileEncryptor#encrypt_string method cannot be larger than the key size + PKCS padding size (11 bytes). So for a 128 byte (1024 bit) key the string should only be 117 bytes (see String#size).
+
+
+require 'openssl'
+
+class Encryptor
+  
+  def initialize(key_pass, key_size = 1024, key_name = 'rsakey.sec', cipher = OpenSSL::Cipher.new('aes-256-cbc'))
+    @key_size = key_size
+    @cipher = cipher
+    @key_name = key_name
+
+    if( File.exists?(@key_name) )
+      @rsakey = OpenSSL::PKey::RSA.new(File.read(@key_name), key_pass)
+    else
+      @rsakey = OpenSSL::PKey::RSA.generate(@key_size)
+      File.open(@key_name,'w+') do |priv|
+        priv.write(@rsakey.to_pem(@cipher, key_pass))
+      end
+    end
+  end
+
+  def encrypt_string(txt)
+    begin
+      @rsakey.public_encrypt(txt)
+    rescue OpenSSL::PKey::RSAError => e
+      if e.message == 'data too large for key size'
+        STDERR.puts 'Your string is too large to encrypt'
+      else
+        raise
+      end
+    end
+  end
+
+  def decrypt_string(etxt)
+    @rsakey.private_decrypt(etxt)
+  end
+end
+
+
+
+
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+## -- See YAML storage of key (excellent).
+
+
+
+This seemed to work quite well until I remembered that I needed to have the encrypted data be readable by multiple parties. I then had to change my approach and decided to use a symmetric AES key that is then shared between the multiple parties and each person encrypts the key with their RSA public key.
+
+Take Two... RSA + AES
+In order to accomplish secure use of the AES key we start out the same way as the previous solution by creating a RSA key. It will only be used to secure AES keys that we use to protect certain data stores. So if I have a data store I want to share with my brother I encrypt the data store with an AES key then I encrypt that key with my public RSA key and my brother's public RSA key. Now both he and I can access anything in that store and the AES key is still secure from prying eyes. Here is an overview of the steps taken before I post the code.
+Create a RSA key-pair
+Create an AES key to encrypt the data store
+Encrypt the AES key with your RSA public key
+To give access to the data store encrypt the AES key with the person's public RSA key send them back the cipher-text.
+
+And here is the class I wrote to make this work:
+
+
+require 'openssl'
+require 'yaml'
+
+class Encryptor
+  
+  def initialize(key_pass, key_size = 2048, key_name = 'rsakey.sec', cipher = OpenSSL::Cipher.new('aes-256-cbc'))
+    @key_size = key_size
+    @cipher = cipher
+    @key_name = key_name
+
+    if( File.exists?(@key_name) )
+      @rsakey = OpenSSL::PKey::RSA.new(File.read(@key_name), key_pass)
+    else
+      @rsakey = OpenSSL::PKey::RSA.generate(@key_size)
+      File.open(@key_name,'w+') do |priv|
+        priv.write(@rsakey.to_pem(@cipher, key_pass))
+      end
+    end
+  end
+
+  def rsa_encrypt(txt, pub_key=nil)
+    begin
+      pub_key.nil? ? @rsakey.public_encrypt(txt) : OpenSSL::PKey::RSA.new(pub_key).public_encrypt(txt)
+    rescue OpenSSL::PKey::RSAError => e
+      if e.message == 'data too large for key size'
+        STDERR.puts 'Your string is too large to encrypt'
+      else
+        raise
+      end
+    end
+  end
+
+  def rsa_decrypt(etxt)
+    @rsakey.private_decrypt(etxt)
+  end
+
+  def aes_encrypt(txt, aes_file)
+    key = get_aes_key(aes_file)
+    cif = OpenSSL::Cipher.new('AES-256-CBC')
+    cif.encrypt
+    cif.key = key[:key]
+    cif.iv  = key[:iv] = cif.random_iv
+    save_aes_key(aes_file, key)
+    etxt = ''
+    etxt << cif.update(txt)
+    etxt << cif.final
+    etxt
+  end
+
+  def aes_decrypt(etxt, aes_file)
+    key = get_aes_key(aes_file)
+    cif = OpenSSL::Cipher.new('AES-256-CBC')
+    cif.decrypt
+    cif.key = key[:key]
+    cif.iv  = key[:iv]
+    txt = ''
+    txt << cif.update(etxt)
+    txt << cif.final
+    txt
+  end
+
+  def gen_aes_key(file)
+    cif = OpenSSL::Cipher.new('AES-256-CBC')
+    key = {key: cif.random_key}
+    save_aes_key(file, key)
+    key
+  end
+
+  def save_aes_key(file,key_iv) 
+    File.open(file,'w+') do |f|
+      f.write(rsa_encrypt(YAML.dump(key_iv)))
+    end
+  end
+
+  def get_aes_key(file)
+    YAML.load(rsa_decrypt(File.read(file)))
+  end
+  
+  def give_aes_key(aes_file, rsa_pub)
+    aes = get_aes_key(aes_file)
+    rsa_encrypt(YAML.dump(aes), rsa_pub)
+  end
+
+end
+
+
+
+Below is a demonstration of the use of this class to accomplish the needs I had for my password protector application. I am aware that this class needs some refactoring and it's a bit annoying to have to pass the AES key file to the methods, but logically it is working the way I had hoped.
+
+
+
+fe = Encryptor.new('mysecret')
+
+aesfile = 'aeskey.sec'
+
+fe.gen_aes_key(aesfile)
+
+etxt = fe.aes_encrypt('this is a test', aesfile)
+
+txt = fe.aes_decrypt(etxt, aesfile)
+
+pub_key = < Assume I got someone's public RSA Key somehow >
+
+fe.give_aes_key(aesfile, pub_key)
+
+Like I mentioned, this code needs to be cleaned up a bit, but it gives me all of the functionality I need to build my application and it doesn't require any external libraries except OpenSSL and YAML which are typically standard in any Ruby installation.
+
+Hopefully someone finds this post useful and if you have any additions, corrections, criticisms please post them below. Feedback is always welcome.

data/lib/keytools/doc.star.schema.strategy.txt

@@ -0,0 +1,77 @@
+
+
+### @@@@@@@@@@@@@@@@@@@@@@@@@@
+### Star Schema Strategy
+### @@@@@@@@@@@@@@@@@@@@@@@@@@
+
+ - We move away from a single private key to one per sealed envelope.
+
+There are 2 huge 72 character keys ( NOT base64 characters but UTF-7 (or 8) possible characters )
+
+   (note - in common mistakes article put the fact that random keys are generated
+           from tools like securerandom that do not cover the full range f possible
+	   chars hence sharpely narrowing the search space for an attacker.
+	   )
+
+   Write test program to prove that EVERY ONE of THE 128 CHARACTERS COUNT.
+   How ----> Create 71 characters that are all AAAAAAA...
+       ====> Rotate round each of the 128 chars as the 72nd character
+       ----> Check that bcrypt generates 128 different outcomes for each of the 128 chars.
+
+
+==> Use get random bytes to get 72 random bytes (not secure random which halves (if not worse) the search space).
+
+
+When password entered we read the salt and retrieve the key and unlock the STRONG 256 BIT key (found by getting 32 random bytes then mapping to 32 bytes with say SHA512 digest) crypt for index file.
+
+We then grab this crypt key (32 bytes long or 64 bytes - the longest a symmetric crypt key can be) - test this out with AES256.
+
+The crypt key unlocks the index file.
+
+We then grab the session ID - then hash that say through Digest.SHA512 - and use the resulting key to decrypt the ENVironment VARIABLE.
+This reveals a 512 (or 256) bit SESSION crypt key which creates a copy of the (in-memory) index file and then writes it with a new name
+that is the alpha-number session ID hash. We encrypt the index file with the new key.
+
+Still in SAME CALL we prepare the NEXT (future) SEAL password. We create a 256 bit key and then derive a new bcrypt salt and out comes the 186bit key (which we hash and use to lock the 256 bit key just created and we save the salt in the configs and save the future SEAL password crypt in the configs.
+The crux move is we use the session key to also lock the seal password for when session ends.
+
+NOW WE THROW AWAY THE PASSWORD THE BEAUTY OF THIS METHOD IS THAT WE NEVER KEEP THE PASSWORD ANYWHERE ENCRYPTED OR OTHERWISE - WE TRHOW IT AWAY WE WONT NEED IT AGAIN UNTIL THE NEXT SESSION.
+
+
+Now we continue with the other open, put, add type calls. They all now accesses the copied session index file and it is changed as appropriate.
+
+
+Finally when the SEAL time comes we use session key to unlock and retrieve the NEXT time (seal) key. We lock index file with this key.
+
+In clean up we throw away all our session bits and bobs.
+
+
+We are ready for "next time" - when the password is entered we derive the key again and can unlock the last saved index file!! Cool plan or what??
+
+
+REDIS Store + more
+
+This plan allows a seamless migration to keeping the index not in a file (risks failure)
+but in a REDIS or etcd or PostgreSQL or other key-value stores.
+
+
+
+### @@@@@@@@@@@@@@@@@@@@@@@@@@
+### Star Schema for Each Envelope Details
+### @@@@@@@@@@@@@@@@@@@@@@@@@@
+
+ - path off session base
+ - path to private key
+ - (inner encrypted private key lock)
+ - path to keyfile
+ - path to crypt file
+ - outer path
+ - inner path
+ - sealed already?
+ - is reopen or no?
+ - last accessed time
+ - who by user@hostname
+ - on what ppi (shell) id@mac address
+ - top level count
+
+

data/lib/keytools/doc.using.pbkdf2.kdf.ruby

@@ -0,0 +1,95 @@
+# coding: utf-8
+
+    # OpenSSL::PKCS5.pbkdf2_hmac has been renamed to OpenSSL::KDF.pbkdf2_hmac.
+    # This method is provided for backwards compatibility.
+
+    def pbkdf2_hmac(pass, salt, iter, keylen, digest)
+
+      OpenSSL::KDF.pbkdf2_hmac(pass, salt: salt, iterations: iter,
+                               length: keylen, hash: digest)
+
+    end
+
+
+##########################################################################################
+##########################################################################################
+##########################################################################################
+##########################################################################################
+##########################################################################################
+
+
+OpenSSL::KDF
+Provides functionality of various KDFs (key derivation function).
+
+KDF is typically used for securely deriving arbitrary length symmetric keys to be used with an OpenSSL::Cipher from passwords. Another use case is for storing passwords: Due to the ability to tweak the effort of computation by increasing the iteration count, computation can be slowed down artificially in order to render possible attacks infeasible.
+
+Currently, OpenSSL::KDF provides implementations for the following KDF:
+
+PKCS #5 PBKDF2 (Password-Based Key Derivation Function 2) in combination with HMAC
+
+scrypt
+
+HKDF
+
+############################
+#### Parameters
+############################
+
+pass = the passphrase
+
+salt = The salt. Salts prevent attacks based on dictionaries of common passwords and attacks based on rainbow tables. It is a public value that can be safely stored along with the password (e.g. if the derived value is used for password storage).
+
+iterations = The iteration count. This provides the ability to tune the algorithm. It is better to use the highest count possible for the maximum resistance to brute-force attacks.
+
+length = The desired length of the derived key in octets.
+
+hash = The hash algorithm used with HMAC for the PRF. May be a String representing the algorithm name, or an instance of OpenSSL::Digest.
+
+
+############################
+#### Examples
+############################
+
+Generating a 128 bit key for a Cipher (e.g. AES)¶ ↑
+pass = "secret"
+salt = OpenSSL::Random.random_bytes(16)
+iter = 20_000
+key_len = 16
+key = OpenSSL::KDF.pbkdf2_hmac(pass, salt: salt, iterations: iter,
+                               length: key_len, hash: "sha1")
+
+############################
+#### Storing Passwords
+############################
+
+pass = "secret"
+# store this with the generated value
+salt = OpenSSL::Random.random_bytes(16)
+iter = 20_000
+hash = OpenSSL::Digest::SHA256.new
+len = hash.digest_length
+# the final value to be stored
+value = OpenSSL::KDF.pbkdf2_hmac(pass, salt: salt, iterations: iter,
+                                 length: len, hash: hash)
+
+############################
+#### Important Note on Checking Passwords
+#### Thwart Timing Attacks by taking the Same Amount of Time to Check Failures and Successes
+############################
+
+When comparing passwords provided by the user with previously stored values, a common mistake made is comparing the two values using "==". Typically, "==" short-circuits on evaluation, and is therefore vulnerable to timing attacks. The proper way is to use a method that always takes the same amount of time when comparing two values, thus not leaking any information to potential attackers. To compare two values, the following could be used:
+
+def eql_time_cmp(a, b)
+  unless a.length == b.length
+    return false
+  end
+  cmp = b.bytes
+  result = 0
+  a.bytes.each_with_index {|c,i|
+    result |= c ^ cmp[i]
+  }
+  result == 0
+end
+                           Please note that the premature return in case of differing lengths typically does not leak valuable information - when using PBKDF2, the length of the values to be compared is of fixed size.
+
+