safedb 0.7.1001 → 0.10.5

data/lib/manual/git-interaction.md

@@ -0,0 +1,176 @@
+
+# safe | git interaction
+
+safe uses git for local synchronization and also as one of the many remote backend storage engines together with S3, SSH and Google Drive. As such, the git client is a prerequisite on systems running the safe command line interface (CLI).
+
+With Git comes the ability to revert state to any point in time after state-changing command transactions.
+
+Now we document the safe use cases that employ the git version control system and we discuss the circumstances surrounding its use.
+
+
+
+## **`safe init`**
+
+The safe init use case creates a book and creates the **`safedb-master-keys.ini`** file and a chapter crypt file under a new book directory.
+
+| **git action** | **when is it done?** | **what is done and/or the circumstances in which git is used**      |
+|:-------------- |:-------------------- |:------------------------------------------------------------------- |
+| _git init_     | at the beginning     | done only once if the master crypts folder is not under git control |
+| _git add_      | just before the end  | every init adds master keys and the just created book folder to git |
+| _git commit_   | at the end           | the git commit is issued if and only if the git add was invoked     |
+
+
+
+## **`safe login`**
+
+The safe init use case creates a book and creates the **`safedb-master-keys.ini`** file and a chapter crypt file under a new book directory.
+
+| **git action**     | **what is done andr the circumstances under which git is used**     |
+|:------------------ |:------------------------------------------------------------------- |
+| _git rm FILENAME_  | remove the outgoing master crypt content file from the repository and working copy |
+| _git add FILENAME_  | add the incoming master crypt content file into git version management |
+| _git add FILENAME_  | add the master indices INI which may have new keys, a new content ID and maybe bootup ID |
+| _git commit_  | commit the hot-swapped crypt files and the master indices file as an atomic (ACID) transaction |
+
+
+
+## **`safe commit`**
+
+The **`safe commit`** use case uses git to **protect the master database from the ills of concurrent access**. The git interactions are not **_spray and pray_**. They are specific to each given file that is added, removed and/or updated.
+
+| **git action**   | **when is it done?** | **what is done and/or the circumstances in which git is used**      |
+|:---------------- |:-------------------- |:------------------------------------------------------------------- |
+| **_git add_**    | just before the end  | new files incoming to the master crypts are brought under vc        |
+| **_git rm_**     | just before the end  | removed files are expunged from the git repository                  |
+| **_git commit_** | at the end           | the git commit is issued if either git add or git rm was invoked    |
+
+
+
+## **`safe pull`**
+
+If the local safe is already under version control the **`safe pull`** command will check for equivalence between the git repository url and the upstream url registered  by **`git remote`**. If it isn't - a **`git clone`** will suffice to pull down the safe repository assets.
+
+### safe pull invalidates every branch
+
+**Important** - **`safe pull`** invalidates both the master branch and all active branches. Before you issue a safe pull you must **`safe commit`** on every shell branch that contains changes.
+
+The commit provides a route back to a previous revision if the pull goes belly up and turns out to be something other than what you expected. Use **`safe compare`** to detail all the active branches that will be invalidated including the their login time and most recent access and change times.
+
+| **git action**   | **when is it done?** | **what is done and/or the circumstances in which git is used**      |
+|:---------------- |:-------------------- |:------------------------------------------------------------------- |
+| **_git remote_** | at the beginning     | the upstream url is compared to the url safe pull has a handle to   |
+| **_git clone_**  | towards the end      | if there are no crypts to speak of the git clone pulls them in      |
+| **_git pull_**   | towards the end      | this is a git fetch and git merge to integrate remote repo changes  |
+
+After a **`safe pull`** you must issue a **`safe login`** to continue working.
+
+### pull first | ask questions later
+
+A **`safe pull`** is recommended at the start of your session. Do a **`safe diff`** which does not require you to login in order to assess the differences between the local and remote master crypts.
+
+
+
+## **`safe push`**
+
+After one or more commits a **`safe push`** is called upon to sync the local crypt state with the registered remote repository.
+
+However a git push may not be possible if the remote has moved further ahead than the local. This matter would be reported and the user encouraged to perform a **`safe pull`** first followed by a **`safe refresh`**, changes, **`safe commit`** and finally another **`safe push`**.
+
+| **git action**   | **when is it done?** | **what is done and/or the circumstances in which git is used**      |
+|:---------------- |:-------------------- |:------------------------------------------------------------------- |
+| **_git push_**   | towards the end      | if this is a git fetch and git merge to integrate remote repo changes  |
+
+
+
+## **`safe remote`** or **`safe stage`**
+
+**`safe remote`** prints a report on the difference between the local and remote databi.
+
+    git ls-files --stage --abbrev=8
+
+The `ls-files` command is used to list the files that are staged and ready to be pushed up to the remote repository.
+
+
+
+## **`safe remote --provision`**
+
+The **`safe remote`** command is primed to do four key tasks. It
+
+- _automagically_ creates a remote repository (using for example **Github's API** integration)
+- it provisions and installs **SSH keypairs** for **`safe push`** to write to the remote backend
+- it uses **`set-upstream-url`** to tell the local repository where to **pull from** and **push to**
+- urges the user commit branch changes **`safe commit`** and mirror them remotely **`safe push`**
+
+A safe remote only acts to provision a remote mirror for your crypts when the **local git reposiotory is virginal** in that it has never been paired with a remote repository. In other words the local crypts have been created using **`safe init <<book-name>>`** as opposed to **`safe pull`**.
+
+| **git action**               | **when is it done?** | **what is done and/or the circumstances in which git is used**    |
+|:---------------------------- |:-------------------- |:----------------------------------------------------------------- |
+| **_git set-upstream-url_**   | at the beginning     | if remote creates the 2nd remote repo the upstream url is changed |
+
+
+The remote's last responsibility is to urge the user to issue a **`safe commit`** followed by a **`safe push`** so as to make the remote repository mirrors the state of the local safe.
+
+
+
+## **`safe compare`**
+
+The **`safe diff`** command reports on the difference between the local master book and the present local branch book.
+
+**`safe compare`** on the other hand tells you that
+
+- the remote branch has changed (leaving you behind needing to **`safe pull`**)
+- the local branch has changed (putting you ahead needing to **`safe push`**)
+- you cannot access the remote repository and cannot ascertain the above info
+
+| **git action**     | **what is done and/or the circumstances in which git is used** |
+|:------------------ |:-------------------------------------------------------------- |
+| **_git compare_**  | this command is used to ascertain local vs remote differences  |
+| **_git rev-diff_** | another command to ascertain local vs remote differences       |
+
+So safe compare reports on the local commits informing you which branch made them, when and a rough change count. On the other hand it tells you about the remote commits, who made them and when.
+
+### the worst of both worlds
+
+When commits have moved on both the local and remote master branches you are in the worst ofboth worlds. Thankfully this scenario is extremely rare. The rough steps to resolve this are to
+
+- export the local books as json
+- pull the remote books down then export them as json
+- externally compare the json and merge them into one
+- import the merged json to create new books
+
+
+
+## safe remote architecture
+
+Due to its meticulous planning the safe adheres to a number of high level design rules. Let's cover these in the context of operating alongside a remote _git_ backend repository.
+
+
+### 1. stand alone
+
+The term **_stand alone_** refers to a non-networked computer that has no access neither to the internet nor to other computers in its vicinity.
+
+The **safe cli** must be able to operate on a stand alone machine. Features like the remote backend mirror must provide succinct legible error messages when remote access is unavailable.
+
+Furthermore, the following commands must not be impaired when the machine is in standalone mode.
+
+- **`safe diff`** (does not report on remote differences)
+- **`safe commit`** (only changes the master branch)
+- **`safe init`** (only creates a local git repository
+
+And these commands must degrade gracefully in standalone mode
+
+- **`safe remote`**
+- **`safe push`**
+- **`safe pull`**
+- **`safe compare`**
+
+
+### 2. cap theorem
+
+#### consistency trumps availability
+
+In reference to the CAP theorem, the safe is designed such that **consistency trumps availability**.
+
+Failed commits in a manner of speaking are preferable to corrupted files and race conditions. Without git, these corruptions would arise through the use of basic file operations.
+
+ Automatons like scripts will typically branch and read the database. It is envisaged that humans will perform the vast majority of commits, pushes and pulls thus reducing the frequency of commit failures.

data/lib/manual/remote.md

@@ -12,7 +12,6 @@ Before you use safe remote to provision a github git repository backend for your
 - you have a github account (with username)
 - you have created a (40 character hexadecimal) github access token
 - you have inserted this data at a suitable chapter and verse
-- you run **`safe remote --provision`** from this opened book/chapter/verse
 
 Below is the sequence of commands leading up to **`safe remote --provision`**
 

data/lib/model/book.rb

@@ -384,13 +384,25 @@ module SafeDb
     end
 
 
+
+    # Print a notie stating the book followed by and open chapter and
+    # verse names only if the book is currently opened at a specific
+    # chapter and verse. Right after book creation for example there
+    # is no open chapter or verse and this method simply prints out
+    # a carriage return.
     def print_book_mark()
-      bcv_name = "#{book_name()}/#{get_open_chapter_name()}/#{get_open_verse_name()}"
+
       puts ""
+      return unless is_opened?()
+
+      bcv_name = "#{book_name()}:#{get_open_chapter_name()}/#{get_open_verse_name()}"
       puts "#{bcv_name} (#{get_open_verse_data().length()})\n"
       puts ""
+
     end
 
+
+
     # Is the verse name in the parameter the book's open verse? An exception
     # is thrown if the parameter verse name is nil.
     # @param this_verse_name [String] the name of the verse to test

data/lib/model/checkin.feature

@@ -1,33 +1,21 @@
 
-## The typical shells
+Feature: test safedb's version command
 
-Learn to run emacs commands in a batch so that we can open the below in an automated fashion. Learn to robotically communicate with and manipulate emacs. This will improve your efficiency many many fold.
+    This test will run both `safe version` and `safe --version`
 
-.%  CORE [dir]              856 Dired by name    /home/apollo/projects/safedb.net/lib/core/
-  * SEARCH [shell]           36 Shell:run        /home/apollo/projects/safedb.net/
-  * RAKE [shell]             94 Shell:run        /home/apollo/projects/safedb.net/
- %  USECASE [dir]          1080 Dired by name    /home/apollo/projects/safedb.net/lib/usecase/
- %  safedb.net              924 Dired by name    /home/apollo/projects/safedb.net/
-  * SAFE 2 [shell]           16 Shell:run        /home/apollo/
-  * SAFE 1 [shell]           16 Shell:run        /home/apollo/
+# -->     Scenario: hip hop hoooray is the value I GET
+# -->         When I run `safe version`
+# -->         Then the output should contain "v0.8"
 
-# Feature Checkin
+# -->       Scenario: right overy here is the blood and the sweat
+# -->         When I run `safe --version`
+# -->         Then the output should contain "v0.8"
 
-## In one shell
 
-safe init boys
-safe login boys
-safe import ~/safedb.test.data.json
-safe view
-safe show (failure because no open chapter has been set)
-safe goto 3
-safe checkin
-
-
-## In Another shell
-
-safe login boys
-
-Failure occurs
-
-R{\370\204m\265^v}\276\223^RZ(\320^N\333e.time":"Wed Apr 10 09:06:37 2019","book.name":"boys","book.init.version":"safedb-v0.3.1012","book.chapter.keys":{"dbms.accounts":{"content.id":"00cxarwaia0s4g","content.iv":"VzbG5IBQEuFdpRFerzGqHPejVKdKekcl","chapter.key.crypt":"Rx5BWXXM9uK5cbiH006WupaHVciXL8q%XooFdogSazMdHnqlJHuulMoeBtZxxJse"},"aws.console.accounts":{"content.id":"r6sc5ope3n3sfi","content.iv":"Jgh4c2aeh2Qjwt@ZaTEJxfK@6A%dzUoz","chapter.key.crypt":"9t5xxjDgyN4jQGQyw%9CHvTzGqJU@MiOWFGFKpGAuz@@fT8FJVO2fjoQIHEjTDfa"},"ops":{"content.id":"uk2u9bnzpynizc","content.iv":"AnvTgyJPbZpBju8kSsT8es0NmHlv2j9d","chapter.key.crypt":"TYyASQ71sgMbzNOxGVgojdb28M1%4iPDDsSDQz8TevjABYY887hqoPTRpvx18xUT"},"git":{"content.id":"r3dso6qr88l5lm","content.iv":"I9hykwLIGrvgp0j2jMQyMJC3QhWg0xSD","chapter.key.crypt":"0QLO7FawspMzEpRnB%6Wb3dVyjiJ36IX5SRp7QNOkU5ut@sMy9%KNYQ42XKA8yTj"}},"book.open.chapter":"aws.console.accounts","book.open.verse":"aimbrain.prod"}
+# --> safe init boys
+# --> safe login boys
+# --> safe import ~/safedb.test.data.json
+# --> safe view
+# --> safe show (failure because no open chapter has been set)
+# --> safe goto 3
+# --> safe checkin

data/lib/model/content.rb

@@ -11,6 +11,24 @@ module SafeDb
   class Content
 
 
+    # Use the content's external id to find the ciphertext file that is to be unlocked.
+    # Then use the unlock key from the parameter along with the random IV that is inside
+    # the {DataMap} or {DataStore} to decrypt and return the ciphertext.
+    #
+    # @param unlock_key [Key] symmetric key that was used to encrypt the ciphertext
+    # @param data_store [DataMap] either {DataMap} or {DataStore} containing content id and random iv
+    # @return [String] the resulting decrypted text that was encrypted with the parameter key
+    def self.unlock_master( unlock_key, data_store )
+
+      book_id = data_store.section()
+      crypt_path = FileTree.master_crypts_filepath( book_id, data_store.get( Indices::CONTENT_IDENTIFIER ) )
+      random_iv = KeyIV.in_binary( data_store.get( Indices::CONTENT_RANDOM_IV ) )
+      return unlock_it( crypt_path, unlock_key, random_iv )
+
+    end
+
+
+
     # Lock the content body provided - place the resulting ciphertext
     # inside a file named by a random identifier, then write this identifier
     # along wih the initialization and encryption key into the provided
@@ -19,6 +37,8 @@ module SafeDb
     # The content ciphertext derived from encrypting the body is stored
     # in a file underneath the provided content header.
     #
+    # Finally 
+    #
     # @param book_id [String] used to determine the book's master crypt folder
     # @param crypt_key [Key] the key used to (symmetrically) encrypt the content provided
     # @param data_store [DataMap] either DataMap or DataStore containing the content id and random iv
@@ -79,16 +99,12 @@ module SafeDb
 
 
 
-    # Lock the content body provided - place the resulting ciphertext
-    # inside a file named by a random identifier, then write this identifier
-    # along wih the initialization and encryption key into the provided
-    # key-value map (hash).
-    #
-    # The content ciphertext derived from encrypting the body is stored
-    # in a file underneath the provided content header.
+    # Use the crypt key and random init vector to transform the plain text
+    # message into an encrypted ciphertext message.
     #
-    # This method returns the highly random key instantiated for the purposes
-    # of encrypting the content.
+    # Use the chunk of plain text (header) information along with the ciphertext
+    # and the expected file location at the crypt path to write out the crypt
+    # and header content into a file.
     #
     # @param crypt_path [File] path to the crypt file holding the encrypted ciphertext
     # @param crypt_key [Key] the key used to (symmetrically) encrypt the content provided
@@ -104,24 +120,6 @@ module SafeDb
 
 
 
-    # Use the content's external id to find the ciphertext file that is to be unlocked.
-    # Then use the unlock key from the parameter along with the random IV that is inside
-    # the {DataMap} or {DataStore} to decrypt and return the ciphertext.
-    #
-    # @param unlock_key [Key] symmetric key that was used to encrypt the ciphertext
-    # @param data_store [DataMap] either {DataMap} or {DataStore} containing content id and random iv
-    # @return [String] the resulting decrypted text that was encrypted with the parameter key
-    def self.unlock_master( unlock_key, data_store )
-
-      book_id = data_store.section()
-      crypt_path = FileTree.master_crypts_filepath( book_id, data_store.get( Indices::CONTENT_IDENTIFIER ) )
-      random_iv = KeyIV.in_binary( data_store.get( Indices::CONTENT_RANDOM_IV ) )
-      return unlock_it( crypt_path, unlock_key, random_iv )
-
-    end
-
-
-
     # Use the content's external id to find the ciphertext file that is to be unlocked.
     # Then use the unlock key from the parameter along with the random IV that is inside
     # the {DataMap} or {DataStore} to decrypt and return the ciphertext.

data/lib/model/indices.rb

@@ -13,11 +13,14 @@ module SafeDb
     # The short url name of the safe personal database.
     SAFE_URL_NAME = "safedb.net"
 
+    # The name of the safe application and the safe gem as known by the operating system.
+    SAFE_APP_NAME = "safedb"
+
     # The desired length of a safe book ergonomic identifier.
     SAFE_BOOK_ID_LENGTH = 12
 
     # The file-system location of the safe database tree
-    SAFE_DATABASE_FOLDER = File.join( Dir.home, ".#{SAFE_URL_NAME}" )
+    SAFE_DATABASE_FOLDER = File.join( File.join( Dir.home, ".config" ), SAFE_APP_NAME )
 
     # The fully qualified domain name of the safedb home website
     SAFE_GEM_WEBSITE = "https://www.#{SAFE_URL_NAME}"
@@ -25,6 +28,11 @@ module SafeDb
     # The safe database github clonable url for the ruby software
     SAFE_GITHUB_URL = "https://github.com/devops4me/#{SAFE_URL_NAME}"
 
+
+    ### #################################### ###
+    ### paths to the master crypts resources ###
+    ### #################################### ###
+
     # The name of the master crypts folder.
     MASTER_CRYPTS_FOLDER_NAME = "safedb-master-crypts"
 
@@ -34,6 +42,23 @@ module SafeDb
     # The path to the master crypts .git directory.
     MASTER_CRYPTS_GIT_PATH = File.join( MASTER_CRYPTS_FOLDER_PATH, ".git" )
 
+    # The master indices file name
+    MASTER_INDICES_FILE_NAME = "safedb-master-keys.ini"
+
+    # The path to the master indices file
+    MASTER_INDICES_FILEPATH = File.join( MASTER_CRYPTS_FOLDER_PATH, MASTER_INDICES_FILE_NAME )
+
+
+
+    # The name of the backup master crypts folder.
+    BACKUP_CRYPTS_FOLDER_NAME = "safedb-backup-crypts"
+
+    # The path of the backup master crypts folder.
+    BACKUP_CRYPTS_FOLDER_PATH = File.join( SAFE_DATABASE_FOLDER, BACKUP_CRYPTS_FOLDER_NAME )
+
+
+
+
     # The name of the branch indices folder.
     BRANCH_INDICES_FOLDER_NAME = "safedb-branch-keys"
 
@@ -46,12 +71,6 @@ module SafeDb
     # The path to the branch crypts folder.
     BRANCH_CRYPTS_FOLDER_PATH = File.join( SAFE_DATABASE_FOLDER, BRANCH_CRYPTS_FOLDER_NAME )
 
-    # The master indices file name
-    MASTER_INDICES_FILE_NAME = "safedb-master-keys.ini"
-
-    # The path to the master indices file
-    MASTER_INDICES_FILEPATH = File.join( SAFE_DATABASE_FOLDER, MASTER_INDICES_FILE_NAME )
-
     # The path to the remote storage configuration INI file
     MACHINE_CONFIG_FILEPATH = File.join( SAFE_DATABASE_FOLDER, "safedb-remote-storage.ini" )
 
@@ -60,6 +79,7 @@ module SafeDb
 
 
 
+
     # The keyname whose value denotes a local folder path to clone to
     GIT_CLONE_BASE_PATH = "git.clone.base.path"
 
@@ -207,7 +227,7 @@ module SafeDb
     OPENED_VERSE_NAME = "book.open.verse"
 
     # The application version that oversaw this book's initialization.
-    SAFE_VERSION_STRING = "safedb-v#{SafeDb::VERSION}"
+    SAFE_PRE_VERSION_STRING = "safedb-v"
 
     # Handle to the key name of the ingested file in the submap verse
     INGESTED_FILE_LINE_NAME_KEY = "safedb.file::"
@@ -231,6 +251,13 @@ module SafeDb
     ELLIPTIC_CURVE_KEY_TYPE = "secp384r1"
 
 
+    ### ##################################### ###
+    ### Strings printed to the user interface ###
+    ### ##################################### ###
+
+    NOTHING_TO_OBLITERATE = "There is nothing to obliterate."
+
+
   end
 
 

data/lib/model/state_evolve.rb

@@ -68,12 +68,26 @@ module SafeDb
       the_crypt_key = old_human_key.do_decrypt_key( book_keys.get( Indices::CRYPT_CIPHER_TEXT ) )
       plain_content = Content.unlock_master( the_crypt_key, book_keys )
 
+      remove_crypt_path = FileTree.master_crypts_filepath( the_book_id, book_keys.get( Indices::CONTENT_IDENTIFIER ) )
+
       first_login_since_boot = StateInspect.is_first_login?( book_keys )
       the_crypt_key = Key.from_random if first_login_since_boot
       recycle_keys( the_crypt_key, the_book_id, human_secret, book_keys, plain_content )
       set_bootup_id( book_keys ) if first_login_since_boot
 
+      create_crypt_path = FileTree.master_crypts_filepath( the_book_id, book_keys.get( Indices::CONTENT_IDENTIFIER ) )
       branch_id = Identifier.derive_branch_id( Branch.to_token() )
+      commit_msg = "safe login to #{the_book_id} at branch #{branch_id} on #{TimeStamp.readable()}."
+
+      # Remove the master chapter crypt file from the local git repository and add
+      # the new master chapter crypt to the local git repository.
+      GitFlow.del_file( Indices::MASTER_CRYPTS_FOLDER_PATH, remove_crypt_path )
+      GitFlow.add_file( Indices::MASTER_CRYPTS_FOLDER_PATH, create_crypt_path )
+      GitFlow.add_file( Indices::MASTER_CRYPTS_FOLDER_PATH, Indices::MASTER_INDICES_FILEPATH )
+      GitFlow.list( Indices::MASTER_CRYPTS_FOLDER_PATH )
+      GitFlow.list( Indices::MASTER_CRYPTS_FOLDER_PATH, true )
+      GitFlow.commit( Indices::MASTER_CRYPTS_FOLDER_PATH, commit_msg )
+
       clone_book_into_branch( the_book_id, branch_id, book_keys, the_crypt_key )
 
       return true
@@ -177,6 +191,13 @@ module SafeDb
       master_keys.set( Indices::CONTENT_IDENTIFIER, branch_keys.get( Indices::CONTENT_IDENTIFIER ) )
       master_keys.set( Indices::CONTENT_RANDOM_IV,  branch_keys.get( Indices::CONTENT_RANDOM_IV  ) )
 
+      commit_msg = "safe commit for #{book.book_name()} in branch #{book.branch_id()} on #{TimeStamp.readable()}."
+
+      GitFlow.stage( Indices::MASTER_CRYPTS_FOLDER_PATH )
+      GitFlow.list( Indices::MASTER_CRYPTS_FOLDER_PATH )
+      GitFlow.list( Indices::MASTER_CRYPTS_FOLDER_PATH, true )
+      GitFlow.commit( Indices::MASTER_CRYPTS_FOLDER_PATH, commit_msg )
+
     end
 
 

data/lib/model/text_chunk.rb

@@ -21,7 +21,7 @@ module SafeDb
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Safe Book Id := #{book_id}
 Time Created := #{TimeStamp.readable()}
-Safe Version := #{Indices::SAFE_VERSION_STRING}
+Safe Version := #{Indices::SAFE_PRE_VERSION_STRING}#{SafeDb::VERSION}
 Safe Website := #{Indices::SAFE_GEM_WEBSITE}
 RubyGems.org := https://rubygems.org/gems/safedb
 CRYPT_HEADER

data/lib/utils/extend/string.rb

@@ -561,6 +561,17 @@ class String
   # The INFO log level is used to log the lines - if this is not
   #   appropriate create a (level) parameterized log lines method.
   def log_lines
+    log_info()
+  end
+
+
+  # Log at the INFO level the string which is expected to be
+  # delineated. If the string originated from a file it will
+  # be logged line by line.
+  #
+  # If no line delineation the string will be dumped just as
+  # a blob.
+  def log_info
 
     self.each_line do |line|
       clean_line = line.chomp.gsub("\\n","")
@@ -569,4 +580,21 @@ class String
 
   end
 
+
+  # Log at the INFO level the string which is expected to be
+  # delineated. If the string originated from a file it will
+  # be logged line by line.
+  #
+  # If no line delineation the string will be dumped just as
+  # a blob.
+  def log_debug
+
+    self.each_line do |line|
+      clean_line = line.chomp.gsub("\\n","")
+      log.debug(x) { line } if clean_line.length > 0
+    end
+
+  end
+
+
 end