opensecret 0.0.9925 → 0.0.9949

data/lib/usecase/jenkins/README.md

@@ -0,0 +1,146 @@
+
+# safe jenkins <command>
+
+
+### safe jenkins post [aws|docker|git] <<jenkins-host-url>> | introduction
+
+Use **`safe jenkins post`** to inject both your **AWS IAM User** and **docker login/password** credentials into your Jenkins 2.0 continuous integration portal reachable by the **jenkins host url** given in the 4th parameter of the safe command.
+
+---
+
+## safe jenkins post | prerequisite
+
+Before you can inject credentials into jenkins using **`safe jenkins post`** you must
+
+- be logged into your safe
+- have opened the appropriate chapter/verse
+- have put the required credential key/value pairs into the safe
+- have the jenkins service up and running
+
+After the post (to jenkins), your continuous integration jobs will be able to access the credential values via their IDs as stated in the below table.
+
+---
+
+## safe jenkins post aws | key names table
+
+As credentials are WORO (write once, read often), safe makes the reading part very very easy (and secure) so your effort is frontloaded.
+
+| Safe Key    | Jenkins Credential IDs | Environment Variable  | Description                                              |
+|:-----------:|:----------------------:|:--------------------- |:-------------------------------------------------------- |
+| @access.key | safe.aws.access.key    | AWS_ACCESS_KEY_ID     | The AWS IAM user's access key credential.                |
+| @secret.key | safe.aws.secret.key    | AWS_SECRET_ACCESS_KEY | The AWS IAM user's secret key credential.                |
+| region.key  | safe.aws.region.key    | AWS_REGION            | The AWS region key that your Jenkins service points to.  |
+
+So you can see that by convention, safe expects the credential keys in the safe to be named a particular way, and likewise, you can be assured of the IDs it gives those credentials when posted to Jenkins.
+
+
+## safe jenkins post | credentials lifecycle
+
+The life of the credentials begins when you create an  IAM user and record its access and secret keys. Then
+
+- you login to safe and store the 3 keys and their values
+- safe jenkins post will read the values and post them to Jenkins
+- Jenkins stores the values in conjunction with the Jenkins Credential IDs
+- pipeline jobs ask Jenkins to put the Credential ID values against environment variables
+- tools like Terraform and AwsCli use the environment variables to work in the cloud
+
+
+## Jenkinsfile | Usage in Pipeline Jobs
+
+Here is a pipeline declaration within a Jenkinsfile that asks Jenkins to put the credential values in its secrets store into the stated environment variables.
+
+    environment
+    {
+        AWS_ACCESS_KEY_ID     = credentials( 'safe.aws.access.key' )
+        AWS_SECRET_ACCESS_KEY = credentials( 'safe.aws.secret.key' )
+        AWS_REGION            = credentials( 'safe.aws.region.key' )
+    }
+
+After **`safe jenkins post aws`** you can **click into the Credentials item in the Jenkins main menu** to assure yourself that the credentials have indeed been properly injected.
+
+---
+
+## How to Write AWS Credentials into your Safe
+
+In order to **`safe terraform apply`** or **`safe jenkins post aws <<jenkins-host-url>>`** or `safe visit` you must first put those ubiquitous IAM programmatic user credentials into your safe.
+
+    $ safe login joebloggs.com                  # open the book
+
+    $ safe open iam dev.s3.reader               # open chapter and verse
+    $ safe put @access.key ABCD1234EFGH5678     # Put IAM access key in safe
+    $ safe put @secret.key xyzabcd1234efgh5678  # Put IAM secret key in safe
+    $ safe put region.key eu-west-3             # infrastructure in Paris
+
+    $ safe open iam canary.admin                # open chapter and verse
+    $ safe put @access.key 4321DCBA8765WXYZ     # Put IAM access key in safe
+    $ safe put @secret.key 5678uvwx4321abcd9876 # Put IAM secret key in safe
+    $ safe put region.key eu-west-1             # infrastructure in Dublin
+
+    $ safe logout
+
+
+---
+
+
+## How to write DockerHub Credentials into your Safe
+
+#### safe jenkins post docker https://jenkins.example.com
+
+Before you can issue a **`safe jenkins post docker http://localhost:8080`** you must insert your docker login credentials in the form of a username and @password into your safe. Remember that any key starting with the `@ sign` tells the safe to keep it a secret like when you issue a **`safe show`** command.
+
+    $ safe login joebloggs.com        # open the book
+    $ safe open docker production     # at the docker (for production) chapter and verse
+    $ safe put username admin         # Put the Docker repository login username into the safe
+    $ safe put @password secret12345  # Put the Docker repository login @password into the safe
+    $ safe logout
+
+When docker credentials are injected into a Jenkins service the safe will expect to find a key at the open chapter and verse called username and another one called password.
+
+The safe promises to inject credentials with an ID of **safe.docker.login.id** so any jenkins jobs that need to use the docker login username and password must specify this ID when talking to the Jenkins credentials service.
+
+
+### DockerHub Credentials Inject Response
+
+Here is an example of posting dockerhub credentials into a Jenkins service running on the local machine.
+
+``` bash
+safe jenkins post docker http://localhost:8080
+```
+
+If successful safe provides a polite response detailing what just happened.
+
+```
+ - Jenkins Host Url : http://localhost:8080/credentials/store/system/domain/_/createCredentials
+ -   Credentials ID : safe.docker.login.id
+ -  Inject Username : devops4me
+ - So what is this? : The docker repository login credentials in the shape of a username and password.
+
+  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
+                                 Dload  Upload   Total   Spent    Left  Speed
+100   428    0     0  100   428      0  47555 --:--:-- --:--:-- --:--:-- 47555
+```
+
+---
+
+
+## safe integrations | we need your help
+
+**You can help to extend safe's integrations.**
+
+By design - safe integrations are simple to write. They primarily integrate with producers and consumers. To deliver efficacy to devops engineers safe will endeavour to
+
+- **send** credentials to **downstream consumers** and
+- **receive** credentials from **upstream producers**
+
+safe needs pull requests from the devops community and it promises to always strive to keep the task of writing an integration extremely simple.
+
+### integrations | what giving takes?
+
+Currently, writing an integration entails delivering 3 or 4 artifacts which are
+
+- 1 simple Ruby class
+- 1 README.md documenting the command structure, the prerequisites and the expected outcome
+- 1 class containing unit tests
+- (optionaly) an INI file if many configuration and facts are involved
+
+Giving doesn't take much so roll up your sleeves (or frocks) and get writing.

data/lib/usecase/jenkins/crazy_ruby_post_attempt.OLD

@@ -0,0 +1,234 @@
+#!/usr/bin/ruby
+	
+module OpenSecret
+
+    # This Jenkins use case handles the to and fro integration of secrets and sensitive information
+    # between the safe database under management and a Jenkins service pinpointed by an incoming 
+    # host url parameter.
+    #
+    # This Jenkins use case injects for example the AWS IAM user access key, secret key and region key
+    # into a running Jenkins CI (Continuous Integration) service at the specified (url) location.
+    #
+    #     safe jenkins post <<[ aws | docker | git ]>> <<jenkins-host-url>>
+
+    class Jenkins < UseCase
+
+        attr_writer :command, :service, :url
+
+        JENKINS_URL_POSTFIX = "credentials/store/system/domain/_/createCredentials"
+        JENKINS_URL_PATH = "/credentials/store/system/domain/_/createCredentials"
+        REQUEST_CONTENT_TYPE = 'application/json;charset=UTF-8'
+
+        DATA_DICTIONARY = {
+                    "scope"       => "GLOBAL",
+                    "id"          => "safe.aws.region.key",
+                    "secret"      => "blahblahblah",
+                    "description" => "The AWS region key for example eu-west-1 for Dublin in Ireland.",
+                    "$class"      => "org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl"
+                }
+
+###        DICTIONARY_TO_POST = { "" => "0", "credentials" => DATA_DICTIONARY }
+        DICTIONARY_TO_POST = { "credentials" => DATA_DICTIONARY }
+
+        def json_api_post_request_2
+
+            require 'net/http'
+            require 'json'
+
+            http = Net::HTTP.new( "localhost", "8080" )
+
+###########            service_url = File.join( @url, JENKINS_URL_POSTFIX )
+
+            request = Net::HTTP::Post.new( JENKINS_URL_PATH )
+            uri_encoded_json = URI::encode( DICTIONARY_TO_POST.to_json )
+            request.body = uri_encoded_json
+            request.content_type = REQUEST_CONTENT_TYPE
+
+            response = http.request( request )
+
+###            uri = URI.parse( "#{service_url}" )
+###            req = Net::HTTP::Post.new(uri)
+###            req.set_form_data('from' => '2005-01-01', 'to' => '2005-03-31')
+
+###            res = Net::HTTP.start(uri.hostname, uri.port) do |http|
+###              http.request(req)
+###            end
+
+            puts response.body
+
+        end
+
+
+        def json_api_post_request_1
+
+            require 'net/http'
+            require 'json'
+
+            begin
+                service_url = File.join( @url, JENKINS_URL_POSTFIX )
+
+                puts ""
+                puts "The service_url is #{service_url}"
+
+                uri = URI( service_url )
+
+                puts "The simple URI hostname is #{uri.host}"
+                puts "Data will be sent via the URI port numbered [#{uri.port}]"
+                puts "The URI path string is #{uri.path}"
+
+                http = Net::HTTP.new(uri.host, uri.port)
+####                req = Net::HTTP::Post.new(uri.path, {'Content-Type' =>'application/json',  
+####                                                     'Authorization' => 'XXXXXXXXXXXXXXXX'})
+    
+
+###                our_request = Net::HTTP::Post.new( uri.path, { 'Content-Type' =>'application/json' } )
+##############                our_request.body = {"pizza_type" => "Margherita", "pizza_no" => "2"}.to_json
+
+                jenkins_inner_dictionary = {
+                    "scope"       => "GLOBAL",
+                    "id"          => "safe.aws.region.key",
+                    "secret"      => "blahblahblah",
+                    "description" => "The AWS region key for example eu-west-1 for Dublin in Ireland.",
+                    "$class"      => "org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl"
+                }
+
+                jenkins_outer_dictionary = { "" => "0", "credentials" => jenkins_inner_dictionary }
+
+                our_request = Net::HTTP::Post.new( uri.path )
+                our_request.set_form_data( jenkins_outer_dictionary.to_json )
+#####                our_request.set_form_data( jenkins_inner_dictionary )
+
+
+
+####                jenkins_jason_dictionary = jenkins_outer_dictionary.to_json
+####                our_request.body = jenkins_jason_dictionary
+
+
+
+## http = Net::HTTP.new("api.restsite.com")
+
+## request = Net::HTTP::Post.new("/users")
+## request.set_form_data({"users[login]" => "quentin"})
+## response = http.request(request)
+
+
+
+## DELETE ME
+## DELETE ME
+## DELETE ME
+                puts "The [[[ SENSITIVE ]] JSON request body is this.\n\n #{our_request.body}"
+                puts ""
+
+                res = http.request( our_request )
+
+                puts res.body
+#####                puts JSON.parse(res.body)
+
+            rescue => e
+
+                puts "failed #{e}"
+
+            end
+
+        end
+
+    def execute
+
+      return unless ops_key_exists?
+      master_db = get_master_database()
+      return if unopened_envelope?( master_db )
+
+      # Get the open chapter identifier (id).
+      # Decide whether chapter already exists.
+      # Then get (or instantiate) the chapter's hash data structure
+      chapter_id = ENVELOPE_KEY_PREFIX + master_db[ ENV_PATH ]
+      verse_id = master_db[ KEY_PATH ]
+      chapter_exists = OpenKey::KeyApi.db_envelope_exists?( master_db[ chapter_id ] )
+
+      # Unlock the chapter data structure by supplying
+      # key/value mini-dictionary breadcrumbs sitting
+      # within the master database at the section labelled
+      # envelope@<<actual_chapter_id>>.
+      chapter_data = OpenKey::KeyDb.from_json( OpenKey::KeyApi.content_unlock( master_db[ chapter_id ] ) )
+
+      # Now read the three AWS IAM credentials @access.key, @secret.key and region.key
+      # into the 3 environment variables terraform expects to find.
+
+      # ----------------------------------------------------------------------------------------------------- #
+      # ----------------------------------------------------------------------------------------------------- #
+
+      # -- 
+      # --  Reading material for CUrl like activities using net/http core ruby library
+      # -- 
+      # --     ssl/https/rest  =>  http://www.rubyinside.com/nethttp-cheat-sheet-2940.html
+      # --     official docs   =>  https://ruby-doc.org/stdlib-2.4.1/libdoc/net/http/rdoc/Net/HTTP.html
+      # --                     =>  
+      # --                     =>  
+      # --                     =>  
+      # --                     =>  
+      # -- 
+
+      # ----------------------------------------------------------------------------------------------------- #
+      # ----------------------------------------------------------------------------------------------------- #
+
+
+      json_api_post_request_2
+
+      puts ""
+      puts "@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@"
+      puts "@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@"
+      puts ""
+
+=begin
+      ENV[ "AWS_ACCESS_KEY_ID"     ] = chapter_data[ verse_id ][ "@access.key" ]
+      ENV[ "AWS_SECRET_ACCESS_KEY" ] = chapter_data[ verse_id ][ "@secret.key" ]
+      ENV[ "AWS_DEFAULT_REGION"    ] = chapter_data[ verse_id ][ "region.key"  ]
+
+      auto_approve = @command && @command.eql?( "plan" ) ? "" : "-auto-approve"
+      command_name = @command ? @command : "apply"
+
+      puts ""
+      puts "@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@"
+      puts ""
+
+      system "terraform #{command_name} #{auto_approve}"
+
+      puts ""
+      puts "@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@"
+      puts ""
+=end
+
+      # ############## | ############################################################
+      # @todo refactor | ############################################################
+      # -------------- | 000000000000000000000000000000000000000000000000000000000000
+      # export-then-execute
+      # -------------------
+      # Refactor all this code into a generic export-then-execute use case
+      # Then you pass in a Key/Value Dictionary
+      #
+      # { "AWS_ACCESS_KEY_ID" => "@access_key",
+      #   "AWS_SECRET_ACCESS_KEY" => "@secret_key",
+      #   "AWS_DEFAULT_REGION" => "region_key"
+      # }
+      #
+      # And pass in a command array [ "terraform #{command_name} #{auto_approve}", "terraform graph ..." ]
+      #
+      # Validation is done by the generic use case (which loops checking that every value exists
+      # as a key at the opened location.
+      #
+      # If all good the generic use case exports the ENV vars and runs each command in the list.
+      # PS - configure map in INI not code file
+      #
+      # The extra power will speed up generation of environment variable use cases including
+      # ansible, s3 bucket operations, git interactions and more.
+      #
+      # ############## | ############################################################
+      # ############## | ############################################################
+
+    end
+
+
+  end
+
+
+end

data/lib/usecase/jenkins/jenkins.rb

@@ -0,0 +1,208 @@
+#!/usr/bin/ruby
+	
+module OpenSecret
+
+    # This Jenkins use case handles the to and fro integration of secrets and sensitive information
+    # between the safe database under management and a Jenkins service pinpointed by an incoming 
+    # host url parameter.
+    #
+    # This Jenkins use case injects for example the AWS IAM user access key, secret key and region key
+    # into a running Jenkins CI (Continuous Integration) service at the specified (url) location.
+    #
+    #     safe jenkins post <<[ aws | docker | git ]>> <<jenkins-host-url>>
+
+    class Jenkins < UseCase
+
+        # The three instance variables provided through the command line like
+        # for example  $ safe jenkins post aws http://localhost:8080
+        # For more info visit the documentation in the command interpreter class.
+        attr_writer :command, :service, :url
+
+        # If string variables EXPLODE throughout (and come to dominate) this class
+        # we should consider introducing an INI factfile like the [vpn] use case.
+        JENKINS_URI_PATH = "credentials/store/system/domain/_/createCredentials"
+
+        # If string variables EXPLODE throughout (and come to dominate) this class
+        # we should consider introducing an INI factfile like the [vpn] use case.
+        SECRET_KEY_VALUE_PAIR_DICTIONARY =
+          {
+            "scope"       => "GLOBAL",
+            "$class"      => "org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl"
+          }
+
+        # If string variables EXPLODE throughout (and come to dominate) this class
+        # we should consider introducing an INI factfile like the [vpn] use case.
+        SECRET_KEY_VALUE_PAIR_TO_POST = { "" => "0", "credentials" => SECRET_KEY_VALUE_PAIR_DICTIONARY }
+
+
+        # If string variables EXPLODE throughout (and come to dominate) this class
+        # we should consider introducing an INI factfile like the [vpn] use case.
+        USERNAME_AND_PASSWORD_DICTIONARY =
+          {
+            "scope"       => "GLOBAL",
+            "$class"      => "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
+          }
+
+        # If string variables EXPLODE throughout (and come to dominate) this class
+        # we should consider introducing an INI factfile like the [vpn] use case.
+        USERNAME_AND_PASSWORD_TO_POST = { "" => "0", "credentials" => USERNAME_AND_PASSWORD_DICTIONARY }
+
+
+
+        # Inject a Jenkins credential key-value pair that is secret and/or sensitive and
+        # needs to be referenced by executing continuous integration jobs.
+        #
+        # @param jenkins_base_url [String]
+        #
+        #    This base url includes the scheme (protocol) which can be either http
+        #    or https. It can include the port if it is not either 80 or 443. A common
+        #    example is http://localhost:8080 but can also be https://jenkins.example.com
+        #    It pays not to provide a trailing backslash on this url.
+        #
+        # @param credentials_id [String]
+        #
+        #    The ID that Jenkins jobs will use to reference this credential's value.
+        #
+        # @param secret_value [String]
+        #
+        #    The value of this credential (secret) that will be injected for SafeKeeping
+        #    to the Jenkins service at the provided URL.
+        #
+        # @param description [String]
+        #
+        #    Description of the credential that will be posted and can be viewed via
+        #    the Jenkins user interface.
+        def inject_secret_key_value_pair( jenkins_base_url, credentials_id, secret_value, description )
+
+            jenkins_url = File.join( jenkins_base_url, JENKINS_URI_PATH )
+
+            credentials_dictionary = SECRET_KEY_VALUE_PAIR_DICTIONARY
+            credentials_dictionary.store( "id", credentials_id )
+            credentials_dictionary.store( "secret", secret_value )
+            credentials_dictionary.store( "description", description )
+
+            curl_cmd = "curl -X POST '#{jenkins_url}' --data-urlencode 'json=#{SECRET_KEY_VALUE_PAIR_TO_POST.to_json}'"
+
+            puts ""
+            puts " - Jenkins Host Url : #{jenkins_url}"
+            puts " -   Credentials ID : #{credentials_id}"
+            puts " - So what is this? : #{description}"
+            puts ""
+
+            %x[ #{curl_cmd} ]
+
+            puts ""
+
+        end
+
+
+
+        # Inject into Jenkins a username and password pairing against an ID key that the
+        # continuous integration jobs know and can use to access the credentials pair.
+        #
+        # @param jenkins_base_url [String]
+        #
+        #    This base url includes the scheme (protocol) which can be either http
+        #    or https. It can include the port if it is not either 80 or 443. A common
+        #    example is http://localhost:8080 but can also be https://jenkins.example.com
+        #    It pays not to provide a trailing backslash on this url.
+        #
+        # @param credentials_id [String]
+        #
+        #    The ID that Jenkins jobs will use to reference this credential's value.
+        #
+        # @param username [String]
+        #
+        #    The value of this username (secret) that will be injected for SafeKeeping
+        #    to the Jenkins service at the provided URL.
+        #
+        # @param password [String]
+        #
+        #    The value of this password (secret) that will be injected for SafeKeeping
+        #    to the Jenkins service at the provided URL.
+        #
+        # @param description [String]
+        #
+        #    Description of the username and password pairing that will be posted and
+        #    can be viewed via the Jenkins user interface.
+        def inject_username_and_password( jenkins_base_url, credentials_id, username, password, description )
+
+            jenkins_url = File.join( jenkins_base_url, JENKINS_URI_PATH )
+
+            credentials_dictionary = USERNAME_AND_PASSWORD_DICTIONARY
+            credentials_dictionary.store( "id", credentials_id )
+            credentials_dictionary.store( "username", username )
+            credentials_dictionary.store( "password", password )
+            credentials_dictionary.store( "description", description )
+
+            curl_cmd = "curl -X POST '#{jenkins_url}' --data-urlencode 'json=#{USERNAME_AND_PASSWORD_TO_POST.to_json}'"
+
+            puts ""
+            puts " - Jenkins Host Url : #{jenkins_url}"
+            puts " -   Credentials ID : #{credentials_id}"
+            puts " -  Inject Username : #{username}"
+            puts " - So what is this? : #{description}"
+            puts ""
+
+            %x[ #{curl_cmd} ]
+
+            puts ""
+
+        end
+
+
+
+        def execute
+
+            return unless ops_key_exists?
+            master_db = get_master_database()
+            return if unopened_envelope?( master_db )
+
+            # Get the open chapter identifier (id).
+            # Decide whether chapter already exists.
+            # Then get (or instantiate) the chapter's hash data structure
+            chapter_id = ENVELOPE_KEY_PREFIX + master_db[ ENV_PATH ]
+            verse_id = master_db[ KEY_PATH ]
+            chapter_exists = OpenKey::KeyApi.db_envelope_exists?( master_db[ chapter_id ] )
+
+            # Unlock the chapter data structure by supplying
+            # key/value mini-dictionary breadcrumbs sitting
+            # within the master database at the section labelled
+            # envelope@<<actual_chapter_id>>.
+            chapter_data = OpenKey::KeyDb.from_json( OpenKey::KeyApi.content_unlock( master_db[ chapter_id ] ) )
+
+            key_value_dictionary = chapter_data[ verse_id ]
+
+            inject_aws_credentials(    key_value_dictionary ) if @service.eql?( "aws" )
+            inject_docker_credentials( key_value_dictionary ) if @service.eql?( "docker" )
+
+        end
+
+
+
+        def inject_aws_credentials( mini_dictionary )
+
+            access_key_desc = "The access key of the AWS IAM (programmatic) user credentials."
+            secret_key_desc = "The secret key of the AWS IAM (programmatic) user credentials."
+            region_key_desc = "The AWS region key for example eu-west-1 for Dublin in Ireland."
+
+            inject_secret_key_value_pair( @url, "safe.aws.access.key", mini_dictionary[ "@access.key" ], access_key_desc )
+            inject_secret_key_value_pair( @url, "safe.aws.secret.key", mini_dictionary[ "@secret.key" ], secret_key_desc )
+            inject_secret_key_value_pair( @url, "safe.aws.region.key", mini_dictionary[ "region.key"  ], region_key_desc )
+
+        end
+
+
+        def inject_docker_credentials( mini_dictionary )
+
+            docker_desc = "The docker repository login credentials in the shape of a username and password."
+
+            inject_username_and_password( @url, "safe.docker.login.id", mini_dictionary[ "docker.username" ], mini_dictionary[ "@docker.password" ], docker_desc )
+
+        end
+
+
+    end
+
+
+end