kpm 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: cfdf51795bb3bfc5198c442337ecd902616e357c
-  data.tar.gz: 5221a817f4b30fc91fa9214c582136d5f00e7532
+  metadata.gz: 6819019089d4dd3219f4e68c7a55323addf194fe
+  data.tar.gz: 47f1774f8355f87424fab114b6bad542159a9beb
 SHA512:
-  metadata.gz: 3995b7421e68b14b4e932e27833202f49dc503433536a314a7a72d26b24b021b8dce192ce3fca680ac36a088cfe6eec04dd45e93ebdf74a4344f68c69eb4f287
-  data.tar.gz: 4087053ae2f9ce4673e8dfa3635331613242e8328bd98edb353c60a0d112806510971324e6b4572320ae2ec934a3c60a2361fb3df3ba7632d494c4edad905046
+  metadata.gz: e383b778cb444ede8ec0b1e4172f600fea63c47ee8b24e7f015727f68ff220cdb0fcdcca2dbd8f420f7c7e7b68909874155923bfb1c941bb7426a1b8d1b29db0
+  data.tar.gz: ee4fb38ad9686232f9302c892b7c1583698faee0c2cd582929bba9e0c5c0e55c26feb888b4f9eb66e0f8c8f27de1e6dc576b2996b3b27138922feff8b64b1e38

data/README.md

@@ -4,12 +4,29 @@ The goal of KPM is to facilitate the installation of Kill Bill, its plugins and
 
 kpm can be used interactively to search and download individual artifacts (Kill Bill war, plugins, etc.) or to perform an automatic Kill Bill installation using a configuration file.
 
+## Prerequisites
+
+### Ruby
+
+Ruby is required to run KPM itself (it is not a dependency of Kill Bill).
+
+Ruby 2.1+ or JRuby 1.7.20+ is recommended. If you don't have a Ruby installation yet, use [RVM](https://rvm.io/rvm/install):
+
+```
+gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
+\curl -sSL https://get.rvm.io | bash -s stable --ruby
+```
+
+After following the post-installation instructions, you should have access to the `ruby` and `gem` executables.
+
+### Java
+
+Kill Bill runs on the [Java](https://www.java.com/en/download/) platform, version 6 and above (8 is recommended).
+
 ## Installation
 
     gem install kpm
 
-Ruby 2.1+ or JRuby 1.7.11+ is recommended.
-
 ## Quick start
 
 The following commands
@@ -18,9 +35,9 @@ The following commands
     cd killbill
     kpm install
 
-will setup Kill Bill and Kaui, i.e.:
+will setup [Kill Bill](https://github.com/killbill/killbill) and [Kaui](https://github.com/killbill/killbill-admin-ui-standalone), i.e.:
 
-* Tomcat (open-source Java web server) is setup in the `killbill` directory
+* [Tomcat](http://tomcat.apache.org/) (open-source Java web server) is setup in the `killbill` directory
 * The Kill Bill application (war) is installed in the `killbill/webapps` directory
 * The Kill Bill UI (Kaui war) is installed in the `killbill/webapps` directory
 * Default plugins are installed in the `/var/tmp/bundles` directory, among them:
@@ -50,7 +67,7 @@ For example:
 This instructs kpm to:
 
 * Download Kill Bill version 0.14.0
-* Setup the Analytics (Java) plugin and the Stripe (Ruby) plugin
+* Setup the [Analytics](https://github.com/killbill/killbill-analytics-plugin) (Java) plugin and the [Stripe](https://github.com/killbill/killbill-stripe-plugin) (Ruby) plugin
 
 To start the installation:
 
@@ -69,16 +86,16 @@ There are many more options you can specify. Take a look at the configuration fi
 
 ### Caching
 
-KPM relies on the kpm.yml file to know what to install, and as it installs the pieces it keeps track of what was installed so that if it is invoked again, it does not download again the same binaries:
+KPM relies on the kpm.yml file to know what to install, and as it installs the pieces, it keeps track of what was installed so that if it is invoked again, it does not download again the same binaries:
 
 * For the Kill Bill artifact itself, kpm will first extract the sha1 from the remote repository and compare with the current sha1 installed under the default `webapp_path`; if those are the same, the file is not downloaded again.
-* For the plugins, since those are downloaded as an archive (*.tgz) and then decompressed/expanded, kpm will use an internal file, `<plugins_dir>/sha1.yml` to keep track of the sha1 archive. If there is an entry and the sha1 matches, then the file is not downloaded again.
+* For the plugins, since some of those are downloaded as an archive (*.tgz) and then decompressed/expanded, kpm will use an internal file, `<plugins_dir>/sha1.yml` to keep track of the sha1 archive. If there is an entry and the sha1 matches, then the file is not downloaded again.
 
 Note that you can override that behavior with the `--force-download` switch.
 
 ### Custom Downloads
 
-Note that you can also download specific versions/artifacts directly with the following commands -- bypassing the kpm.yml file:
+You can also download specific versions/artifacts directly with the following commands -- bypassing the kpm.yml file:
 
 * `kpm pull_kaui_war <version>`
 * `kpm pull_kb_server_war <version>`

data/lib/kpm/base_installer.rb

@@ -7,6 +7,9 @@ module KPM
     SHA1_FILENAME = 'sha1.yml'
     DEFAULT_BUNDLES_DIR = Pathname.new('/var').join('tmp').join('bundles').to_s
 
+
+
+
     def initialize(logger, nexus_config, nexus_ssl_verify)
       @logger = logger
       @nexus_config = nexus_config
@@ -20,7 +23,6 @@ module KPM
       classifier = specified_classifier || KPM::BaseArtifact::KILLBILL_CLASSIFIER
       version = specified_version || LATEST_VERSION
       webapp_path = specified_webapp_path || KPM::root
-
       @logger.debug("Installing Kill Bill server: group_id=#{group_id} artifact_id=#{artifact_id} packaging=#{packaging} classifier=#{classifier} version=#{version} webapp_path=#{webapp_path}")
       KPM::KillbillServerArtifact.pull(@logger,
                                        group_id,
@@ -59,33 +61,60 @@ module KPM
                              @nexus_ssl_verify)
     end
 
-    def install_plugin(specified_group_id, specified_artifact_id, specified_packaging=nil, specified_classifier=nil, specified_version=nil, bundles_dir=nil, specified_type='java', force_download=false, verify_sha1=true)
-      looked_up_group_id, looked_up_artifact_id, looked_up_packaging, looked_up_classifier, looked_up_version, looked_up_type = KPM::PluginsDirectory.lookup(specified_artifact_id, true)
 
-      # Specified parameters have always precedence except for the artifact_id (to map stripe to stripe-plugin)
-      artifact_id = looked_up_artifact_id || specified_artifact_id
-      if artifact_id.nil?
-        @logger.warn("Aborting installation: unable to lookup plugin #{specified_artifact_id}")
+    def install_plugin(plugin_key, specified_group_id, specified_artifact_id, specified_packaging=nil, specified_classifier=nil, specified_version=nil, bundles_dir=nil, specified_type='java', force_download=false, verify_sha1=true)
+
+      if plugin_key.nil?
+        @logger.warn("Aborting installation: User needs to specify a pluginKey")
         return nil
       end
 
+      # Since we allow to only specify a the artifact_id to identify a given plugin we set the default on the other fields
+      specified_group_id = specified_group_id || (specified_type.to_s == 'java' ? KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_GROUP_ID : KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_GROUP_ID)
+      specified_packaging = specified_packaging || (specified_type.to_s == 'java' ? KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_PACKAGING : KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_PACKAGING)
+      specified_classifier = specified_classifier || (specified_type.to_s == 'java' ? KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_CLASSIFIER : KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_CLASSIFIER)
+
+      # Create a Proc to validate and resolve arguments
+      resolve_and_validate_args_proc = Proc.new { |plugin_key, arg_type, specified_arg, lookup_arg|
+
+        # There is an entry in plugin_directory.yml but does not match with input
+        if lookup_arg && specified_arg && lookup_arg.to_s != specified_arg.to_s
+          @logger.warn("Aborting installation: User specified #{arg_type}=#{specified_arg} for pluginKey = #{plugin_key}, but plugin_directory.yml resolves with #{lookup_arg}")
+          return nil
+        end
+
+        # There is no entry and user did not specify anything (or we did not have any default)
+        if lookup_arg.nil? && specified_arg.nil?
+          @logger.warn("Aborting installation: need to specify an #{arg_type} for pluginKey = #{plugin_key}")
+          return nil
+        end
+        # If validation is successful we return resolved value
+        specified_arg || lookup_arg
+      }
+
+
+      # Lookup entry (will come with null everywhere if entry does not exist in plugin_directory.yml)
+      looked_up_group_id, looked_up_artifact_id, looked_up_packaging, looked_up_classifier, looked_up_version, looked_up_type = KPM::PluginsDirectory.lookup(plugin_key, true)
+
+      # If there is no entry in plugins_directory.yml, the key provided must be a user key and must have a namespace
+      if looked_up_artifact_id.nil? && plugin_key.split(':').size == 1
+        @logger.warn("Aborting installation: pluginKey = #{plugin_key} does not exist in plugin_directory.yml so format of the key must have a user namespace (e.g namespace:key)")
+        return nil
+      end
+
+      # Validate and resolve the value to use (user input has precedence)
+      group_id = resolve_and_validate_args_proc.call(plugin_key, 'group_id', specified_group_id, looked_up_group_id)
+      artifact_id = resolve_and_validate_args_proc.call(plugin_key, 'artifact_id', specified_artifact_id, looked_up_artifact_id)
+      packaging = resolve_and_validate_args_proc.call(plugin_key, 'packaging', specified_packaging, looked_up_packaging)
+      classifier = specified_classifier || looked_up_classifier
+      type = resolve_and_validate_args_proc.call(plugin_key, 'type', specified_type, looked_up_type).to_s
+      version = specified_version || looked_up_version || LATEST_VERSION
+
       bundles_dir = Pathname.new(bundles_dir || DEFAULT_BUNDLES_DIR).expand_path
       plugins_dir = bundles_dir.join('plugins')
 
-      type = specified_type || looked_up_type
-      if type.to_s == 'java'
-        group_id = specified_group_id || looked_up_group_id || KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_GROUP_ID
-        packaging = specified_packaging || looked_up_packaging || KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_PACKAGING
-        classifier = specified_classifier || looked_up_classifier || KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_CLASSIFIER
-        version = specified_version || looked_up_version || LATEST_VERSION
-        destination = plugins_dir.join('java').join(artifact_id).join(version)
-      else
-        group_id = specified_group_id || looked_up_group_id || KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_GROUP_ID
-        packaging = specified_packaging || looked_up_packaging || KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_PACKAGING
-        classifier = specified_classifier || looked_up_classifier || KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_CLASSIFIER
-        version = specified_version || looked_up_version || LATEST_VERSION
-        destination = plugins_dir.join('ruby')
-      end
+      destination = type == 'java' ? plugins_dir.join('java').join(artifact_id).join(version) : plugins_dir.join('ruby')
+
       sha1_file = "#{bundles_dir}/#{SHA1_FILENAME}"
 
       @logger.debug("Installing plugin: group_id=#{group_id} artifact_id=#{artifact_id} packaging=#{packaging} classifier=#{classifier} version=#{version} destination=#{destination}")
@@ -101,13 +130,15 @@ module KPM
                                                        verify_sha1,
                                                        @nexus_config,
                                                        @nexus_ssl_verify)
-
       mark_as_active(plugins_dir, artifact_info, artifact_id)
+      update_plugin_identifier(plugins_dir, plugin_key, artifact_info)
 
       artifact_info
     end
 
-    def install_plugin_from_fs(file_path, name, version, bundles_dir=nil, type='java')
+
+
+    def install_plugin_from_fs(plugin_key, file_path, name, version, bundles_dir=nil, type='java')
       bundles_dir = Pathname.new(bundles_dir || DEFAULT_BUNDLES_DIR).expand_path
       plugins_dir = bundles_dir.join('plugins')
 
@@ -121,6 +152,7 @@ module KPM
       artifact_info[:version] ||= version
 
       mark_as_active(plugins_dir, artifact_info)
+      update_plugin_identifier(plugins_dir, plugin_key, artifact_info)
 
       artifact_info
     end
@@ -167,6 +199,20 @@ module KPM
 
     private
 
+    def update_plugin_identifier(plugins_dir, plugin_key, artifact_info)
+      # In case the artifact on disk already existed and the installation is skipped, we don't try to update the pluginKey mapping
+      # (of course if the install is retried with a different pluginKey that may be confusing for the user)
+      if artifact_info[:bundle_dir].nil?
+        @logger.info("Skipping updating plugin identifier for already installed plugin")
+        return
+      end
+
+      # The plugin_name needs to be computed after the fact (after the installation) because some plugin archive embed their directory structure
+      plugin_name = Pathname.new(artifact_info[:bundle_dir]).parent.split[1].to_s
+      plugins_manager = PluginsManager.new(plugins_dir, @logger)
+      plugins_manager.update_plugin_identifier(plugin_key, plugin_name)
+    end
+
     def mark_as_active(plugins_dir, artifact_info, artifact_id=nil)
       # Mark this bundle as active
       plugins_manager = PluginsManager.new(plugins_dir, @logger)

data/lib/kpm/installer.rb

@@ -79,7 +79,7 @@ module KPM
 
       infos = []
       @config['plugins']['java'].each do |plugin|
-        infos << install_plugin(plugin['group_id'], plugin['artifact_id'] || plugin['name'], plugin['packaging'], plugin['classifier'], plugin['version'], @config['plugins_dir'], 'java', force_download, verify_sha1)
+        infos << install_plugin(plugin['name'], plugin['group_id'], plugin['artifact_id'], plugin['packaging'], plugin['classifier'], plugin['version'], @config['plugins_dir'], 'java', force_download, verify_sha1)
       end
 
       infos
@@ -90,7 +90,7 @@ module KPM
 
       infos = []
       @config['plugins']['ruby'].each do |plugin|
-        infos << install_plugin(plugin['group_id'], plugin['artifact_id'] || plugin['name'], plugin['packaging'], plugin['classifier'], plugin['version'], @config['plugins_dir'], 'ruby', force_download, verify_sha1)
+        infos << install_plugin(plugin['name'], plugin['group_id'], plugin['artifact_id'], plugin['packaging'], plugin['classifier'], plugin['version'], @config['plugins_dir'], 'ruby', force_download, verify_sha1)
       end
 
       infos

data/lib/kpm/plugins_directory.rb

@@ -15,16 +15,16 @@ module KPM
     end
 
     # Note: this API is used in Docker images (see kpm_generator.rb, careful when changing it!)
-    def self.lookup(raw_plugin_name, latest=false, raw_kb_version=nil)
-      plugin_name = raw_plugin_name.to_s.downcase
-      plugin = all(latest)[plugin_name.to_sym]
+    def self.lookup(raw_plugin_key, latest=false, raw_kb_version=nil)
+      plugin_key = raw_plugin_key.to_s.downcase
+      plugin = all(latest)[plugin_key.to_sym]
       return nil if plugin.nil?
 
       type = plugin[:type]
       is_ruby = type == :ruby
 
       group_id    = plugin[:group_id] || (is_ruby ? KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_GROUP_ID : KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_GROUP_ID)
-      artifact_id = plugin[:artifact_id] || "#{plugin_name}-plugin"
+      artifact_id = plugin[:artifact_id] || "#{plugin_key}-plugin"
       packaging   = plugin[:packaging] || (is_ruby ? KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_PACKAGING : KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_PACKAGING)
       classifier  = plugin[:classifier] || (is_ruby ? KPM::BaseArtifact::KILLBILL_RUBY_PLUGIN_CLASSIFIER : KPM::BaseArtifact::KILLBILL_JAVA_PLUGIN_CLASSIFIER)
 

data/lib/kpm/plugins_directory.yml

@@ -89,7 +89,7 @@
 :kpm:
   :type: :ruby
   :versions:
-    :0.15: 0.0.1
+    :0.15: 0.0.2
 :litle:
   :type: :ruby
   :versions:

data/lib/kpm/plugins_manager.rb

@@ -1,4 +1,5 @@
 require 'pathname'
+require 'json'
 
 module KPM
   class PluginsManager
@@ -53,6 +54,32 @@ module KPM
       end
     end
 
+    def update_plugin_identifier(plugin_key, plugin_name)
+      path = Pathname.new(@plugins_dir).join('plugin_identifiers.json')
+      backup_path = Pathname.new(path.to_s + ".back")
+
+      identifiers = {}
+      begin
+        identifiers = File.open(path, 'r') do |f|
+          JSON.parse(f.read)
+        end
+        # Move file in case something happens until we complete the operation
+        FileUtils.mv(path, backup_path)
+      rescue Errno::ENOENT
+      end
+
+      identifiers[plugin_key] = plugin_name
+      File.open(path, 'w') do |f|
+        f.write(identifiers.to_json)
+      end
+
+      # Cleanup backup entry
+      FileUtils.rm(backup_path, :force => true)
+
+      identifiers
+    end
+
+
     def guess_plugin_name(artifact_id)
       return nil if artifact_id.nil?
       captures = artifact_id.scan(/(.*)-plugin/)

data/lib/kpm/version.rb

@@ -1,3 +1,3 @@
 module KPM
-  VERSION = '0.1.0'
+  VERSION = '0.1.1'
 end

data/spec/kpm/remote/installer_spec.rb

@@ -1,4 +1,5 @@
 require 'spec_helper'
+require 'json'
 
 describe KPM::Installer do
 
@@ -16,21 +17,34 @@ describe KPM::Installer do
                                                 'killbill' => {
                                                     'webapp_path' => kb_webapp_path,
                                                     'plugins_dir' => plugins_dir,
-                                                    'plugins'     => {
+                                                    'plugins' => {
                                                         'java' => [{
-                                                                       'name'    => 'analytics-plugin',
+                                                                       'name' => 'analytics',
+                                                                       'version' => '0.7.1'
+                                                                   },
+                                                                   # Re-add a second time the same plugin to validate the idempotency of installation
+                                                                   {
+                                                                       'name' => 'analytics',
                                                                        'version' => '0.7.1'
                                                                    }],
                                                         'ruby' => [{
-                                                                       'name'    => 'payment-test-plugin',
+                                                                       'name' => 'killbill:payment-test-plugin',
+                                                                       'artifact_id' => 'payment-test-plugin',
                                                                        'version' => '1.8.7'
                                                                    },
                                                                    {
-                                                                       'name'    => 'stripe'
-                                                                   }]
+                                                                       'name' => 'stripe'
+                                                                   },
+                                                                   # Re-add a second time the same plugin to validate the idempotency of installation
+                                                                   {
+                                                                       'name' => 'killbill:payment-test-plugin',
+                                                                       'artifact_id' => 'payment-test-plugin',
+                                                                       'version' => '1.8.7'
+                                                                   }
+                                                        ]
                                                     },
                                                 },
-                                                'kaui'     => {
+                                                'kaui' => {
                                                     'webapp_path' => kaui_webapp_path
                                                 }
 
@@ -49,6 +63,7 @@ describe KPM::Installer do
   private
 
   def check_installation(plugins_dir, kb_webapp_path, kaui_webapp_path)
+
     [
         plugins_dir,
         plugins_dir + '/platform',
@@ -68,10 +83,21 @@ describe KPM::Installer do
         kb_webapp_path,
         kaui_webapp_path,
         plugins_dir + '/platform/jruby.jar',
+        plugins_dir + '/plugins/plugin_identifiers.json',
         plugins_dir + '/plugins/java/analytics-plugin/0.7.1/analytics-plugin-0.7.1.jar',
         plugins_dir + '/plugins/ruby/killbill-payment-test/1.8.7/killbill.properties'
     ].each do |file|
       File.file?(file).should be_true
     end
+
+    plugin_identifiers = File.open(plugins_dir + '/plugins/plugin_identifiers.json', 'r') do |f|
+      JSON.parse(f.read)
+    end
+
+    plugin_identifiers.size.should == 3
+    plugin_identifiers['killbill:payment-test-plugin'].should == 'killbill-payment-test'
+    plugin_identifiers['stripe'].should == 'killbill-stripe'
+    plugin_identifiers['analytics'].should == 'analytics-plugin'
+
   end
 end

data/spec/kpm/unit/plugins_manager_spec.rb

@@ -21,6 +21,29 @@ describe KPM::PluginsManager do
     FileUtils.remove_entry_secure @plugins_dir
   end
 
+
+  it 'it creates a plugin identifiers file and add entries' do
+    # Verifies file gets created if does not exist
+    identifiers = @manager.update_plugin_identifier('foo', 'foo_name')
+    identifiers.size.should == 1
+    identifiers['foo'].should == 'foo_name'
+
+    # Verify file was created from previous entry (prev value was read)
+    identifiers = @manager.update_plugin_identifier('bar', 'bar_name')
+    identifiers.size.should == 2
+    identifiers['foo'].should == 'foo_name'
+    identifiers['bar'].should == 'bar_name'
+
+
+    # Verify file was created from previous entry (prev value was read)
+    identifiers = @manager.update_plugin_identifier('zoe', 'zoe_name')
+    identifiers.size.should == 3
+    identifiers['bar'].should == 'bar_name'
+    identifiers['foo'].should == 'foo_name'
+    identifiers['zoe'].should == 'zoe_name'
+  end
+
+
   it 'sets a path as active' do
     @manager.set_active(@plugin_dir.join('1.0.0'))
     File.exists?(@plugin_dir.join('ACTIVE')).should be_true

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: kpm
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - Kill Bill core team
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2015-12-07 00:00:00.000000000 Z
+date: 2015-12-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: highline
@@ -145,8 +145,20 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project:
-rubygems_version: 2.1.9
+rubygems_version: 2.4.6
 signing_key:
 specification_version: 4
 summary: Kill Bill package manager.
-test_files: []
+test_files:
+- spec/kpm/remote/base_artifact_spec.rb
+- spec/kpm/remote/installer_spec.rb
+- spec/kpm/remote/kaui_artifact_spec.rb
+- spec/kpm/remote/killbill_plugin_artifact_spec.rb
+- spec/kpm/remote/killbill_server_artifact_spec.rb
+- spec/kpm/remote/tomcat_manager_spec.rb
+- spec/kpm/unit/base_artifact_spec.rb
+- spec/kpm/unit/plugins_directory_spec.rb
+- spec/kpm/unit/plugins_manager_spec.rb
+- spec/kpm/unit/sha1_checker_spec.rb
+- spec/kpm/unit/sha1_test.yml
+- spec/spec_helper.rb