fastlane-plugin-mango 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 75221491bc8b2b1f16e769b0208203efbd55ac56a5577a632e522b1caa5348ba
-  data.tar.gz: facf07278586a34970216bf5bef7984bdd0e2cebb9854ae2f75e84d62693f310
+  metadata.gz: 968c3231a114d8f5b371c05d887cc9e9dcb908f7baa8729370e9ba97dfab4c2c
+  data.tar.gz: c0a3a9c50e99ca15806fbb253e10349f6cd263677cf6e45dad03078e04ebc584
 SHA512:
-  metadata.gz: be523d5eaaf2475812c9aac88e1a82a8135b06649228d0c6684d3201064d4fceeaf28551dfcef541a5b820c8bb27d11bee76bfdb99e78a05600240585cf2c818
-  data.tar.gz: f095125f165f15695d06fb6d11589c57a61be876fb54bf1f9201919289ac93532c1d1f0eafd74af29577f062ab6a210539ec9a2b60fd3b2e22662a57faa1a9b4
+  metadata.gz: db22d989c74376ec53583f7acb8254b4b01cabaff64a8742fa068b137bd023c1bc74dd80b3b0d8ebdde6f60e87b1e17324acf4968433239e1d3c5ac1023d4547
+  data.tar.gz: 7d87f2df5052ade2b4aa40eb3999e28b2fe6f30c9eaf0974681048042a0ac53a2270ccaab4579db8178e82ef6797ebe9f54a37efe3be30d02d5799c5c4c33729

data/README.md

@@ -1,13 +1,14 @@
 
 # Mango - Fastlane plugin
 
-[![fastlane Plugin Badge](https://rawcdn.githack.com/fastlane/fastlane/master/fastlane/assets/plugin-badge.svg)](https://rubygems.org/gems/fastlane-plugin-mango)
+[![fastlane Plugin Badge](https://rawcdn.githack.com/fastlane/fastlane/master/fastlane/assets/plugin-badge.svg)](https://rubygems.org/gems/fastlane-plugin-mango) [![License](https://img.shields.io/badge/license-MIT-green.svg?style=flat)](https://github.com/xing/mango/blob/master/LICENSE)
+[![Gem](https://img.shields.io/gem/v/fastlane-plugin-mango.svg?style=flat)](http://rubygems.org/gems/fastlane-plugin-mango)
 
 A fastlane plugin that runs Android tasks on a specified [Docker](https://www.docker.com/) image
 
 <img src="assets/mango_logo.png" alt="Mango Logo" width="256px" height="256px"/>
 
-Running Android tests, especially [Espresso](https://developer.android.com/training/testing/espresso/) on a continuous integration environment like [Jenkins](https://jenkins.io/) can be a hassle. You need to boot, manage and destroy an [Android Virtual Device (AVD)](https://developer.android.com/studio/run/managing-avds) during the test run. This is why we, the mobile releases team at [XING](https://www.xing/com), built this plugin. It spins up a specified [Docker](https://www.docker.com/) image and runs a given task on it.
+Running Android tests, especially [Espresso](https://developer.android.com/training/testing/espresso/) on a continuous integration environment like [Jenkins](https://jenkins.io/) can be a hassle. You need to boot, manage and destroy an [Android Virtual Device (AVD)](https://developer.android.com/studio/run/managing-avds) during the test run. This is why we, the mobile releases team at [XING](https://www.xing.com), built this plugin. It spins up a specified [Docker](https://www.docker.com/) image and runs a given task on it.
 
 Another requirement we had was to run on a clean environment, which is why Mango also helps us to run our unit test and more. For an example check out the [example `Fastfile`](sample-android/fastlane/Fastfile)
 
@@ -16,7 +17,7 @@ Another requirement we had was to run on a clean environment, which is why Mango
 In order to use this plugin you will need to to have [Docker](https://www.docker.com/) installed on the machine you are using.
 Documentation on how to set it up properly on a Linux (ubuntu) machine can be found [here](docs/docker-linux.md).
 
-If you need an Android Virtual Device (AVD) to run your tests (for example Espresso, Calabash or Appium), it's necessary to check that your CPU supports kvm virtualisation. We already experienced, that it doesn't fully work on macOS and are using Linux for that.
+If you need an Android Virtual Device (AVD) to run your tests (for example Espresso, Calabash or Appium), it's necessary to check that your CPU supports kvm virtualisation. We already experienced that it doesn't fully work on macOS and are using Linux for that.
 
 ## Getting Started
 
@@ -30,16 +31,33 @@ fastlane add_plugin mango
 
 After installing this plugin you have access to one additional action (`mango`) in your `Fastfile`.
 
-So a lane in your `Fastfile` could look similar to this:
+So a lane in your `Fastfile` could look similar to this for Espresso tests:
 ```ruby
 desc "Run espresso tests on docker images"
-  lane :Espresso_Tests do |options|
-     mango(
+  lane :Espresso_Tests do
+     run_dockerized_task(
        container_name: "espresso_container",
-       docker_image: "thyrlian/android-sdk:latest",
+       port_factor: options[:port_factor],
+       docker_image: "joesss/mango-docker:latest",
        container_timeout: 120,
        android_task: "./gradlew connectedAndroidTest",
-       post_actions: "adb logcat -d > logcat.txt"          
+       post_actions: "adb logcat -d > logcat.txt",
+       pull_latest_image: true          
+     )
+   end
+```
+
+or to this for unit tests or other gradle tasks:
+```ruby
+desc "Run unit tests on docker images"
+  lane :Unit_Tests do
+     run_dockerized_task(
+       container_name: "unit_tests_container",
+       port_factor: options[:port_factor],
+       docker_image: "joesss/mango-base:latest",
+       is_running_on_emulator: false,
+       android_task: "./gradlew testDebug",
+       pull_latest_image: true         
      )
    end
 ```
@@ -48,8 +66,6 @@ Now you can call this new lane by calling `bundle exec fastlane Espresso_Tests`.
 
 The Plugin will start up the given `docker_image`, execute the given `android_task` and afterwards execute the `post_actions`.
 
-Of
-
 ## Configuration options
 The `mango` action has plenty of options to configure it.
 

data/lib/fastlane/plugin/mango/actions/run_dockerized_task_action.rb

@@ -1,11 +1,12 @@
+require_relative '../helper/docker_commander'
+
 module Fastlane
   module Actions
     class RunDockerizedTaskAction < Action
       def self.run(params)
         UI.important("The mango plugin is working!")
-        emulator_name = params[:emulator_name]
-        docker_emulator = Fastlane::Helper::MangoHelper.new(params)
-        docker_emulator.setup_container
+        mango_helper = Fastlane::Helper::MangoHelper.new(params)
+        mango_helper.setup_container
 
         failure_buffer_timeout = 5
         timeout_command = "timeout #{params[:maximal_run_time] - failure_buffer_timeout}m"
@@ -16,17 +17,17 @@ module Fastlane
           UI.success("Starting Android Task.")
           bundle_install = params[:bundle_install] ? '&& bundle install ' : ''
 
-          docker_emulator.docker_exec("cd #{workspace_dir} #{bundle_install}&& #{timeout_command} #{android_task} || exit 1")
+          Helper::DockerCommander.docker_exec(command: "cd #{workspace_dir} #{bundle_install}&& #{timeout_command} #{android_task} || exit 1", container_name: mango_helper.container_name)
         end
 
       ensure
         post_actions = params[:post_actions]
-        if post_actions
-          docker_emulator&.docker_exec("cd #{workspace_dir} && #{post_actions}")
+        if post_actions && !mango_helper.kvm_disabled?
+          Helper::DockerCommander.docker_exec(command: "cd #{workspace_dir} && #{post_actions}", container_name: mango_helper.container_name)
         end
 
-        UI.important("Cleaning up #{emulator_name} container")
-        docker_emulator.clean_container if docker_emulator.instance_variable_get('@container')
+        UI.important("Cleaning up #{params[:emulator_name]} container")
+        mango_helper.clean_container if mango_helper.instance_variable_get('@container')
       end
 
       def self.description
@@ -115,7 +116,7 @@ module Fastlane
           FastlaneCore::ConfigItem.new(key: :workspace_dir,
                                        env_name: "WORKSPACE_DIR",
                                        default_value: '/root/tests/',
-                                       description: "Path to the workspace to execute commands",
+                                       description: "Path to the workspace to execute commands. If you want to execute your `android_task` from a different directory you have to specify `workspace_dir`",
                                        optional: true,
                                        type: String),
 

data/lib/fastlane/plugin/mango/helper/docker_commander.rb

@@ -0,0 +1,66 @@
+require 'docker'
+require 'os'
+
+module Fastlane
+  module Helper
+    module DockerCommander
+
+      def self.pull_image(docker_image_name:)
+        handle_thin_pool_exception do
+          Actions.sh("docker pull #{docker_image_name}")
+        end
+      end
+
+      def self.start_container(emulator_args:, docker_name:, docker_image:)
+        docker_name = if docker_name
+                        "--name #{docker_name}"
+                      else
+                        ''
+                      end
+
+        # Action.sh returns all output that the command produced but we are only
+        # interested in the last line, since it contains the id of the created container.
+        UI.important("Attaching #{ENV['PWD']} to the docker container")
+        handle_thin_pool_exception do
+          Actions.sh("docker run -v $PWD:/root/tests --privileged -t -d #{emulator_args} #{docker_name} #{docker_image}").chomp
+        end
+      end
+
+      def self.stop_container(container_name:)
+        Actions.sh("docker stop #{container_name}") if container_name
+      end
+
+      def self.delete_container(container_name:)
+        Actions.sh("docker rm #{container_name}") if container_name
+      end
+
+      def self.disconnect_network_bridge(container_name:)
+        Actions.sh("docker network disconnect -f bridge #{container_name}") if container_name
+      rescue StandardError
+        # Do nothing if the network bridge is already gone
+      end
+
+      def self.prune
+        Action.sh('docker system prune -f')
+      end
+
+      def self.handle_thin_pool_exception(&block)
+        begin
+          block.call
+        rescue FastlaneCore::Interface::FastlaneShellError => exception
+          retry_counter = retry_counter.to_i + 1
+          if exception.message =~ /Create more free space in thin pool/ && retry_counter < 2
+            prune
+            retry
+          else
+            raise exception
+          end
+        end
+      end
+
+      def self.docker_exec(command:, container_name:)
+        Actions.sh("docker exec -i #{container_name} bash -l -c \"#{command}\"") if container_name
+      end
+    end
+  end
+end

data/lib/fastlane/plugin/mango/helper/emulator_commander.rb

@@ -0,0 +1,36 @@
+require_relative 'docker_commander'
+
+module Fastlane
+  module Helper
+    module EmulatorCommander
+
+      def self.disable_animations(container_name:)
+        DockerCommander.docker_exec(command: 'adb shell settings put global window_animation_scale 0.0', container_name: container_name)
+        DockerCommander.docker_exec(command: 'adb shell settings put global transition_animation_scale 0.0', container_name: container_name)
+        DockerCommander.docker_exec(command: 'adb shell settings put global animator_duration_scale 0.0', container_name: container_name)
+      end
+
+      def self.increase_logcat_storage(container_name:)
+        DockerCommander.docker_exec(command: 'adb logcat -G 16m', container_name: container_name)
+      end
+
+      # Checks if created emulator is connected
+      def self.check_connection(container_name:)
+        UI.success('Checking if emulator is connected to ADB.')
+
+        if emulator_is_healthy?(container_name: container_name)
+          UI.success('Emulator connected successfully')
+          true
+        else
+          UI.important("Something went wrong. Newly created device couldn't connect to the adb")
+          false
+        end
+      end
+
+      def self.emulator_is_healthy?(container_name: container_name)
+        list_devices = DockerCommander.docker_exec(command: 'adb devices', container_name: container_name)
+        list_devices.include? "\tdevice"
+      end
+    end
+  end
+end

data/lib/fastlane/plugin/mango/helper/mango_helper.rb

@@ -1,7 +1,9 @@
 require 'docker'
-require 'socket'
 require 'timeout'
 require 'os'
+require 'net/http'
+require_relative 'docker_commander'
+require_relative 'emulator_commander'
 
 module Fastlane
   module Helper
@@ -46,22 +48,53 @@ module Fastlane
         pull_from_registry if @pull_latest_image
 
         # Make sure that network bridge for the current container is not already used
-        disconnect_network_bridge if container_name
+        DockerCommander.disconnect_network_bridge(container_name: container_name)
 
         create_container
 
-        begin
-          wait_for_healthy_container false
-          check_emulator_connection if is_running_on_emulator
-        rescue StandardError
+        if is_running_on_emulator && kvm_disabled?
+          raise 'Linux requires GPU acceleration for running emulators, but KVM virtualization is not supported by your CPU. Exiting..'
+        end
+
+        container_state = wait_for_healthy_container
+
+        if is_running_on_emulator
+          connection_state = EmulatorCommander.check_connection(container_name: container_name)
+          container_state = connection_state && connection_state
+        end
+
+        unless container_state
           UI.important("Will retry checking for a healthy docker container after #{sleep_interval} seconds")
           @container.stop
           @container.delete(force: true)
           sleep @sleep_interval
           create_container
-          wait_for_healthy_container
-          check_emulator_connection if is_running_on_emulator
+
+          unless wait_for_healthy_container
+            UI.important('Container is unhealthy. Exiting..')
+            # We use code "2" as we need something than just standard error code 1, so we can differentiate the next step in CI
+            exit 2
+          end
+
+          if is_running_on_emulator && !EmulatorCommander.check_connection(container_name: container_name)
+            UI.important('Cannot connect to emulator. Exiting..')
+            exit 2
+          end
+        end
+
+        if is_running_on_emulator
+          EmulatorCommander.disable_animations(container_name: container_name)
+          EmulatorCommander.increase_logcat_storage(container_name: container_name)
+        end
+      end
+
+      def kvm_disabled?
+        begin
+          DockerCommander.docker_exec(command: 'kvm-ok > kvm-ok.txt', container_name: container_name)
+        rescue StandardError
+          # kvm-ok will always throw regardless of the result. therefore we save the output in the file and ignore the error
         end
+        DockerCommander.docker_exec(command: 'cat kvm-ok.txt', container_name: container_name).include?('KVM acceleration can NOT be used')
       end
 
       # Stops and remove container
@@ -70,11 +103,6 @@ module Fastlane
         @container.delete(force: true)
       end
 
-      # Executes commands inside docker container
-      def docker_exec(command)
-        Actions.sh("docker exec -i #{container_name} bash -l -c \"#{command}\"")
-      end
-
       private
 
       # Sets path to adb
@@ -90,25 +118,6 @@ module Fastlane
         UI.success("Link to VNC: http://#{@host_ip_address}:#{@no_vnc_port}")
       end
 
-      # Restarts adb on the separate port and checks if created emulator is connected
-      def check_emulator_connection
-        UI.success('Checking if emulator is connected to ADB.')
-
-        if emulator_is_healthy?
-          UI.success('Emulator connected successfully')
-        else
-          raise "Something went wrong. Newly created device couldn't connect to the adb"
-        end
-
-        disable_animations
-        increase_logcat_storage
-      end
-
-      def emulator_is_healthy?
-        list_devices = docker_exec('adb devices')
-        list_devices.include? "\tdevice"
-      end
-
       # Creates new container using params
       def create_container
         UI.important("Creating container: #{container_name}")
@@ -119,8 +128,8 @@ module Fastlane
         rescue StandardError
           UI.important("Something went wrong while creating: #{container_name}, will retry in #{@sleep_interval} seconds")
           print_cpu_load
-          `docker stop #{container_name}` if container_name
-          `docker rm #{container_name}` if container_name
+          DockerCommander.stop_container(container_name: container_name)
+          DockerCommander.delete_container(container_name: container_name)
           sleep @sleep_interval
           container = create_container_call
           @container_name = container unless container_name
@@ -143,19 +152,9 @@ module Fastlane
         # When CPU is under load we cannot create a healthy container
         wait_cpu_to_idle
 
-        docker_name = if container_name
-                        "--name #{container_name}"
-                      else
-                        ''
-                      end
-
         emulator_args = is_running_on_emulator ? "-p #{no_vnc_port}:6080 -e DEVICE='#{device_name}'" : ''
 
-        # Action.sh returns all output that the command produced but we are only
-        # interested in the last line, since it contains the id of the created container.
-        UI.important("Attaching #{ENV['PWD']} to the docker container")
-        output = Actions.sh("docker run -v $PWD:/root/tests --privileged -t -d #{emulator_args} #{docker_name} #{docker_image}").chomp
-        output.split("\n").last
+        DockerCommander.start_container(emulator_args: emulator_args, docker_name: container_name, docker_image: docker_image)
       end
 
       def execute_pre_action
@@ -166,7 +165,7 @@ module Fastlane
       def pull_from_registry
         docker_image_name = docker_image.gsub(':latest', '')
         Actions.sh(@docker_registry_login) if @docker_registry_login
-        Actions.sh("docker pull #{docker_image_name}")
+        DockerCommander.pull_image(docker_image_name: docker_image_name)
       end
 
       # Checks that chosen ports are not already allocated. If they are, it will stop the allocated container
@@ -177,11 +176,11 @@ module Fastlane
           vnc_allocated_container.stop
         end
 
-        if ports_open?('0.0.0.0', [@no_vnc_port])
-          UI.important('Something went wrong. One of the required ports is still busy')
+        if port_open?('0.0.0.0', @no_vnc_port)
+          UI.important('Something went wrong. VNC port is still busy')
           sleep @sleep_interval
-          `docker stop #{container_name}` if container_name
-          `docker rm #{container_name}` if container_name
+          DockerCommander.stop_container(container_name: container_name)
+          DockerCommander.delete_container(container_name: container_name)
         end
       end
 
@@ -222,7 +221,7 @@ module Fastlane
       end
 
       # Waits until container is healthy using specified timeout
-      def wait_for_healthy_container(will_exit = true)
+      def wait_for_healthy_container
         UI.important('Waiting for Container to be in the Healthy state.')
 
         number_of_tries = timeout / sleep_interval
@@ -232,31 +231,25 @@ module Fastlane
             UI.success('Your container is ready to work')
             return true
           end
-          UI.important("Container status: #{@container.json['State']['Status']}")
+
+          if @container.json['State']['Health']
+            UI.important("Container status: #{@container.json['State']['Health']['Status']}")
+          else
+            UI.important("Container status: #{@container.json['State']['Status']}")
+          end
+
           sleep sleep_interval
         end
         UI.important("The Container failed to load after '#{timeout}' seconds timeout. Reason: '#{@container.json['State']['Status']}'")
-        # We use code "2" as we need something than just standard error code 1, so we can differentiate the next step in CI
-        exit 2 if will_exit
-        raise 'Fail'
+        false
       end
 
-      # Checks if port is already open
-      def ports_open?(ip, ports)
-        raise "'ports' should be an array" unless ports.is_a? Array
-        ports.each do |port|
-          begin
-            Timeout.timeout(1) do
-              begin
-                s = TCPSocket.new(ip, port)
-                s.close
-                return true
-              rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH
-              end
-            end
-          rescue Timeout::Error
-          end
-        end
+      # Checks if port is already openZ
+      def port_open?(server, port)
+        http = Net::HTTP.start(server, port, open_timeout: 5, read_timeout: 5)
+        response = http.head('/')
+        response.code == '200'
+      rescue Timeout::Error, SocketError, Errno::ECONNREFUSED
         false
       end
 
@@ -288,23 +281,6 @@ module Fastlane
         raise "CPU was overloaded. Couldn't start emulator"
       end
 
-      # Disables animation for faster and stable testing
-      def disable_animations
-        docker_exec('adb shell settings put global window_animation_scale 0.0')
-        docker_exec('adb shell settings put global transition_animation_scale 0.0')
-        docker_exec('adb shell settings put global animator_duration_scale 0.0')
-      end
-
-      # Increases logcat storage
-      def increase_logcat_storage
-        docker_exec('adb logcat -G 16m')
-      end
-
-      def disconnect_network_bridge
-        `docker network disconnect -f bridge #{container_name}`
-      rescue StandardError
-        # Do nothing if the network bridge is already gone
-      end
     end
   end
 end

data/lib/fastlane/plugin/mango/version.rb

@@ -1,5 +1,5 @@
 module Fastlane
   module Mango
-    VERSION = '1.0.0'.freeze
+    VERSION = '1.1.0'.freeze
   end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: fastlane-plugin-mango
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.1.0
 platform: ruby
 authors:
 - Serghei Moret, Daniel Hartwich
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2018-08-15 00:00:00.000000000 Z
+date: 2018-08-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: docker-api
@@ -146,6 +146,8 @@ files:
 - README.md
 - lib/fastlane/plugin/mango.rb
 - lib/fastlane/plugin/mango/actions/run_dockerized_task_action.rb
+- lib/fastlane/plugin/mango/helper/docker_commander.rb
+- lib/fastlane/plugin/mango/helper/emulator_commander.rb
 - lib/fastlane/plugin/mango/helper/mango_helper.rb
 - lib/fastlane/plugin/mango/version.rb
 homepage: https://github.com/xing/mango
@@ -168,7 +170,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.7.7
+rubygems_version: 2.7.4
 signing_key: 
 specification_version: 4
 summary: This plugin Android tasks on docker images