smart_monkey 0.1.2 → 0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 0f738b9ed13db8b304d0294e755569da6085a99f
-  data.tar.gz: 43029b8fefa454d818e2ad4b3e9a52cc0e8923e9
+  metadata.gz: 654c78960662ad050551e259c70eb156943f1a5b
+  data.tar.gz: e9d4cd8bf1d0352594231fd885cb869877ad534d
 SHA512:
-  metadata.gz: dadfa51c02979055e92c8ae6620e1dd47b3ce008d547b86ef50b4429339c27558dad499a782b0d351a46dc11039b48e6239a78b0577497c0a99120665074759a
-  data.tar.gz: 9c94e9632a44b457c851b8760b5e53af82af01188deb863969ffe11fcebfe7af7a3c39abdd230ee548f843828c4700bf7a815df8aa613453197800d1070a746d
+  metadata.gz: a1cf9150d136402f4381625ad51370f46dc0b8d958b5a2f9bfbf025f370e22805a9a53eb0a7de9d61b3490470e30e6df01a6be37870ac2f52d087e014a7e6362
+  data.tar.gz: 566dc5f0252c36409c7dbcc88278f59b3501e74232c75daafae59e6d571902303309ffcd75a28ed9c53519971f75afd8a9ab959ccbc44478ea9e398d71680152

data/README.md

@@ -1,18 +1,15 @@
 # CrashMonkey4IOS
 iOS Monkey Test Tool.
 
-###改进点:
-1. 原先的CrashMonkey只支持iPhone Simulator测试，修改后**支持真机测试**(目前模拟器的兼容没有做)
-2. 支持真机收集**系统日志(Systemlog)**，利用[deviceconsole][dc]实现
-3. 支持真机收集**崩溃日志(Crashlog)**，利用[libimobiledevice][lidvc]的**idevicecrashreport**实现
-4. 解决在iPhone5及以上分辨率的设备上测试报告截图手势坐标绘制错误的问题
-5. 解决判定crash出现失败的情况。
-6. 测试报告中添加设备信息及应用信息(Summary下)
-7. 使用最新版的[UIAutoMonkey][uiatmonkey]，加入`UI Holes`与`Application Not Repsonding ("ANR")`的处理，添加[custom.js][custom]作为入口脚本.
-8. 加入[tuneup][tp]依赖
-9. 修改**UIAutoMonkey.js**中截图策略，为每个Event Action进行截图
-10. 修改**CrashMonkey**中测试报告显示截图策略，增加至最近的50张
-11. 解决大家一直所诟病的iOS App执行Monkey测试过程当中跳出程序后，导致脚本block，App无法自动返回的问题，解决方案：利用一个线程去监听instruments执行过程当中的日志，每隔20s进行前后比对，如果日志没有更新则说明App hanged，利用**idevicedebug**恢复App至前台。
+###简要说明:
+1. 支持**真机测试、模拟器测试**
+2. 支持收集**系统日志(Systemlog)**、**崩溃日志(Crashlog)**、***instrument行为日志***
+3. 支持测试报告截图，绘制行为轨迹
+4. 支持测试设备信息收集
+5. 使用最新版的[UIAutoMonkey][uiatmonkey]，加入`UI Holes`与`Application Not Repsonding ("ANR")`的处理，添加[custom.js][custom]作为入口脚本.
+6. 加入[tuneup][tp]依赖
+7. 修改**UIAutoMonkey.js**中截图策略，为每个Event Action进行截图
+8. 支持测试执行过程中App进入后台，自动恢复(测试不会block)
   [lidvc]:https://github.com/libimobiledevice/libimobiledevice
   [dc]:https://github.com/rpetrich/deviceconsole
   [uiatmonkey]: https://github.com/jonathanpenn/ui-auto-monkey/blob/master/UIAutoMonkey.js
@@ -20,24 +17,32 @@ iOS Monkey Test Tool.
   [tp]:https://github.com/vigossjjj/CrashMonkey4IOS/tree/master/lib/ui-auto-monkey/tuneup
   [troubleshooting]:https://github.com/vigossjjj/CrashMonkey4IOS/tree/master/Troubleshooting.md
 
-###依赖及安装:
+###环境及依赖安装:
+###### 系统要求
 1. 安装Ruby运行环境，建议不要使用OS X自带版本，可自行使用RVM安装最新版的Ruby。建议使用淘宝镜像安装，速度比较快，`$ sed -i -e 's/ftp\.ruby-lang\.org\/pub\/ruby/ruby\.taobao\.org\/mirrors\/ruby/g' ~/.rvm/config/db`
 2. 确保gem可用，也建议使用淘宝镜像 `gem sources --remove https://rubygems.org/;gem sources -a http://ruby.taobao.org/;gem sources -l`
 3. 安装**Homebrew** `ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"`
+4. 建议Xcode 6.x +
+
+###### 依赖安装
+1. `brew install -HEAD ideviceinstaller`
+2. `brew install libimobiledevice`
+3. `brew install imagemagick`
+4. `gem install smart_monkey`
 
 ###使用说明:
-1. 运行`sh reset.sh`安装相关依赖
-2. 执行前需要先配置[custom.js][custom]相关参数
-3. 执行命令`/CrashMonkey4IOS/bin/smart_monkey -a ${App_BunnelID} -w ${iPhone_UDID} -n 1`
+1. 执行前需要先配置[custom.js][custom]相关参数
+2. 执行命令`smart_monkey -a ${App_BunnelID} -w ${iPhone_UDID}`
 
 ###参数说明:
 ```
+$ smart_monkey -h
 Usage: smart_monkey [options]
-    -a app_name                      Target Application, abs path to simulator-compiled .app file or the bundle_id of the desired target on device(Required)
+    -a app_name                      Bundle ID of the desired target on device(Required)
     -w device                        Target Device UDID(Required)
     -n run_count                     How many times monkeys run(default: 1)
     -d result_dir                    Where to output result(default: ./smart_monkey_result)
-    -t time_limit_sec                Time limit of running(default: 100 sec)
+    -t time_limit_sec                Time limit of running
     -s dsym_file                     Use .dSYM file to symbolicating crash logs
     -c custom_path                   Configuration custom.js Path
     -e extend_javascript_path        Extend Uiautomation Javascript for such Login scripts
@@ -50,12 +55,11 @@ Usage: smart_monkey [options]
         --list-app                   Show List of Installed Apps in iPhone/iPhone Simulator
         --list-devices               Show List of Devices
         --reset-iPhone-Simulator     Reset iPhone Simulator
-        --version                    print crash monkey version
+        --version                    print smart monkey version
 ```
 
 ###TODO List:
-1. 适配iPhone Simulator
-2. 修改配置脚本及文档。
+2. 配置文件参数优化
 
 ###Troubleshooting:
 安装和执行测试遇到的问题解决方案请参看:[Troubleshooting.md][troubleshooting]

data/VERSION

@@ -1 +1 @@
-0.1.2
+0.2

data/bin/smart_monkey

@@ -11,11 +11,11 @@ require 'smart_monkey'
 
 opts = {}
 ARGV.options do |o|
-  o.on('-a app_name', 'Target Application, abs path to simulator-compiled .app file or the bundle_id of the desired target on device(Required)') {|b| opts[:app_path] = b}
+  o.on('-a app_name', 'Bundle ID of the desired target on device(Required)') {|b| opts[:app_path] = b}
   o.on('-w device', 'Target Device UDID(Required)') {|b| opts[:device] = b}
   o.on('-n run_count', 'How many times monkeys run(default: 1)') {|b| opts[:run_count] = b.to_i}
   o.on('-d result_dir', 'Where to output result(default: ./smart_monkey_result)') {|b| opts[:result_base_dir] = File.expand_path(b)}
-  o.on('-t time_limit_sec', 'Time limit of running(default: 100 sec)') {|b| opts[:time_limit_sec] = b.to_i}
+  o.on('-t time_limit_sec', 'Time limit of running') {|b| opts[:time_limit_sec] = b.to_i}
   o.on('-s dsym_file', 'Use .dSYM file to symbolicating crash logs') {|b| opts[:dsym_file_path] = File.expand_path(b)}
   o.on('-c custom_path', 'Configuration custom.js Path') {|b| opts[:custom_path] = File.expand_path(b)}
   o.on('-e extend_javascript_path', 'Extend Uiautomation Javascript for such Login scripts') {|b| opts[:extend_javascript_path] = File.expand_path(b)}
@@ -26,7 +26,7 @@ ARGV.options do |o|
   o.on('--list-app', 'Show List of Installed Apps in iPhone/iPhone Simulator') {|_| opts[:list_app] = true}
   o.on('--list-devices', 'Show List of Devices') {|_| opts[:list_devices] = true}
   o.on('--reset-iPhone-Simulator', 'Reset iPhone Simulator'){|_| opts[:reset_iphone_simulator] = true}
-  o.on('--version', 'print crash monkey version'){|_| opts[:version] = true}
+  o.on('--version', 'print smart monkey version'){|_| opts[:version] = true}
   o.parse!
 end
 

data/lib/smart_monkey/monkey_runner.rb

@@ -17,11 +17,6 @@ module UIAutoMonkey
 
     def run(opts)
       @options = opts
-      res_dir = @options[:result_base_dir] || RESULT_BASE_PATH
-
-      puts "INSTRUMENTS_TRACE_PATH : #{INSTRUMENTS_TRACE_PATH}"
-      puts "RESULT_BASE_PATH : #{res_dir}"
-
       if @options[:show_config]
         show_config
         return true
@@ -35,6 +30,11 @@ module UIAutoMonkey
         reset_iphone_simulator
         return true
       end
+
+      res_dir = @options[:result_base_dir] || RESULT_BASE_PATH
+      puts "INSTRUMENTS_TRACE_PATH : #{INSTRUMENTS_TRACE_PATH}"
+      puts "RESULT_BASE_PATH : #{res_dir}"
+
       ###########
       log @options.inspect
       FileUtils.remove_dir(result_base_dir, true)
@@ -76,7 +76,7 @@ module UIAutoMonkey
     def run_a_case
       log "=================================== Start Test (#{@times+1}/#{total_test_count}) ======================================="
       FileUtils.makedirs(crash_save_dir(@times+1)) unless File.exists?(crash_save_dir(@times+1))
-      pull_crash_from_iphone(@times+1)
+      pull_crash_files(@times+1)
       cr_list = crash_report_list(@times+1)
       start_time = Time.now
       watch_syslog do
@@ -91,7 +91,7 @@ module UIAutoMonkey
         end
       end
 
-      pull_crash_from_iphone(@times+1)
+      pull_crash_files(@times+1)
       new_cr_list = crash_report_list(@times+1)
       # increase crash report?
       diff_cr_list = new_cr_list - cr_list
@@ -212,6 +212,19 @@ module UIAutoMonkey
       `"instruments" -s devices`.strip.split(/\n/).drop(2)
     end
 
+    def instruments_deviceinfo(device)
+      `"instruments" -s devices | grep #{device}`.strip
+    end
+
+    def is_simulator
+      deviceinfo = instruments_deviceinfo(device)
+      if deviceinfo.include? "Simulator"
+        true
+      else
+        false
+      end
+    end
+
     def product_type(device)
       product_hash={
           "iPhone7,2"=>"iPhone 6",
@@ -229,16 +242,25 @@ module UIAutoMonkey
           "iPhone1,2"=>"iPhone 3G",
           "iPhone1,1"=>"iPhone",
         }
-      type=`ideviceinfo -u #{device} -k ProductType`.strip
-      product_hash[type]
+
+      if is_simulator
+        instruments_deviceinfo(device).split("[")[0]
+      else
+        type = `ideviceinfo -u #{device} -k ProductType`.strip
+        product_hash[type]
+      end
     end
 
     def product_version(device)
-      `ideviceinfo -u #{device} -k ProductVersion`.strip
+      if !is_simulator
+        `ideviceinfo -u #{device} -k ProductVersion`.strip
+      end
     end
 
     def device_name(device)
-      `ideviceinfo -u #{device} -k DeviceName`.strip
+      if !is_simulator
+        `ideviceinfo -u #{device} -k DeviceName`.strip
+      end
     end
 
     def compress_image(path)
@@ -345,12 +367,16 @@ module UIAutoMonkey
       "#{result_base_dir}/result_#{sprintf('%03d', times)}"
     end
 
-    def crash_report_dir
+    def sim_crash_report_dir
       "#{ENV['HOME']}/Library/Logs/DiagnosticReports"
     end
 
-    def pull_crash_from_iphone(times)
-      `idevicecrashreport -u #{device} -e -k #{crash_save_dir(times)}`
+    def pull_crash_files(times)
+      if !is_simulator
+        `idevicecrashreport -u #{device} -e -k #{crash_save_dir(times)}`
+      else
+        `cp #{sim_crash_report_dir}/* #{crash_save_dir(times)}`
+      end
     end
 
     def crash_report_list(times)
@@ -363,12 +389,14 @@ module UIAutoMonkey
       `rm -rf #{traces}`
     end
 
-    def grep_syslog
-      'tail -n 0 -f /var/log/system.log'
-    end
-
     def grep_ios_syslog
-      "#{deviceconsole_original_path} -u #{device}"
+      if is_simulator
+        puts "Attempting iOS Simulator system log capture via tail system.log."
+        "tail -n 0 -f ~/Library/Logs/CoreSimulator/#{device}*/system.log"
+      else
+        puts "Attempting iOS device system log capture via deviceconsole."
+        "#{deviceconsole_original_path} -u #{device}"
+      end
     end
 
     def console_log_path
@@ -460,7 +488,6 @@ module UIAutoMonkey
 
     def watch_syslog
       STDOUT.sync = true
-      puts "Attempting iOS device system log capture via deviceconsole."
       stdin, stdout, stderr = Open3.popen3(grep_ios_syslog)
       log_filename = "#{result_base_dir}/console.txt"
       thread = Thread.new do

data/lib/smart_monkey/templates/index.html.erb

@@ -12,7 +12,7 @@
       <h1>Summary</h1>
       <div>
         <ul>
-          <li>Hardware: <%= ProductType %>, <%= ProductVersion %>, <%= UniqueDeviceID %></li>
+          <li>Hardware: <%= ProductType %> <%= ProductVersion %> <%= UniqueDeviceID %></li>
           <li>Application: <%= Application %> </li>
           <li>Span: <%= start_time %> ~ <%= end_time %></li>
           <li>Results: <%= test_count %> Test, <%= ok_count %> Ok, <%= cr_count %> Crash, <%= nr_count %> No Run</li>

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: smart_monkey
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: '0.2'
 platform: ruby
 authors:
 - vigossjjj