watir-schema 0.0.2

Sign up to get free protection for your applications and to get access to all the features.
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: 5f035e29b54d53c796de2183ab54ee2396838c0a
4
+ data.tar.gz: 8ff51c485ac615d7312cd40f56e2279445fc0d70
5
+ SHA512:
6
+ metadata.gz: a214469c05a4720e328ae3bfd9273c9768dc14dbe0ccba276a89d55a632bccdc59248320a2b3cfbb4f990b3aa564b9ec9e83771950bb89f3fedc19dd93dbbd48
7
+ data.tar.gz: 34a050e10ae61a462eb7612059d6ba2781246ddc882f092d1e5320ae2b2e155c6bd029f95619a0d2f18bef3272462881c88f87c54514004070040be0247d2132
@@ -0,0 +1,21 @@
1
+ *.gem
2
+ *.rbc
3
+ .bundle
4
+ .config
5
+ .yardoc
6
+ Gemfile.lock
7
+ InstalledFiles
8
+ _yardoc
9
+ coverage
10
+ doc/
11
+ lib/bundler/man
12
+ pkg
13
+ rdoc
14
+ spec/reports
15
+ test/tmp
16
+ test/version_tmp
17
+ tmp
18
+ *.html
19
+ *.swp
20
+ *~
21
+
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in watir-schema.gemspec
4
+ gemspec
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2013 Malo Skrylevo
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
@@ -0,0 +1,107 @@
1
+ # Watir::Schema
2
+
3
+ [![Gem Version](https://badge.fury.io/rb/watir-schema.png)](http://rubygems.org/gems/watir-schema)
4
+ [![Build Status](https://travis-ci.org/3aHyga/watir-schema.png?branch=master)](https://travis-ci.org/3aHyga/watir-schema)
5
+ [![Coverage Status](https://coveralls.io/repos/3aHyga/watir-schema/badge.png)](https://coveralls.io/r/3aHyga/watir-schema)
6
+ [![Code Climate](https://codeclimate.com/github/3aHyga/watir-schema.png)](https://codeclimate.com/github/3aHyga/watir-schema)
7
+ [![Endorse Count](http://api.coderwall.com/3aHyga/endorsecount.png)](http://coderwall.com/3aHyga)
8
+
9
+
10
+ Support multiple platforms (Windows, OS X, etc.) and browsers (Chrome, Firefox, Safari, IE, etc.):
11
+ - Chrome
12
+ - Firefox
13
+ - IE*
14
+ - Safari*
15
+ * - no proxy support
16
+
17
+ The tool is able to run around the clock with report at least every hour. Run of the script can be done with any scheduler like cron.
18
+
19
+ Functions:
20
+ - Launch a web surfer that will:
21
+ - Allow to use a proxy on 8080 port;
22
+ - Do a deep surf using the specified schema, including, for example, authenticated browsing on numerous websites, interacting with web forms, and random exploration of webpages;
23
+ - Return the browser's work folder;
24
+ - Store all of the surf data in a programmatically searchable log file immediately upon completion of the automated web surfing;
25
+ - Store the log results to the body of an email and send the results to the specified address at the end of each surf;
26
+
27
+ ## Requirements
28
+
29
+ ###Linux/MacOSX:
30
+ - chromedriver: http://code.google.com/p/selenium/wiki/ChromeDriver
31
+
32
+ ###Windows:
33
+ - autohotkey.exe: http://www.autohotkey.com/
34
+ - IEDriverServer.exe: http://code.google.com/p/selenium/wiki/InternetExplorerDriver
35
+ - chromedriver.exe: http://code.google.com/p/selenium/wiki/ChromeDriver
36
+ All Programs must be included in PATH variable.
37
+
38
+ ## Installation
39
+
40
+ Add this line to your application's Gemfile:
41
+
42
+ gem 'watir-schema'
43
+
44
+ And then execute:
45
+
46
+ $ bundle
47
+
48
+ Or install it yourself as:
49
+
50
+ $ gem install watir-schema
51
+
52
+ ## Usage
53
+
54
+ Sample run:
55
+ $ watir-schema -v 5 -s schema.yaml -d ~/ -e sample@gmail.com
56
+
57
+ This will set log level 5 (extremely excessive), surf the web using schema.yaml config, and store browser data in the '~/' folder, and then send report to collected log specified sample@gmail.com.
58
+
59
+ Help info on the tool:
60
+
61
+ $ watir-schema -h
62
+ This is a watir-schema script, how to use it see below
63
+ -v, --verbose 1 enable verbose output, values: 0 to 5
64
+ -s, --schema set schema YAML-file to proceed
65
+ -b, --browser firefox use browser to surf, values: msie|firefox|chrome|safari
66
+ -p, --[no-]proxy use proxy
67
+ -d, --dir set base dir to store results
68
+ -t, --store store web-browser state into a folder
69
+ -e, --email set an email to report test results
70
+ -l, --log set output flow to a file
71
+ -h, --help Show this message
72
+ -V, --version Print version
73
+
74
+ ## File schema.yaml
75
+
76
+ Sample schema.yaml is shewn below:
77
+
78
+ ---
79
+ report:
80
+ server: smtp.gmail.com
81
+ port: 587
82
+ domain: networkadvertising.org
83
+ login: monitoring@networkadvertising.org
84
+ tls: true
85
+ password: '#rrrjjhdr@'
86
+ auth_type: :plain #or :plain or :login or :cram_md5
87
+ from: 'monitoring@networkadvertising.org'
88
+ from_name: 'NAI Monitoring'
89
+ body: ! "From: %from_name <%from>\n
90
+ Subject: Watir web-surfing report"
91
+ sites:
92
+ google.com:
93
+ schema:
94
+ - 'a:id:gb_70:%C'
95
+ - 't:id:Email:%S=monitoring@networkadvertising.org='
96
+ - 't:id:Passwd:%S=#rrrjjhdr@='
97
+ - 'i:id:signIn:C'
98
+ - 't:name:q:%S=2012 Audi A6='
99
+ - 'b:name:btnG:C'
100
+
101
+ ## Contributing
102
+
103
+ 1. Fork it
104
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
105
+ 3. Commit your changes (`git commit -am 'Add some feature'`)
106
+ 4. Push to the branch (`git push origin my-new-feature`)
107
+ 5. Create new Pull Request
@@ -0,0 +1,88 @@
1
+ #!/usr/bin/env rake
2
+
3
+ desc "Prepare bundler"
4
+ task :bundleup do
5
+ sh 'gem install bundler --version "~> 1.3.1" --no-ri --no-rdoc'
6
+ end
7
+
8
+ desc "Requires"
9
+ task :req do
10
+ $: << File.expand_path( '../lib', __FILE__ )
11
+ require 'bundler/gem_helper'
12
+ require 'watir'
13
+
14
+ Bundler::GemHelper.install_tasks
15
+ end
16
+
17
+ desc "Prepare bundle environment"
18
+ task :up do
19
+ sh 'bundle install'
20
+ end
21
+
22
+ desc "Test with cucumber"
23
+ task :test do
24
+ sh 'if [ -d features ]; then tests=$(ls features/*.feature) ; cucumber $tests; fi'
25
+ end
26
+
27
+ desc "Distilled clean"
28
+ task :distclean do
29
+ sh 'git clean -fd'
30
+ sh 'cat .gitignore | while read mask; do rm -rf $(find -iname "$mask"); done'
31
+ end
32
+
33
+ desc "Prepare environment"
34
+ task :env => [ :req ] do
35
+ files = [ 'http://selenium.googlecode.com/files/selenium-server-2.31.0.zip' ]
36
+ drivers = case RUBY_PLATFORM
37
+ when /linux/
38
+ sh "#{Watir::Schema.root}/share/setup_linux #{Watir::Schema.root}"
39
+ [ RUBY_PLATFORM =~ /64/ && 'chromedriver_linux64_26.0.1383.0.zip' ||
40
+ 'chromedriver_linux32_26.0.1383.0.zip' ]
41
+ when /windows/
42
+ [ RUBY_PLATFORM =~ /64/ && 'IEDriverServer_x64_2.31.0.zip' ||
43
+ 'IEDriverServer_Win32_2.31.0.zip',
44
+ 'chromedriver_win_26.0.1383.0.zip' ]
45
+ when /macosx/
46
+ [ 'chromedriver_mac_26.0.1383.0.zip' ]
47
+ else [] ; end
48
+ drivers.each do |driver|
49
+ files << "http://chromedriver.googlecode.com/files/#{driver}" ; end
50
+
51
+ files.each do |file|
52
+ filename = File.basename file
53
+ target = File.join Watir::Schema.root, filename
54
+ sh "wget #{file} -O #{target}"
55
+ sh "7z x #{target}"
56
+ end
57
+
58
+ puts "-" * 30 + "\nPlease, reload your environment"
59
+ end
60
+
61
+ desc "Generate gem"
62
+ namespace :gem do
63
+ task :build => [ :req ] do
64
+ sh 'gem build watir-schema.gemspec'
65
+ end
66
+
67
+ task :install do
68
+ require File.expand_path( '../lib/watir/schema/version', __FILE__ )
69
+ sh "gem install watir-schema-#{Watir::Schema::VERSION}.gem"
70
+ end
71
+
72
+ task :publish => [ :req ] do
73
+ require File.expand_path( '../lib/watir/schema/version', __FILE__ )
74
+ sh "git tag v#{Watir::Schema::VERSION}"
75
+ sh "gem push watir-schema-#{Watir::Schema::VERSION}.gem"
76
+ sh "git push"
77
+ sh "git push --tag"
78
+ end
79
+
80
+ task :make => [ :build, :install, :publish ]
81
+ task :default => :make
82
+ end
83
+
84
+ task(:default).clear
85
+ task :default => :test
86
+ task :all => [ :bundleup, :up, :test, :'gem:make', :distclean ]
87
+ task :build => [ :bundleup, :up, :test, :'gem:build', :'gem:install', :distclean ]
88
+
@@ -0,0 +1,64 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'micro-optparse'
4
+ require 'yaml'
5
+ require 'watir/schema'
6
+
7
+ options = Parser.new do |p|
8
+ p.banner = "This is a watir-schema script, how to use it see below"
9
+ p.version = "watir-schema #{Watir::Schema::VERSION}"
10
+ p.option :verbose, "enable verbose output, values: 0 to 5",
11
+ :short => 'v', :default => 1, :value_in_set => ( 0..5 ).to_a
12
+ p.option :schema, "set schema YAML-file to proceed",
13
+ :short => 's', :default => '', :value_matches => /^.+$/
14
+ p.option :browser, "use browser to surf, values: " +
15
+ "msie|firefox|chrome|safari",
16
+ :short => 'b', :default => 'firefox',
17
+ :value_matches => /^(msie|firefox|chrome|safari)$/
18
+ p.option :proxy, "use proxy", :short => 'p'
19
+ p.option :dir, "set base dir to store results",
20
+ :short => 'd', :default => ''
21
+ p.option :store, "store web-browser state into a folder",
22
+ :short => 't', :default => ''
23
+ p.option :email, "set an email to report test results",
24
+ :short => 'e', :default => '',
25
+ :value_satisfies => lambda {|x| x.empty? || x =~ /[^@]+@[^@]+/ }
26
+ p.option :log, "set output flow to a file", :short => 'l', :default => ''
27
+ end.process!
28
+ if !ARGV.empty?
29
+ $stderr.puts "Invalid option(s): #{ARGV.inspect}" ; exit ; end
30
+
31
+ f = case options[ :verbose ]
32
+ when 0
33
+ [ :warn ]
34
+ when 1
35
+ [ :info, :warn ]
36
+ when 2
37
+ [ :info, :warn, :basic ]
38
+ when 3
39
+ [ :info, :warn, :basic, :extended ]
40
+ when 4
41
+ [ :info, :warn, :basic, :enter, :leave ]
42
+ when 5
43
+ [ :info, :warn, :basic, :extended, :enter, :leave ] ; end
44
+
45
+ io = ( options[ :log ] == '-' || !options[ :email ].empty? ) &&
46
+ ( require 'tempfile' ; Tempfile.new( 'tmp.log' ) ) ||
47
+ options[ :log ].empty? && $stdout ||
48
+ File.open( options[ :log ], 'w+' )
49
+
50
+ require 'rdoba'
51
+ settings = YAML::load( open Watir::Schema.root( 'share/settings.yaml' ) )
52
+
53
+ $stdout.puts "The script is running..."
54
+
55
+ rdoba :log => { :in => Watir::Schema, :as => :log, :prefix => [ :timestamp ],
56
+ :io => io, :functions => f }
57
+
58
+ surfer = Watir::Schema.new settings, options[ :browser ],
59
+ options[ :dir ]
60
+
61
+ schema = YAML::load( open options[ :schema ] )
62
+ surfer.crawl schema, options[ :proxy ], options[ :store ]
63
+ surfer.report schema, options[ :email ], io, options[ :log ]
64
+
@@ -0,0 +1,26 @@
1
+ require 'rbconfig'
2
+ require 'watir/schema'
3
+ require 'watir/schema/version'
4
+
5
+ module Watir
6
+ def self.os
7
+ @@os ||= (
8
+ host_os = RbConfig::CONFIG['host_os']
9
+ case host_os
10
+ when /(mswin|msys|mingw|cygwin|bccwin|wince|emc)/
11
+ plat = $1 == 'mswin' && 'native' || $1
12
+ out = `ver`.encode( 'US-ASCII',
13
+ :invalid => :replace, :undef => :replace )
14
+ if out =~ /\[.* (\d+)\.([\d\.]+)\]/
15
+ "windows-#{plat}-#{$1 == '5' && 'xp' || 'vista'}-#{$1}.#{$2}"
16
+ else
17
+ "windows-#{plat}" ; end
18
+ when /darwin|mac os/
19
+ 'macosx'
20
+ when /linux/
21
+ 'linux'
22
+ when /(solaris|bsd)/
23
+ "unix-#{$1}"
24
+ else
25
+ raise Error::WebDriverError, "unknown os: #{host_os.inspect}"
26
+ end) ; end ; end
@@ -0,0 +1,298 @@
1
+ require 'watir'
2
+ require 'watir-webdriver'
3
+ require 'fileutils'
4
+ require 'open3'
5
+ require 'yaml'
6
+ require 'timeout'
7
+ require 'net/smtp'
8
+
9
+ module Watir
10
+ class Browser
11
+ module Firefox
12
+ def browser_run options
13
+ b = Watir::Browser.new :firefox, options
14
+ profile_dir = b.driver.send( :bridge ).
15
+ instance_variable_get( :@launcher ).
16
+ instance_variable_get( :@profile_dir )
17
+ [ b, [ profile_dir ].freeze ] ; end
18
+
19
+ def browser_init browser, proxy = false
20
+ profile = Selenium::WebDriver::Firefox::Profile.new
21
+ profile.native_events = true
22
+ @options = { :profile => profile }
23
+ if proxy
24
+ @options.merge :proxy => { :http => "http://localhost:8080" } ; end
25
+
26
+ @b, @profile_data = browser_run @options ; end ; end
27
+
28
+ module Chrome
29
+ def browser_init browser, proxy = false
30
+ @profile_data = [ File.join( @path, 'profile' ) ].freeze
31
+ switches = [ "--user-data-dir=#{@profile_data.last}" ]
32
+ if proxy
33
+ switches << "--proxy-server=localhost:8080" ; end
34
+ @options = { :switches => switches }
35
+ @b = Watir::Browser.new :chrome, @options ; end ; end
36
+
37
+ module Msie
38
+ def browser_init browser, proxy = false
39
+ clear_current_state
40
+ # MSIE has no plain proxy settings
41
+ si, so, se, @wthr = Open3.popen3( "AutoHotkey " +
42
+ "\"#{fix_path Watir::Schema.root( "share/msie_err_control.ahk" )}\"" )
43
+ @b = Watir::Browser.new :ie ; end ; end
44
+
45
+ module Safari
46
+ def browser_init browser, proxy = false
47
+ clear_current_state
48
+ # Safari has no plain proxy settings
49
+ @b = Watir::Browser.new :safari
50
+ end ; end
51
+
52
+ module Opera
53
+ def browser_init browser, proxy = false
54
+ # http://code.google.com/p/selenium/wiki/OperaDriver
55
+ # Opera driver have no proxy caps
56
+ @profile_data = [ File.join( @path, 'profile' ) ].freeze
57
+ @b = Watir::Browser.new :opera, { :profile => @profile_data.last }
58
+ end ; end ; end
59
+
60
+ class Schema
61
+ Actions = {
62
+ 's' => '.select_list',
63
+ 'o' => '.option',
64
+ 't' => '.text_field',
65
+ 'a' => '.a',
66
+ 'i' => '.input',
67
+ 'b' => '.button',
68
+ 'c' => '.checkbox',
69
+ 'f' => '.form',
70
+ 'd' => '.div',
71
+ 'p' => '.span',
72
+ 'e' => '.element',
73
+ '%' => '.when_present',
74
+ 'L' => '.select',
75
+ 'S' => '.set',
76
+ 'C' => '.click',
77
+ 'T' => '.text',
78
+ }
79
+ AKeys = Actions.keys.join().gsub(/[?]/) do |e| "\\#{e}" ; end
80
+
81
+ def self.root path = ''
82
+ root = File.expand_path '../../..', __FILE__
83
+ File.join root, path
84
+ end
85
+
86
+ def initialize settings, browser = 'firefox', basedir = nil
87
+ # Initialize browser
88
+ @browser = browser.to_sym
89
+ if ![ :chrome, :firefox, :msie, :opera, :safari ].include? @browser
90
+ raise "#{@browser.inspect} isn't supported" ; end
91
+ @browser_settings = settings[ 'browsers' ][ @browser.to_s ]
92
+ eval "extend Browser::#{@browser.capitalize}"
93
+ log * { Browser: browser }
94
+
95
+ # Initialize paths for data logs
96
+ basedir = !basedir.empty? && basedir || ENV[ 'TMP' ]
97
+ @path = File.join( basedir, 'watir-scheme', ::Time.now.strftime('%m-%d-%Y'),
98
+ "#{browser.capitalize}-#{::Time.now.strftime("%H_%M_%S")}" )
99
+ log * { Path: @path }
100
+ FileUtils.mkdir_p @path
101
+
102
+ # Initialize timeouts
103
+ @load_timeout = @browser_settings[ 'timeouts' ] &&
104
+ @browser_settings[ 'timeouts' ][ 'load' ] || settings[ :timeouts ] &&
105
+ settings[ :timeouts ][ :load ] || 60
106
+ @automate_timeout = @browser_settings[ 'timeouts' ] &&
107
+ @browser_settings[ 'timeouts' ][ 'automate' ] ||
108
+ settings[ :timeouts ] && settings[ :timeouts ][ :automate ] || 600
109
+
110
+ # Data folders to store
111
+ ( @browser_settings[ 'data' ] || {} ).each do |os, data|
112
+ if Watir.os =~ /#{os}/
113
+ @profile_data = data ; end ; end
114
+
115
+ # Store settings
116
+ log >> { Settings: settings }
117
+ @settings = settings ; end
118
+
119
+ def crawl schema, proxy, storedir
120
+ log + { schema: schema, proxy: proxy, storedir: storedir }
121
+
122
+ #Initialize testing variables
123
+ pid = nil
124
+
125
+ #Initialize browser
126
+ browser_init @browser,
127
+ proxy && ( schema[ :proxy ] || settings[ :proxy ] )
128
+
129
+ #Begin web crawl
130
+ surf schema[ 'sites' ] || []
131
+ if storedir && !storedir.empty?
132
+ store_state storedir ; end
133
+ ensure
134
+ if @b
135
+ # close browser
136
+ @b.close rescue nil ; end ; end
137
+
138
+ def report schema, email, io, l = nil
139
+ if io == $stdout || !email || email.empty?
140
+ return ; end
141
+
142
+ rep = schema[ 'report' ]
143
+ status = ''
144
+ smtp = Net::SMTP.new rep[ 'server' ], rep[ 'port' ]
145
+ if rep[ 'tls' ]
146
+ smtp.enable_starttls ; end
147
+ io.rewind
148
+ begin
149
+ smtp.start( rep[ 'domain' ],
150
+ rep[ 'login' ], rep[ 'password' ], rep[ 'auth_type' ] ) do |s|
151
+ msg = rep[ 'body' ].gsub( /%[_\w]+/ ) do |str|
152
+ rep[ str.sub( '%', '' ) ] ; end.split( "\n" ).map do |x|
153
+ x.strip end.join( "\n")
154
+ msg << "\nTo: #{email}\n\nThe log is below:\n\n"
155
+ msg << io.read
156
+ s.send_message msg, rep[ 'from' ], email
157
+ end
158
+ status = "Report has been sent"
159
+ rescue
160
+ log.e
161
+ status = "It is unable to send the log"
162
+ l = true ; end
163
+
164
+ if l && !l.empty?
165
+ io.rewind
166
+ $stdout.puts io.read ; end
167
+ $stdout.puts status ; end
168
+
169
+ private
170
+
171
+ def fix_path path
172
+ if Watir.os =~ /windows/
173
+ path.gsub!( /\//, "\\" ) ; end
174
+ path ; end
175
+
176
+ def is_fullpath? path
177
+ if Watir.os =~ /windows/
178
+ path[ 1 ] == ':' && ( path[ 2 ] == "\\" || path[ 2 ] == '/' )
179
+ else
180
+ path[ 0 ] == '/' ; end ; end
181
+
182
+ def store_state folder = nil
183
+ log + { folder: folder }
184
+ target_base = folder
185
+ target_dir = target_base
186
+ log > "Storing browser state to #{target_dir}"
187
+ home = Dir.home
188
+ init = if @profile_data.is_a? Hash
189
+ Proc.new do |dir|
190
+ source = dir[ 1 ]
191
+ Dir.chdir( is_fullpath?( source ) && source ||
192
+ File.join( home, source ) )
193
+ target_dir = File.join target_base, dir[ 0 ] ; end
194
+ else
195
+ Proc.new do |dir|
196
+ Dir.chdir( is_fullpath?( dir ) && dir ||
197
+ File.join( home, dir ) ) ; end ; end
198
+
199
+ @profile_data.each do |dir|
200
+ init.call dir
201
+ Dir.glob( File.join '**', '*' ).each do |file|
202
+ tfile = File.join( target_dir, file )
203
+ FileUtils.mkdir_p( File.dirname tfile )
204
+ FileUtils.cp_r( file, tfile ) rescue Exception
205
+ end ; end
206
+
207
+ clear_current_state ; end
208
+
209
+ def clear_current_state
210
+ if @profile_data.frozen?
211
+ return ; end
212
+
213
+ home = Dir.home
214
+ init = if @profile_data.is_a? Hash
215
+ Proc.new do |dir|
216
+ source = dir[ 1 ]
217
+ Dir.chdir( is_fullpath?( source ) && source ||
218
+ File.join( home, source ) ) ; end
219
+ else
220
+ Proc.new do |dir|
221
+ Dir.chdir( is_fullpath( dir ) && dir ||
222
+ File.join( home, dir ) ) ; end ; end
223
+
224
+ @profile_data.each do |dir|
225
+ init.call dir
226
+ list = Dir.glob '*'
227
+ list.delete 'index.dat'
228
+ FileUtils.rm_rf list rescue Exception ; end ; end
229
+
230
+ def surf sites
231
+ log + { :sites => sites }
232
+ b = @b
233
+ log >> { :b => b }
234
+ sites.each do |site, opts|
235
+ log > { Site: site }
236
+ log >> { Opts: opts }
237
+
238
+ uri = site + ( opts[ 'path' ] || '' )
239
+ log > { uri: uri }
240
+ begin
241
+ Timeout::timeout( @load_timeout ) do
242
+ b.goto uri ; end
243
+ rescue Timeout::Error
244
+ log % "Error: can't complete the site #{uri} surfing"
245
+ next ; end
246
+
247
+ schema = opts[ 'schema' ]
248
+ log > "#{schema.size} steps in the schema for the site have found"
249
+ schema.each do |act|
250
+ log >> { act: act }
251
+ e = 'b'
252
+ while act.size > 0
253
+ if act =~ /^(?:([@?#{AKeys}])|(:([\w\-]+):([^:]+):)|([\(=`~](.*[^\\\)=`~])[\)=`~]))/
254
+ log >> { Matches: [ $1, $2, $3, $4, $5, $6 ] }
255
+ if $1
256
+ case $1
257
+ when '@'
258
+ e = "Watir::Wait.until(@automate_timeout) {(#{e})}"
259
+ when '?'
260
+ e = "(#{e}) if b"
261
+ else
262
+ e << Actions[ $1 ] ; end
263
+ act = act[ 1..-1 ]
264
+ elsif $2
265
+ e << "(:'#{$3}'=>'#{$4}')"
266
+ act = act[ $2.size..-1 ]
267
+ elsif $5
268
+ match = $5
269
+ word = $6.gsub( /\\(.)/, '\1' )
270
+ case match[ 0 ]
271
+ when '~'
272
+ e << " =~ /#{word}/"
273
+ when '='
274
+ e << "('#{word}')"
275
+ when '('
276
+ e = "sleep (#{word}); #{e}"
277
+ else
278
+ e << " !~ /#{word}/" ; end
279
+ act = act[ match.size..-1 ] ; end
280
+ else
281
+ log.e "Unknown operation #{act[ 0 ]} in schema part: #{act}"
282
+ act = act[ 1..-1 ] ; end ; end
283
+ log > "Step: eval #{e}"
284
+ begin
285
+ Timeout::timeout( @automate_timeout ) do
286
+ eval e ; end
287
+ rescue Timeout::Error
288
+ log > "Error: We didn't wait out the Opt-out analyzing completed"
289
+ rescue Watir::Wait::TimeoutError;
290
+ log > "Error: Can't execute #{e}"
291
+ rescue Interrupt
292
+ return nil
293
+ rescue Exception
294
+ log.e
295
+ end ; end ; end
296
+
297
+ true ; end ; end ; end
298
+
@@ -0,0 +1,5 @@
1
+ module Watir
2
+ class Schema
3
+ VERSION = "0.0.2"
4
+ end
5
+ end
@@ -0,0 +1,22 @@
1
+ #ErrorStdOut
2
+ SetTitleMatchMode, RegEx
3
+ SetControlDelay, 1000
4
+
5
+ WinWait, ahk_class IEFrame
6
+ WinActivate
7
+ Loop,
8
+ {
9
+ WinWaitActive, ahk_class #32770,,5
10
+ if ! ErrorLevel
11
+ Loop,
12
+ {
13
+ ControlClick, Button1 ; OK
14
+ WinWaitClose, ahk_class #32770,,1
15
+ if ! ErrorLevel
16
+ Break
17
+ }
18
+
19
+ WinWaitClose, ahk_class IEFrame,,1
20
+ if ! ErrorLevel
21
+ Break
22
+ }
@@ -0,0 +1,46 @@
1
+ ---
2
+ default_browser: firefox
3
+ optouts: true
4
+ proxy:
5
+ run: false
6
+ port: 8080
7
+ timeouts:
8
+ load: 120
9
+ automate: 1200
10
+ browsers:
11
+ firefox:
12
+ timeouts:
13
+ load: 60
14
+ automate: 600
15
+ chrome:
16
+ timeouts:
17
+ load: 60
18
+ automate: 300
19
+ msie:
20
+ timeouts:
21
+ load: 60
22
+ automate: 600
23
+ data:
24
+ xp:
25
+ Cookies: Cookies
26
+ Cache: Local Settings/Temporary Internet Files/Content.IE5
27
+ History: Local Settings/History/History.IE5
28
+ vista:
29
+ Cookies: AppData/Roaming/Microsoft/Windows/Cookies
30
+ Content: AppData/Local/Microsoft/Windows/Temporary Internet Files/Content.IE5
31
+ History: AppData/Local/Microsoft/Windows/History
32
+ safari:
33
+ timeouts:
34
+ load: 30
35
+ automate: 300
36
+ data:
37
+ xp:
38
+ - Local Settings/Application Data/Apple Computer/Safari
39
+ - Application Data/Apple Computer/Safari
40
+ vista:
41
+ - AppData/Roaming/Apple Computer/Safari
42
+ - AppData/Local/Apple Computer/Safari
43
+ ios:
44
+ - /private/var/mobile/Library
45
+ macosx:
46
+ - Library
@@ -0,0 +1,14 @@
1
+ #!/bin/bash
2
+
3
+ root=$1
4
+
5
+ env=$(cat ~/.bashrc | grep "SELENIUM_SERVER_JAR.*selenium-server-2.31.0.jar")
6
+ if [ -z "$env" ]; then
7
+ echo "export SELENIUM_SERVER_JAR=$root/selenium-2.31.0/selenium-2.31.0/selenium-server-2.31.0.jar" >> ~/.bashrc
8
+ fi
9
+
10
+ env=$(cat ~/.bashrc | grep "chromedriver")
11
+ if [ -z "$env" ]; then
12
+ echo "export PATH=\$PATH:$root" >> ~/.bashrc
13
+ fi
14
+
@@ -0,0 +1,28 @@
1
+ # coding: utf-8
2
+ lib = File.expand_path('../lib', __FILE__)
3
+ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
4
+ require 'watir/schema/version'
5
+
6
+ Gem::Specification.new do |spec|
7
+ spec.name = "watir-schema"
8
+ spec.version = Watir::Schema::VERSION
9
+ spec.authors = ["Malo Skrylevo"]
10
+ spec.email = ["majioa@yandex.ru"]
11
+ spec.description = %q{The gem is able to run around the clock with report at least every hour. Run of the script can be done with any scheduler like cron}
12
+ spec.summary = %q{The gem allows automated surfing over web using watir-webdriver}
13
+ spec.homepage = "https://github.com/3aHyga/watir-schema"
14
+ spec.license = "MIT"
15
+
16
+ spec.files = `git ls-files`.split($/)
17
+ spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
18
+ spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
19
+ spec.require_paths = ["lib"]
20
+
21
+ spec.add_dependency 'bundler', '~> 1.3.1'
22
+ spec.add_dependency 'watir-webdriver', '~> 0.6.2'
23
+ spec.add_dependency 'micro-optparse'
24
+ spec.add_dependency 'rdoba'
25
+
26
+ spec.add_development_dependency "bundler", "~> 1.3"
27
+ spec.add_development_dependency "rake"
28
+ end
metadata ADDED
@@ -0,0 +1,118 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: watir-schema
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.2
5
+ platform: ruby
6
+ authors:
7
+ - Malo Skrylevo
8
+ autorequire:
9
+ bindir: bin
10
+ cert_chain: []
11
+
12
+ date: 2013-10-01 00:00:00 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: bundler
16
+ requirement: &id001 !ruby/object:Gem::Requirement
17
+ requirements:
18
+ - - ~>
19
+ - !ruby/object:Gem::Version
20
+ version: 1.3.1
21
+ type: :runtime
22
+ prerelease: false
23
+ version_requirements: *id001
24
+ - !ruby/object:Gem::Dependency
25
+ name: watir-webdriver
26
+ requirement: &id002 !ruby/object:Gem::Requirement
27
+ requirements:
28
+ - - ~>
29
+ - !ruby/object:Gem::Version
30
+ version: 0.6.2
31
+ type: :runtime
32
+ prerelease: false
33
+ version_requirements: *id002
34
+ - !ruby/object:Gem::Dependency
35
+ name: micro-optparse
36
+ requirement: &id003 !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - &id004
39
+ - ">="
40
+ - !ruby/object:Gem::Version
41
+ version: "0"
42
+ type: :runtime
43
+ prerelease: false
44
+ version_requirements: *id003
45
+ - !ruby/object:Gem::Dependency
46
+ name: rdoba
47
+ requirement: &id005 !ruby/object:Gem::Requirement
48
+ requirements:
49
+ - *id004
50
+ type: :runtime
51
+ prerelease: false
52
+ version_requirements: *id005
53
+ - !ruby/object:Gem::Dependency
54
+ name: bundler
55
+ requirement: &id006 !ruby/object:Gem::Requirement
56
+ requirements:
57
+ - - ~>
58
+ - !ruby/object:Gem::Version
59
+ version: "1.3"
60
+ type: :development
61
+ prerelease: false
62
+ version_requirements: *id006
63
+ - !ruby/object:Gem::Dependency
64
+ name: rake
65
+ requirement: &id007 !ruby/object:Gem::Requirement
66
+ requirements:
67
+ - *id004
68
+ type: :development
69
+ prerelease: false
70
+ version_requirements: *id007
71
+ description: The gem is able to run around the clock with report at least every hour. Run of the script can be done with any scheduler like cron
72
+ email:
73
+ - majioa@yandex.ru
74
+ executables:
75
+ - watir-schema
76
+ extensions: []
77
+
78
+ extra_rdoc_files: []
79
+
80
+ files:
81
+ - .gitignore
82
+ - Gemfile
83
+ - LICENSE.txt
84
+ - README.md
85
+ - Rakefile
86
+ - bin/watir-schema
87
+ - lib/watir.rb
88
+ - lib/watir/schema.rb
89
+ - lib/watir/schema/version.rb
90
+ - share/msie_err_control.ahk
91
+ - share/settings.yaml
92
+ - share/setup_linux
93
+ - watir-schema.gemspec
94
+ homepage: https://github.com/3aHyga/watir-schema
95
+ licenses:
96
+ - MIT
97
+ metadata: {}
98
+
99
+ post_install_message:
100
+ rdoc_options: []
101
+
102
+ require_paths:
103
+ - lib
104
+ required_ruby_version: !ruby/object:Gem::Requirement
105
+ requirements:
106
+ - *id004
107
+ required_rubygems_version: !ruby/object:Gem::Requirement
108
+ requirements:
109
+ - *id004
110
+ requirements: []
111
+
112
+ rubyforge_project:
113
+ rubygems_version: 2.1.4
114
+ signing_key:
115
+ specification_version: 4
116
+ summary: The gem allows automated surfing over web using watir-webdriver
117
+ test_files: []
118
+