mechanize-progressbar 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/CHANGELOG +11 -0
- data/Gemfile +5 -4
- data/LICENSE.txt +18 -340
- data/README.rdoc +28 -32
- data/README.rdoc.ja +155 -0
- data/Rakefile +8 -2
- data/VERSION +1 -1
- data/lib/mechanize/progressbar.rb +9 -1
- data/lib/mechanize/progressbar/api.rb +50 -0
- data/lib/mechanize/progressbar/mechanize_1_0.rb +39 -0
- data/lib/mechanize/progressbar/mechanize_2_0.rb +44 -0
- data/lib/mechanize/progressbar/mechanize_progressbar.rb +89 -0
- data/spec/mechanize/progressbar/mech_progressbar_spec.rb +128 -67
- metadata +94 -124
- data/.document +0 -5
- data/.rspec +0 -1
- data/lib/mechanize/progressbar/mech_progressbar.rb +0 -149
data/README.rdoc.ja
ADDED
@@ -0,0 +1,155 @@
|
|
1
|
+
= mechanize-progressbar
|
2
|
+
|
3
|
+
Mechanize-Progressbar は Mechanize で get や click したときにテキストの進捗バーを表示します(ProgressBar gem 使用)。
|
4
|
+
サーバからファイルを取得するときのみ有効です。GET や POST の送信進行状況を表示させることはできません。
|
5
|
+
|
6
|
+
== できること
|
7
|
+
script:
|
8
|
+
require 'mechanize'
|
9
|
+
require 'mechanize/progressbar'
|
10
|
+
agent = Mechanize.new
|
11
|
+
agent.progressbar{ agent.get(large_file) }
|
12
|
+
|
13
|
+
$stderr output:
|
14
|
+
http://host/large_file.zip
|
15
|
+
15% |ooooooo | 135.2KB 21.9KB/s ETA: 00:00:10
|
16
|
+
|
17
|
+
== 必要なもの
|
18
|
+
|
19
|
+
- Ruby 1.8.6 か 1.8.7、1.9.1、1.9.2。jruby では未テスト
|
20
|
+
- 最近の Rubygems
|
21
|
+
- gem ライブラリ Mechanize (1.0.0 と 2.0 に対応)
|
22
|
+
- gem ライブラリ ProgressBar
|
23
|
+
|
24
|
+
== USAGE
|
25
|
+
|
26
|
+
Mechanize#progressbar のブロック内でファイルを get してください。
|
27
|
+
|
28
|
+
require 'mechanize'
|
29
|
+
require 'mechanize/progressbar'
|
30
|
+
agent = Mechanize.new
|
31
|
+
agent.progressbar{ agent.get(large_file) }
|
32
|
+
|
33
|
+
リンクをクリックしても動作します。
|
34
|
+
|
35
|
+
agent = Mechanize.new
|
36
|
+
agent.get(some_page)
|
37
|
+
agent.progressbar do
|
38
|
+
agent.page.link_with(:text => 'download here').click
|
39
|
+
end
|
40
|
+
|
41
|
+
ProgressBar の設定を変更したい場合は、キーワード風引数で指定します。
|
42
|
+
ProgressBar で以下のように設定するはずだった場合は、
|
43
|
+
|
44
|
+
pbar = ProgressBar.new(@title, @total, @out)
|
45
|
+
pbar.format = @format
|
46
|
+
pbar.format_arguments = @format_arguments
|
47
|
+
|
48
|
+
次のように引数に書くことになります。
|
49
|
+
今のところ設定は保存されず、ブロックを作成するたびに引数を指定する必要があります。
|
50
|
+
|
51
|
+
agent = Mechanize.new
|
52
|
+
agent.progressbar(
|
53
|
+
:title => @title,
|
54
|
+
:total => @total,
|
55
|
+
:out => @out,
|
56
|
+
:format => @format,
|
57
|
+
:format_arguments => @format_arguments
|
58
|
+
){ agent.get(large_file) }
|
59
|
+
|
60
|
+
=== {:title => @title}
|
61
|
+
|
62
|
+
タイトルを指定します。デフォルトの 2行表示では1行目に、1行表示モード(:single)では左端に表示されます。
|
63
|
+
ProgressBar 側の制限で、1行表示モードでは半角15文字までしか表示されません。
|
64
|
+
1行モード時には ProgressBar#initialize の第 1引数に渡されます。ProgressBar のマニュアルを参照してください。
|
65
|
+
|
66
|
+
=== {:total => @total}
|
67
|
+
|
68
|
+
最終的なバイト数を指定します。
|
69
|
+
ProgressBar#initialize の第 2引数に渡されます。ProgressBar のマニュアルを参照してください。
|
70
|
+
デフォルトではレスポンスヘッダの Content-Length を利用しています。
|
71
|
+
|
72
|
+
=== {:out => @out}
|
73
|
+
|
74
|
+
出力先の IO オブジェクトを指定します。デフォルトでは $stderr です。
|
75
|
+
ProgressBar#initialize の第 3引数に渡されます。ProgressBar のマニュアルを参照してください。
|
76
|
+
|
77
|
+
=== {:format => @format}
|
78
|
+
|
79
|
+
ProgressBar のフォーマットを指定しますその1です。
|
80
|
+
ProgressBar#format= のマニュアルを参照してください。
|
81
|
+
|
82
|
+
=== {:format_arguments => @format_arguments}
|
83
|
+
|
84
|
+
ProgressBar のフォーマットを指定しますその2です。
|
85
|
+
ProgressBar#format_arguments= のマニュアルを参照してください。
|
86
|
+
|
87
|
+
=== {:reversed => true/false}
|
88
|
+
|
89
|
+
ProgressBar の代わりに ReversedProgressBar を使用します。
|
90
|
+
バーの増加表示方向が逆になります。
|
91
|
+
|
92
|
+
=== {:file_transfer_mode => true/false}
|
93
|
+
|
94
|
+
ProgressBar をファイル転送用の表示にします。デフォルトで true です。
|
95
|
+
ProgressBar#file_transfer_mode= のマニュアルを参照してください。
|
96
|
+
|
97
|
+
=== {:single => true/false}
|
98
|
+
|
99
|
+
1行モードの可否を設定します。デフォルトでは false で、2行表示になっています。
|
100
|
+
|
101
|
+
Mechanize-Progressbar は取得する URL を適当に表示してからプログレスバーを表示しますが、
|
102
|
+
|
103
|
+
script:
|
104
|
+
agent.progressbar{ agent.get(large_file) }
|
105
|
+
output:
|
106
|
+
http://uri.host/large_file.zip
|
107
|
+
15% |ooooo | 135.2KB 21.9KB/s ETA: 00:00:10
|
108
|
+
|
109
|
+
この部分が2行になってるのが嫌だという場合は、引数に (:single => true) を指定します。
|
110
|
+
1行モードだと、ProgressBar gem の本来の表示になります。
|
111
|
+
|
112
|
+
script:
|
113
|
+
agent.progressbar(:single => true){ agent.get(large_file) }
|
114
|
+
output:
|
115
|
+
uri.host: 15% |ooo | 135.2KB 21.9KB/s ETA: 00:00:10
|
116
|
+
|
117
|
+
=== {:suppress_logger => true/false}
|
118
|
+
|
119
|
+
Mechanize に Logger が設定されていて、その出力先が ProgressBar と同じだった場合(デフォルトでは$stderr)、
|
120
|
+
"Read 1234 bytes" というような逐次のデータ取得表示は自動で抑制されます。
|
121
|
+
|
122
|
+
script:
|
123
|
+
agent.log = Logger.new($stderr)
|
124
|
+
agent.progressbar{ agent.get(large_file) }
|
125
|
+
output:
|
126
|
+
http://uri.host/large_file.zip
|
127
|
+
100% |ooooooooooooooooooooooooooooooo| 1024.0KB 21.9KB/s Time: 00:00:20
|
128
|
+
D, [...] DEBUG -- : Read 102400 bytes
|
129
|
+
|
130
|
+
最終的なファイルサイズのみ表示されています。
|
131
|
+
これを出力先が異なっても行ないたいという場合は、(:suppress_logger => true) としてください。
|
132
|
+
Logger の出力に逐次表示が含まれなくなります(あまり使い出がないとは思うのですが)。
|
133
|
+
|
134
|
+
(:suppress_logger => false) とすると、出力先が被っていた場合はブログレスバーがぐちゃぐちゃになります。
|
135
|
+
おすすめできません。
|
136
|
+
|
137
|
+
== ファイル取得時のメモリ使用仕様
|
138
|
+
|
139
|
+
Mechanize は取得した全ファイルを履歴兼キャッシュとしてスクリプト終了まで保持し続けています(GCの対象にはなりません)。
|
140
|
+
100MBのファイルを5個取得したら、使用メモリは最低でも500MBになっています。
|
141
|
+
Mechanize の仕様なのでどうにもならないのですが、履歴数を最低の 1にすることで軽減することはできます。
|
142
|
+
|
143
|
+
agent.max_history = 1
|
144
|
+
|
145
|
+
== Licence
|
146
|
+
|
147
|
+
Mechanize がないと動作しないので、 Mechanize と同様に MIT ライセンスです。
|
148
|
+
|
149
|
+
== Author
|
150
|
+
|
151
|
+
kitamomonga
|
152
|
+
kitamomonga@gmail.com
|
153
|
+
|
154
|
+
きたももんががきたん。
|
155
|
+
http://d.hatena.ne.jp/kitamomonga/
|
data/Rakefile
CHANGED
@@ -13,7 +13,7 @@ require 'jeweler'
|
|
13
13
|
Jeweler::Tasks.new do |gem|
|
14
14
|
gem.name = "mechanize-progressbar"
|
15
15
|
gem.homepage = "http://github.com/kitamomonga/mechanize-progressbar"
|
16
|
-
gem.license = "
|
16
|
+
gem.license = "MIT"
|
17
17
|
gem.summary = "Mechanize-Progressbar provides ProgressBar for Mechanize#get and Link#click. requires 'progressbar' gem."
|
18
18
|
gem.description = "Mechanize-Progressbar shows ProgressBar when HTTP GET access of Mechanize(Mechanize#get, Page::Link#click, response of Form#submit). It requires 'progressbar' gem. HTTP POST is not supported."
|
19
19
|
gem.email = "kitamomonga@gmail.com"
|
@@ -39,7 +39,13 @@ end
|
|
39
39
|
|
40
40
|
task :default => :spec
|
41
41
|
|
42
|
-
|
42
|
+
begin
|
43
|
+
gem 'rdoc'
|
44
|
+
require 'rdoc'
|
45
|
+
require 'rdoc/task'
|
46
|
+
rescue LoadError, Gem::LoadError
|
47
|
+
require 'rake/rdoctask'
|
48
|
+
end
|
43
49
|
Rake::RDocTask.new do |rdoc|
|
44
50
|
version = File.exist?('VERSION') ? File.read('VERSION') : ""
|
45
51
|
|
data/VERSION
CHANGED
@@ -1 +1 @@
|
|
1
|
-
0.
|
1
|
+
0.2.0
|
@@ -1,2 +1,10 @@
|
|
1
1
|
require 'progressbar'
|
2
|
-
require
|
2
|
+
raise LoadError, "require mechanize before mechanize/progressbar" unless defined?(Mechanize)
|
3
|
+
|
4
|
+
require 'mechanize/progressbar/api'
|
5
|
+
require 'mechanize/progressbar/mechanize_progressbar'
|
6
|
+
if Mechanize::VERSION == '1.0.0'
|
7
|
+
require 'mechanize/progressbar/mechanize_1_0'
|
8
|
+
else
|
9
|
+
require 'mechanize/progressbar/mechanize_2_0'
|
10
|
+
end
|
@@ -0,0 +1,50 @@
|
|
1
|
+
module MechanizeProgressBarAPI
|
2
|
+
### Usage:
|
3
|
+
### require 'mechanize'
|
4
|
+
### require 'mechanize/progressbar'
|
5
|
+
### agent = Mechanize.new
|
6
|
+
### agent.progressbar{ agent.get(large_file) }
|
7
|
+
### agent.get(some_page) #=> shows nothing
|
8
|
+
### agent.progressbar{ agent.page.link_with(:text => 'download here').click }
|
9
|
+
###
|
10
|
+
### If you want to set parameters to ProgressBar object,
|
11
|
+
### agent.progressbar(:title => 'large_file', :format_arguments => @fmt) do
|
12
|
+
### agent.get(large_file)
|
13
|
+
### end
|
14
|
+
### ProgressBar is gem lib. Please install with 'gem install progressbar' or bundle.
|
15
|
+
###
|
16
|
+
### options:
|
17
|
+
### [:single] enables single line mode, original ProgressBar gem behavior. default is false
|
18
|
+
### [:title] ProgressBar.new(*title*, total, out), default is empty
|
19
|
+
### [:total] ProgressBar.new(title, *total*, out), default is 'Content-Length' response header
|
20
|
+
### [:out] ProgressBar.new(title, total, *out*), default is $stderr
|
21
|
+
### [:format] ProgressBar.new(*args).format=*format*, default is nil
|
22
|
+
### [:format_arguments] ProgressBar.new(*args).format_arguments=*format_arguments*, default is nil
|
23
|
+
### [:reversed] ReversedProgressBar.new(*args), default is false
|
24
|
+
### [:file_transfer_mode] enables ProgressBar#file_transfer_mode, default is true
|
25
|
+
### [:suppress_logger] stops Logger output in socket reading. default true when agent.log output and :out is same
|
26
|
+
def progressbar(mech_or_pbar_opts = self, pbar_opts = {})
|
27
|
+
if mech_or_pbar_opts.kind_of?(Hash) then
|
28
|
+
mech, pbar_opts = self, mech_or_pbar_opts
|
29
|
+
else
|
30
|
+
mech = mech_or_pbar_opts
|
31
|
+
end
|
32
|
+
MechanizeProgressBar.register(mech, pbar_opts)
|
33
|
+
if block_given?
|
34
|
+
begin
|
35
|
+
yield mech
|
36
|
+
rescue Exception, StandardError
|
37
|
+
MechanizeProgressBar.unregister(mech)
|
38
|
+
raise
|
39
|
+
end
|
40
|
+
MechanizeProgressBar.unregister(mech)
|
41
|
+
else
|
42
|
+
MechanizeProgressBar.unregister_next_init(mech)
|
43
|
+
end
|
44
|
+
mech
|
45
|
+
end
|
46
|
+
|
47
|
+
def progressbar_option # :nodoc:
|
48
|
+
@progressbar_option ||= {}
|
49
|
+
end
|
50
|
+
end
|
@@ -0,0 +1,39 @@
|
|
1
|
+
# is for Mechanize 1.0.0 only
|
2
|
+
class Mechanize # has 6 added/modified lines
|
3
|
+
include MechanizeProgressBarAPI # (1of6)a
|
4
|
+
class Chain
|
5
|
+
class ResponseReader
|
6
|
+
def handle(ctx, params)
|
7
|
+
params[:response] = @response
|
8
|
+
body = StringIO.new
|
9
|
+
total = 0
|
10
|
+
mpbar = MechanizeProgressBar.new(params[:agent], params[:request], params[:response]) # (2of6)a
|
11
|
+
@response.read_body { |part|
|
12
|
+
total += part.length
|
13
|
+
body.write(part)
|
14
|
+
# Mechanize.log.debug("Read #{total} bytes") if Mechanize.log
|
15
|
+
Mechanize.log.debug("Read #{total} bytes") if Mechanize.log && !mpbar.suppress_logger? # (3of6)m
|
16
|
+
mpbar.inc(part.length) # (4of6)a
|
17
|
+
}
|
18
|
+
body.rewind
|
19
|
+
mpbar.finish # (5of6)a
|
20
|
+
Mechanize.log.debug("Read #{total} bytes") if Mechanize.log && !mpbar.suppress_logger? # (6of6)a
|
21
|
+
res_klass = Net::HTTPResponse::CODE_TO_OBJ[@response.code.to_s]
|
22
|
+
raise ResponseCodeError.new(@response) unless res_klass
|
23
|
+
|
24
|
+
# Net::HTTP ignores EOFError if Content-length is given, so we emulate it here.
|
25
|
+
unless res_klass <= Net::HTTPRedirection
|
26
|
+
raise EOFError if (!params[:request].is_a?(Net::HTTP::Head)) && @response.content_length() && @response.content_length() != total
|
27
|
+
end
|
28
|
+
|
29
|
+
@response.each_header { |k,v|
|
30
|
+
Mechanize.log.debug("response-header: #{ k } => #{ v }")
|
31
|
+
} if Mechanize.log
|
32
|
+
|
33
|
+
params[:response_body] = body
|
34
|
+
params[:res_klass] = res_klass
|
35
|
+
super
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
@@ -0,0 +1,44 @@
|
|
1
|
+
class Mechanize
|
2
|
+
include MechanizeProgressBarAPI # (1of7)a
|
3
|
+
class HTTP
|
4
|
+
class Agent
|
5
|
+
def response_read response, request
|
6
|
+
body_io = StringIO.new
|
7
|
+
body_io.set_encoding Encoding::BINARY if body_io.respond_to? :set_encoding
|
8
|
+
total = 0
|
9
|
+
mpbar = MechanizeProgressBar.new(self.context, request, response) # (2of7)a
|
10
|
+
|
11
|
+
begin
|
12
|
+
response.read_body { |part|
|
13
|
+
total += part.length
|
14
|
+
body_io.write(part)
|
15
|
+
# log.debug("Read #{part.length} bytes (#{total} total)") if log
|
16
|
+
log.debug("Read #{part.length} bytes (#{total} total)") if log && !mpbar.suppress_logger? # (3of7)m
|
17
|
+
mpbar.inc(part.length) # (4of7)a
|
18
|
+
}
|
19
|
+
rescue Net::HTTP::Persistent::Error => e
|
20
|
+
body_io.rewind
|
21
|
+
raise Mechanize::ResponseReadError.new(e, response, body_io)
|
22
|
+
ensure # (5of7)a
|
23
|
+
mpbar.finish # (6of7)a
|
24
|
+
end
|
25
|
+
|
26
|
+
body_io.rewind
|
27
|
+
log.debug("Read #{total} bytes total") if log && !mpbar.suppress_logger? # (7of7)a
|
28
|
+
|
29
|
+
raise Mechanize::ResponseCodeError, response if
|
30
|
+
Net::HTTPUnknownResponse === response
|
31
|
+
|
32
|
+
content_length = response.content_length
|
33
|
+
|
34
|
+
unless Net::HTTP::Head === request or Net::HTTPRedirection === response then
|
35
|
+
raise EOFError, "Content-Length (#{content_length}) does not match " \
|
36
|
+
"response body length (#{body_io.length})" if
|
37
|
+
content_length and content_length != body_io.length
|
38
|
+
end
|
39
|
+
|
40
|
+
body_io
|
41
|
+
end
|
42
|
+
end
|
43
|
+
end
|
44
|
+
end
|
@@ -0,0 +1,89 @@
|
|
1
|
+
class MechanizeProgressBar # :nodoc:
|
2
|
+
|
3
|
+
def self.register(mech, pbar_opts)
|
4
|
+
unregister(mech)
|
5
|
+
mech.progressbar_option.update(pbar_opts)
|
6
|
+
|
7
|
+
reset_reservation = mech.progressbar_option['reserve!']
|
8
|
+
mech.progressbar_option['reset!'] = reset_reservation
|
9
|
+
end
|
10
|
+
|
11
|
+
def self.unregister(mech)
|
12
|
+
mech.progressbar_option.clear
|
13
|
+
end
|
14
|
+
|
15
|
+
def self.unregister_next_init(mech)
|
16
|
+
mech.progressbar_option['reserve!'] = true
|
17
|
+
end
|
18
|
+
|
19
|
+
def initialize(mech, request, response)
|
20
|
+
pbar_opts = mech.progressbar_option
|
21
|
+
self.class.unregister(mech) if pbar_opts['reset!']
|
22
|
+
|
23
|
+
_first_time = pbar_opts.empty?
|
24
|
+
_unusual_response = response.code != '200'
|
25
|
+
|
26
|
+
case
|
27
|
+
when _first_time, _unusual_response
|
28
|
+
@progressbar = nil
|
29
|
+
else
|
30
|
+
@progressbar = progressbar_new(pbar_opts, request, response)
|
31
|
+
end
|
32
|
+
|
33
|
+
if mech.log
|
34
|
+
log_output = mech.log.instance_variable_get(:@logdev).dev
|
35
|
+
pbar_output = @progressbar.instance_variable_get(:@out)
|
36
|
+
@suppress_logger = true if log_output == pbar_output
|
37
|
+
end
|
38
|
+
|
39
|
+
@suppress_logger = pbar_opts[:suppress_logger] if pbar_opts.has_key?(:suppress_logger)
|
40
|
+
end
|
41
|
+
|
42
|
+
def inc(step)
|
43
|
+
@progressbar.inc(step) if @progressbar
|
44
|
+
end
|
45
|
+
|
46
|
+
def finish
|
47
|
+
@progressbar.finish if @progressbar
|
48
|
+
end
|
49
|
+
|
50
|
+
def suppress_logger?
|
51
|
+
@suppress_logger
|
52
|
+
end
|
53
|
+
|
54
|
+
private
|
55
|
+
|
56
|
+
def progressbar_new(pbar_opts, request, response)
|
57
|
+
out = pbar_opts[:out] || pbar_opts[:output] || $stderr
|
58
|
+
if pbar_opts[:single] then
|
59
|
+
title = pbar_opts[:title] || request['Host']
|
60
|
+
format = pbar_opts[:format]
|
61
|
+
format_arguments = pbar_opts[:format_arguments]
|
62
|
+
else
|
63
|
+
title = pbar_opts[:title] || ""
|
64
|
+
format = pbar_opts[:format] || "%3d%% %s %s"
|
65
|
+
format_arguments = pbar_opts[:format_arguments] || [:percentage, :bar, :stat_for_file_transfer]
|
66
|
+
out.print "#{pbar_opts[:title]||uri(request)}\n"
|
67
|
+
end
|
68
|
+
total = pbar_opts[:total] || filesize(response)
|
69
|
+
pbar_class = pbar_opts[:reversed] ? ReversedProgressBar : ProgressBar
|
70
|
+
|
71
|
+
progressbar = pbar_class.new(title, total, out)
|
72
|
+
|
73
|
+
progressbar.file_transfer_mode unless pbar_opts[:file_transfer_mode]
|
74
|
+
progressbar.format = format if format
|
75
|
+
progressbar.format_arguments = format_arguments if format_arguments
|
76
|
+
progressbar
|
77
|
+
end
|
78
|
+
|
79
|
+
def filesize(response)
|
80
|
+
content_length = response['content-length']
|
81
|
+
content_length ? content_length.to_i : 0
|
82
|
+
end
|
83
|
+
|
84
|
+
def uri(request)
|
85
|
+
scheme = request.class.to_s =~ /https/i ? 'https' : 'http'
|
86
|
+
URI.parse("#{scheme}://#{request['Host']}#{request.path}").to_s
|
87
|
+
end
|
88
|
+
|
89
|
+
end
|
@@ -1,6 +1,6 @@
|
|
1
1
|
require File.expand_path(File.dirname(__FILE__) + '../../../spec_helper')
|
2
2
|
|
3
|
-
### NOTE:
|
3
|
+
### NOTE: we swap $stderr on MechanizeProgressBarAPI - "accepts agent as agent" spec
|
4
4
|
|
5
5
|
describe MechanizeProgressBarAPI do
|
6
6
|
|
@@ -13,6 +13,7 @@ describe MechanizeProgressBarAPI do
|
|
13
13
|
end
|
14
14
|
|
15
15
|
before :each do
|
16
|
+
WebMock.reset!
|
16
17
|
WebMock.stub_request(:get, @url).to_return(@headers_and_body)
|
17
18
|
end
|
18
19
|
|
@@ -32,40 +33,55 @@ describe MechanizeProgressBarAPI do
|
|
32
33
|
@agent = Mechanize.new
|
33
34
|
@output = StringIO.new
|
34
35
|
end
|
36
|
+
|
35
37
|
it "accepts Hash as ProgressBar option" do
|
36
38
|
@agent.progressbar(:out => @output){@agent.get(@url)}
|
37
39
|
progressbar.should work_fine
|
38
40
|
end
|
39
41
|
|
40
|
-
it "
|
41
|
-
|
42
|
-
o.extend(MechanizeProgressBarAPI)
|
43
|
-
backup = $stderr
|
44
|
-
$stderr = @output
|
45
|
-
o.progressbar(@agent){@agent.get(@url)}
|
46
|
-
$stderr = backup
|
42
|
+
it "with block" do
|
43
|
+
@agent.progressbar(:out => @output){@agent.get(@url)}
|
47
44
|
progressbar.should work_fine
|
48
45
|
end
|
49
46
|
|
50
|
-
|
51
|
-
|
52
|
-
|
53
|
-
|
54
|
-
|
55
|
-
|
47
|
+
describe "included/extended use" do
|
48
|
+
it "accepts agent as agent" do
|
49
|
+
o = Object.new
|
50
|
+
o.extend(MechanizeProgressBarAPI)
|
51
|
+
backup = $stderr
|
52
|
+
$stderr = @output
|
53
|
+
o.progressbar(@agent){@agent.get(@url)}
|
54
|
+
$stderr = backup
|
55
|
+
progressbar.should work_fine
|
56
|
+
end
|
56
57
|
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
58
|
+
it "accepts (agent, pbar_opt) arguments" do
|
59
|
+
o = Object.new
|
60
|
+
o.extend(MechanizeProgressBarAPI)
|
61
|
+
o.progressbar(@agent, :out => @output){@agent.get(@url)}
|
62
|
+
progressbar.should work_fine
|
63
|
+
end
|
62
64
|
end
|
63
65
|
|
64
|
-
|
65
|
-
|
66
|
-
|
67
|
-
|
68
|
-
|
66
|
+
describe "method chain" do
|
67
|
+
it "works fine" do
|
68
|
+
@agent.progressbar(:out => @output).get(@url)
|
69
|
+
progressbar.should work_fine
|
70
|
+
end
|
71
|
+
|
72
|
+
it "options to method chain are reseted next time" do
|
73
|
+
out1 = StringIO.new
|
74
|
+
@agent.progressbar(:out => out1, :title => 'AAAA').get(@url)
|
75
|
+
out1.string.should match(/AAAA/)
|
76
|
+
|
77
|
+
out2 = StringIO.new
|
78
|
+
@agent.progressbar(:out => out2, :title => 'BBBB').get(@url)
|
79
|
+
out2.string.should match(/BBBB/)
|
80
|
+
|
81
|
+
out3 = StringIO.new
|
82
|
+
@agent.progressbar(:out => out3).get(@url)
|
83
|
+
out3.string.should_not match(/BBBB/)
|
84
|
+
end
|
69
85
|
end
|
70
86
|
|
71
87
|
it "default ProgressBar title is uri.host" do
|
@@ -102,10 +118,6 @@ describe MechanizeProgressBarAPI do
|
|
102
118
|
progressbar.should match(/999B/)
|
103
119
|
end
|
104
120
|
|
105
|
-
it "default ProgressBar out is $stderr" do
|
106
|
-
# TODO:
|
107
|
-
end
|
108
|
-
|
109
121
|
it "agent.progressbar(:out => io) changes output to io object" do
|
110
122
|
@agent.progressbar(:out => @output){
|
111
123
|
lambda{@agent.get(@url)}.should_not raise_error
|
@@ -130,62 +142,111 @@ describe MechanizeProgressBarAPI do
|
|
130
142
|
progressbar.should be_include(stat)
|
131
143
|
end
|
132
144
|
|
133
|
-
it "when
|
145
|
+
it "when Exception occurs before fetching, raise it and show nothing" do
|
146
|
+
WebMock.stub_request(:get, @url).to_timeout
|
147
|
+
if Mechanize::VERSION == '1.0.0'
|
148
|
+
error = Timeout::Error
|
149
|
+
error_message = /execution expired/
|
150
|
+
else
|
151
|
+
error = Net::HTTP::Persistent::Error
|
152
|
+
error_message = /execution expired - Timeout::Error/
|
153
|
+
end
|
154
|
+
|
155
|
+
@agent.progressbar(:output => @output) do
|
156
|
+
lambda{ @agent.get(@url) }.should raise_error(error, error_message)
|
157
|
+
end
|
158
|
+
progressbar.should be_empty
|
159
|
+
end
|
160
|
+
|
161
|
+
## TODO: it requires a webmock adapter for net-http-persistent
|
162
|
+
# it "when HTTP status error occurs in reading, raise it and show nothing" do
|
163
|
+
# WebMock.stub_request(:get, @url).to_raise(Errno::ECONNREFUSED)
|
164
|
+
# lambda{
|
165
|
+
# @agent.progressbar{ @agent.get(@url)}
|
166
|
+
# }.should raise_error(Net::HTTP::Persistent::Error) # raises ECONNREFUSED
|
167
|
+
# end
|
168
|
+
|
169
|
+
it "when HTTP status error occurs after fetching, raise it and show nothing" do
|
134
170
|
url_404 = 'http://host/404'
|
135
171
|
WebMock.stub_request(:get, url_404).to_return(:status => ['404', 'Not Found'])
|
136
172
|
lambda{
|
137
173
|
@agent.progressbar(:output => @output) do
|
138
|
-
@agent.pre_connect_hooks[0].should be_kind_of(Proc)
|
139
174
|
@agent.get(url_404)
|
140
175
|
end
|
141
176
|
}.should raise_error(Mechanize::ResponseCodeError)
|
142
|
-
@agent.pre_connect_hooks.should be_empty
|
143
177
|
progressbar.should be_empty
|
144
178
|
end
|
145
179
|
|
146
|
-
|
147
|
-
|
148
|
-
|
149
|
-
|
150
|
-
lambda{@agent.get(@url)}.should raise_error(Timeout::Error)
|
180
|
+
describe "with Logger" do
|
181
|
+
|
182
|
+
before :all do
|
183
|
+
require 'logger'
|
151
184
|
end
|
152
|
-
@agent.pre_connect_hooks.should be_empty
|
153
|
-
progressbar.should be_empty
|
154
|
-
end
|
155
185
|
|
156
|
-
|
157
|
-
|
158
|
-
|
159
|
-
|
160
|
-
@agent.progressbar(:output => out){@agent.get(@url)}
|
186
|
+
before :each do
|
187
|
+
@out = StringIO.new
|
188
|
+
@agent.log = Logger.new(@out)
|
189
|
+
end
|
161
190
|
|
162
|
-
|
163
|
-
|
164
|
-
|
165
|
-
### 100% |ooooooooooooooooooooooooooooooooooooooo| 1000B nnn.nKB/s Time: 00:00:00
|
166
|
-
re = /http:\/\/uri\.host\/uri\.path\s+D, \[.+?\] DEBUG -- : Read 1000 bytes\s+100% |o+|\s+1000B/
|
167
|
-
out.string.should match(re)
|
168
|
-
end
|
191
|
+
def progressbar
|
192
|
+
@out.string
|
193
|
+
end
|
169
194
|
|
170
|
-
|
171
|
-
|
172
|
-
|
173
|
-
|
174
|
-
|
195
|
+
it "{:suppress_logger => true} does nothing when no logger" do
|
196
|
+
@agent.log = nil
|
197
|
+
@agent.progressbar(:output => @out, :suppress_logger => false){@agent.get(@url)}
|
198
|
+
progressbar.should work_fine
|
199
|
+
progressbar.should_not match(/Read 1000 bytes/)
|
200
|
+
end
|
201
|
+
|
202
|
+
it "When Logger and ProgressBar have same outputs, Log output is suppressed " do
|
203
|
+
@agent.progressbar(:output => @out){@agent.get(@url)}
|
204
|
+
if Mechanize::VERSION == '1.0.0'
|
205
|
+
re = /^I, \[.+^http:\/\/.+?Time: \d\d:\d\d:\d\d\nD, /m
|
206
|
+
# matches:
|
207
|
+
### D, [2011-06-29...
|
208
|
+
### http://uri.host//uri.path...
|
209
|
+
### ...
|
210
|
+
### (showing progressbar)
|
211
|
+
### ...
|
212
|
+
### ...1000B 195.7KB/s Time: 00:00:00
|
213
|
+
### D, [2011-06-29...
|
214
|
+
progressbar.should match(re)
|
215
|
+
else
|
216
|
+
re = /^D, \[.+^http:\/\/.+?Time: \d\d:\d\d:\d\d\n\Z/m
|
217
|
+
# matches:
|
218
|
+
### D, [2011-06-29...
|
219
|
+
### http://uri.host//uri.path...
|
220
|
+
### ...
|
221
|
+
### (showing progressbar)
|
222
|
+
### ...
|
223
|
+
### ...1000B 195.7KB/s Time: 00:00:00\n
|
224
|
+
progressbar.should match(re)
|
225
|
+
end
|
226
|
+
end
|
175
227
|
|
176
|
-
|
177
|
-
|
178
|
-
|
179
|
-
|
180
|
-
out.string.should match(before_progressbar)
|
228
|
+
it "When Logger output is diffrent from ProgressBar one, Mechanize shows Log as usual" do
|
229
|
+
logger_out = StringIO.new
|
230
|
+
@agent.log = Logger.new(logger_out)
|
231
|
+
@agent.progressbar(:output => @out){@agent.get(@url)}
|
181
232
|
|
182
|
-
|
183
|
-
|
184
|
-
|
185
|
-
|
186
|
-
out.string.should match(after_progressbar)
|
187
|
-
end
|
233
|
+
progressbar.should work_fine
|
234
|
+
progressbar.should_not match(/Read 1000 bytes/)
|
235
|
+
logger_out.string.should match(/Read 1000 bytes/)
|
236
|
+
end
|
188
237
|
|
238
|
+
it "{:suppress_logger => false} crashes ProgressBar output" do
|
239
|
+
@agent.progressbar(:output => @out, :suppress_logger => false){@agent.get(@url)}
|
240
|
+
|
241
|
+
re = /D, \[.+?\] DEBUG -- : Read 1000 bytes \(1000 total\)\r?\n100% \|o+\|\s+1000B/
|
242
|
+
|
243
|
+
# matches:
|
244
|
+
### D, [(date+time)] DEBUG -- : Read 1000 bytes
|
245
|
+
### 100% |ooooooooooooooooooooooooooooooooooooooo| 1000B nnn.nKB/s Time: 00:00:00
|
246
|
+
progressbar.should match(re)
|
247
|
+
end
|
248
|
+
|
249
|
+
end
|
189
250
|
end
|
190
251
|
end
|
191
252
|
|