httplog 0.2.0 → 0.2.1
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +102 -0
- data/lib/httplog.rb +3 -1
- data/lib/httplog/adapters/httpclient.rb +39 -0
- data/lib/httplog/adapters/net_http.rb +38 -0
- data/lib/httplog/http_log.rb +6 -66
- data/lib/httplog/version.rb +1 -1
- metadata +53 -4
- data/README.rdoc +0 -101
data/README.md
ADDED
@@ -0,0 +1,102 @@
|
|
1
|
+
## httplog
|
2
|
+
|
3
|
+
Log outgoing HTTP requests made from your application.
|
4
|
+
See the [blog post](http://trusche.github.com/blog/2011/09/29/logging-outgoing-http-requests/)
|
5
|
+
for more details.
|
6
|
+
|
7
|
+
So far this gem works with the following HTTP libraries:
|
8
|
+
|
9
|
+
* Net::HTTP
|
10
|
+
* OpenURI
|
11
|
+
* HTTPClient
|
12
|
+
|
13
|
+
In theory, it should also work with any library built on top of these. But since
|
14
|
+
the difference between theory and practice is bigger in practice than in theory, YMMV.
|
15
|
+
|
16
|
+
This is very much a development and debugging tool; it is **not recommended** to
|
17
|
+
use this in a production environment.
|
18
|
+
|
19
|
+
### Installation
|
20
|
+
|
21
|
+
gem install httplog
|
22
|
+
|
23
|
+
### Usage
|
24
|
+
|
25
|
+
require 'httplog'
|
26
|
+
|
27
|
+
By default, this will log all outgoing HTTP requests and their responses to $stdout on DEBUG level.
|
28
|
+
|
29
|
+
### Configuration
|
30
|
+
|
31
|
+
You can override the following default options:
|
32
|
+
|
33
|
+
HttpLog.options[:logger] = Logger.new($stdout)
|
34
|
+
HttpLog.options[:severity] = Logger::Severity::DEBUG
|
35
|
+
HttpLog.options[:log_connect] = true
|
36
|
+
HttpLog.options[:log_request] = true
|
37
|
+
HttpLog.options[:log_headers] = false
|
38
|
+
HttpLog.options[:log_data] = true
|
39
|
+
HttpLog.options[:log_status] = true
|
40
|
+
HttpLog.options[:log_response] = true
|
41
|
+
HttpLog.options[:log_benchmark] = true
|
42
|
+
HttpLog.options[:compact_log] = false # setting this to true will make all "log_*" options redundant
|
43
|
+
|
44
|
+
So if you want to use this in a Rails app:
|
45
|
+
|
46
|
+
# config/initializers/httplog.rb
|
47
|
+
HttpLog.options[:logger] = Rails.logger
|
48
|
+
|
49
|
+
### Example
|
50
|
+
|
51
|
+
With the default configuration, the log output might look like this:
|
52
|
+
|
53
|
+
D, [2012-11-21T15:09:03.532970 #6857] DEBUG -- : [httplog] Connecting: localhost
|
54
|
+
D, [2012-11-21T15:09:03.533877 #6857] DEBUG -- : [httplog] Sending: GET http://localhost:9292/index.html
|
55
|
+
D, [2012-11-21T15:09:03.534499 #6857] DEBUG -- : [httplog] Status: 200
|
56
|
+
D, [2012-11-21T15:09:03.534544 #6857] DEBUG -- : [httplog] Benchmark: 0.00057 seconds
|
57
|
+
D, [2012-11-21T15:09:03.534578 #6857] DEBUG -- : [httplog] Response:
|
58
|
+
<html>
|
59
|
+
<head>
|
60
|
+
<title>Test Page</title>
|
61
|
+
</head>
|
62
|
+
<body>
|
63
|
+
<h1>This is the test page.</h1>
|
64
|
+
</body>
|
65
|
+
</html>
|
66
|
+
|
67
|
+
|
68
|
+
### Known Issues
|
69
|
+
|
70
|
+
* When using OpenURI, the reading of the HTTP response body is deferred,
|
71
|
+
so it is not available for logging. This will be noted in the logging statement:
|
72
|
+
|
73
|
+
D, [2012-11-21T15:09:03.547005 #6857] DEBUG -- : [httplog] Connecting: localhost
|
74
|
+
D, [2012-11-21T15:09:03.547938 #6857] DEBUG -- : [httplog] Sending: GET http://localhost:9292/index.html
|
75
|
+
D, [2012-11-21T15:09:03.548615 #6857] DEBUG -- : [httplog] Status: 200
|
76
|
+
D, [2012-11-21T15:09:03.548662 #6857] DEBUG -- : [httplog] Benchmark: 0.000617 seconds
|
77
|
+
D, [2012-11-21T15:09:03.548695 #6857] DEBUG -- : [httplog] Response: (not available yet)
|
78
|
+
|
79
|
+
* When using HTTPClient, the TCP connection establishment will be logged
|
80
|
+
*after* the HTTP request and headers, due to the way HTTPClient is organized.
|
81
|
+
|
82
|
+
### Running the specs
|
83
|
+
|
84
|
+
Make sure you have the necessary dependencies installed by running `bundle install`.
|
85
|
+
Then simply run `bundle exec rspec spec`.
|
86
|
+
This will launch a simple rack server on port 9292 and run all tests locally against that server.
|
87
|
+
|
88
|
+
### Contributing
|
89
|
+
|
90
|
+
If you have any issues with httplog,
|
91
|
+
or feature requests,
|
92
|
+
please [add an issue](https://github.com/trusche/httplog/issues) on GitHub
|
93
|
+
or fork the project and send a pull request.
|
94
|
+
Please include passing specs with all pull requests.
|
95
|
+
|
96
|
+
### Contributors
|
97
|
+
|
98
|
+
Thanks to these fine folks for contributing pull requests:
|
99
|
+
|
100
|
+
* [Eric Cohen](https://github.com/eirc)
|
101
|
+
* [Nikos Dimitrakopoulos](https://github.com/nikosd)
|
102
|
+
* [Marcos Hack](https://github.com/marcoshack)
|
data/lib/httplog.rb
CHANGED
@@ -0,0 +1,39 @@
|
|
1
|
+
|
2
|
+
if defined?(::HTTPClient)
|
3
|
+
class HTTPClient
|
4
|
+
private
|
5
|
+
alias_method :orig_do_get_block, :do_get_block
|
6
|
+
|
7
|
+
def do_get_block(req, proxy, conn, &block)
|
8
|
+
HttpLog.log_request(req.header.request_method, req.header.request_uri)
|
9
|
+
HttpLog.log_headers(req.headers)
|
10
|
+
HttpLog.log_data(req.body) if req.header.request_method == "POST"
|
11
|
+
|
12
|
+
retryable_response = nil
|
13
|
+
bm = Benchmark.realtime do
|
14
|
+
begin
|
15
|
+
orig_do_get_block(req, proxy, conn, &block)
|
16
|
+
rescue RetryableResponse => e
|
17
|
+
retryable_response = e
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
res = conn.pop
|
22
|
+
HttpLog.log_compact(req.header.request_method, req.header.request_uri, res.status_code, bm)
|
23
|
+
HttpLog.log_status(res.status_code)
|
24
|
+
HttpLog.log_benchmark(bm)
|
25
|
+
HttpLog.log_body(res.body)
|
26
|
+
conn.push(res)
|
27
|
+
|
28
|
+
raise retryable_response if retryable_response != nil
|
29
|
+
end
|
30
|
+
|
31
|
+
class Session
|
32
|
+
alias_method :orig_create_socket, :create_socket
|
33
|
+
def create_socket(site)
|
34
|
+
HttpLog.log_connection(site.host, site.port)
|
35
|
+
orig_create_socket(site)
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
39
|
+
end
|
@@ -0,0 +1,38 @@
|
|
1
|
+
module Net
|
2
|
+
class HTTP
|
3
|
+
alias_method(:orig_request, :request) unless method_defined?(:orig_request)
|
4
|
+
alias_method(:orig_connect, :connect) unless method_defined?(:orig_connect)
|
5
|
+
|
6
|
+
def request(req, body = nil, &block)
|
7
|
+
|
8
|
+
url = "http://#{@address}:#{@port}#{req.path}"
|
9
|
+
|
10
|
+
if started? && !HttpLog.options[:compact_log]
|
11
|
+
HttpLog.log_request(req.method, url)
|
12
|
+
HttpLog.log_headers(req.each_header.collect)
|
13
|
+
# A bit convoluted becase post_form uses form_data= to assign the data, so
|
14
|
+
# in that case req.body will be empty.
|
15
|
+
HttpLog::log_data(req.body.nil? || req.body.size == 0 ? body : req.body) if req.method == 'POST'
|
16
|
+
end
|
17
|
+
|
18
|
+
bm = Benchmark.realtime do
|
19
|
+
@response = orig_request(req, body, &block)
|
20
|
+
end
|
21
|
+
|
22
|
+
if started?
|
23
|
+
HttpLog.log_compact(req.method, url, @response.code, bm)
|
24
|
+
HttpLog.log_status(@response.code)
|
25
|
+
HttpLog.log_benchmark(bm)
|
26
|
+
HttpLog.log_body(@response.body)
|
27
|
+
end
|
28
|
+
|
29
|
+
@response
|
30
|
+
end
|
31
|
+
|
32
|
+
def connect
|
33
|
+
HttpLog.log_connection(@address, @port) unless started?
|
34
|
+
orig_connect
|
35
|
+
end
|
36
|
+
end
|
37
|
+
|
38
|
+
end
|
data/lib/httplog/http_log.rb
CHANGED
@@ -30,6 +30,11 @@ module HttpLog
|
|
30
30
|
@@options[:logger].add(@@options[:severity]) { "[httplog] #{msg}" }
|
31
31
|
end
|
32
32
|
|
33
|
+
def log_connection(host, port = nil)
|
34
|
+
return if options[:compact_log] || !options[:log_connect]
|
35
|
+
log("Connecting: #{[host, port].compact.join(":")}")
|
36
|
+
end
|
37
|
+
|
33
38
|
def log_request(method, uri)
|
34
39
|
return if options[:compact_log] || !options[:log_request]
|
35
40
|
log("Sending: #{method.to_s.upcase} #{uri}")
|
@@ -70,72 +75,7 @@ module HttpLog
|
|
70
75
|
|
71
76
|
def log_compact(method, uri, status, seconds)
|
72
77
|
return unless options[:compact_log]
|
73
|
-
log("#{method} #{uri} completed with status code #{status} in #{seconds} seconds")
|
74
|
-
end
|
75
|
-
end
|
76
|
-
end
|
77
|
-
|
78
|
-
module Net
|
79
|
-
class HTTP
|
80
|
-
alias_method(:orig_request, :request) unless method_defined?(:orig_request)
|
81
|
-
alias_method(:orig_connect, :connect) unless method_defined?(:orig_connect)
|
82
|
-
|
83
|
-
def request(req, body = nil, &block)
|
84
|
-
|
85
|
-
url = "http://#{@address}:#{@port}#{req.path}"
|
86
|
-
|
87
|
-
if started? && !HttpLog.options[:compact_log]
|
88
|
-
HttpLog.log_request(req.method, url)
|
89
|
-
HttpLog.log_headers(req.each_header.collect)
|
90
|
-
# A bit convoluted becase post_form uses form_data= to assign the data, so
|
91
|
-
# in that case req.body will be empty.
|
92
|
-
HttpLog::log_data(req.body.nil? || req.body.size == 0 ? body : req.body) if req.method == 'POST'
|
93
|
-
end
|
94
|
-
|
95
|
-
bm = Benchmark.realtime do
|
96
|
-
@response = orig_request(req, body, &block)
|
97
|
-
end
|
98
|
-
|
99
|
-
if started?
|
100
|
-
HttpLog.log_compact(req.method, url, @response.code, bm)
|
101
|
-
HttpLog.log_status(@response.code)
|
102
|
-
HttpLog.log_benchmark(bm)
|
103
|
-
HttpLog.log_body(@response.body)
|
104
|
-
end
|
105
|
-
|
106
|
-
@response
|
107
|
-
end
|
108
|
-
|
109
|
-
def connect
|
110
|
-
unless started? || HttpLog.options[:compact_log]
|
111
|
-
HttpLog::log("Connecting: #{@address}") if HttpLog.options[:log_connect]
|
112
|
-
end
|
113
|
-
orig_connect
|
114
|
-
end
|
115
|
-
end
|
116
|
-
|
117
|
-
end
|
118
|
-
|
119
|
-
if defined?(::HTTPClient)
|
120
|
-
class HTTPClient
|
121
|
-
private
|
122
|
-
alias_method :orig_do_request, :do_request
|
123
|
-
|
124
|
-
def do_request(method, uri, query, body, header, &block)
|
125
|
-
HttpLog.log_request(method, uri)
|
126
|
-
HttpLog.log_headers(header)
|
127
|
-
HttpLog.log_data(body) if method == :post
|
128
|
-
|
129
|
-
bm = Benchmark.realtime do
|
130
|
-
@response = orig_do_request(method, uri, query, body, header, &block)
|
131
|
-
end
|
132
|
-
|
133
|
-
HttpLog.log_compact(method, uri, @response.status, bm)
|
134
|
-
HttpLog.log_status(@response.status)
|
135
|
-
HttpLog.log_benchmark(bm)
|
136
|
-
HttpLog.log_body(@response.body)
|
137
|
-
|
138
|
-
@response
|
78
|
+
log("#{method.to_s.upcase} #{uri} completed with status code #{status} in #{seconds} seconds")
|
139
79
|
end
|
140
80
|
end
|
141
81
|
end
|
data/lib/httplog/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: httplog
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
4
|
+
version: 0.2.1
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2012-11-
|
12
|
+
date: 2012-11-22 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: rspec
|
@@ -27,6 +27,54 @@ dependencies:
|
|
27
27
|
- - ! '>='
|
28
28
|
- !ruby/object:Gem::Version
|
29
29
|
version: '0'
|
30
|
+
- !ruby/object:Gem::Dependency
|
31
|
+
name: rack
|
32
|
+
requirement: !ruby/object:Gem::Requirement
|
33
|
+
none: false
|
34
|
+
requirements:
|
35
|
+
- - ! '>='
|
36
|
+
- !ruby/object:Gem::Version
|
37
|
+
version: '0'
|
38
|
+
type: :development
|
39
|
+
prerelease: false
|
40
|
+
version_requirements: !ruby/object:Gem::Requirement
|
41
|
+
none: false
|
42
|
+
requirements:
|
43
|
+
- - ! '>='
|
44
|
+
- !ruby/object:Gem::Version
|
45
|
+
version: '0'
|
46
|
+
- !ruby/object:Gem::Dependency
|
47
|
+
name: thin
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
49
|
+
none: false
|
50
|
+
requirements:
|
51
|
+
- - ! '>='
|
52
|
+
- !ruby/object:Gem::Version
|
53
|
+
version: '0'
|
54
|
+
type: :development
|
55
|
+
prerelease: false
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
57
|
+
none: false
|
58
|
+
requirements:
|
59
|
+
- - ! '>='
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '0'
|
62
|
+
- !ruby/object:Gem::Dependency
|
63
|
+
name: httpclient
|
64
|
+
requirement: !ruby/object:Gem::Requirement
|
65
|
+
none: false
|
66
|
+
requirements:
|
67
|
+
- - ! '>='
|
68
|
+
- !ruby/object:Gem::Version
|
69
|
+
version: '0'
|
70
|
+
type: :development
|
71
|
+
prerelease: false
|
72
|
+
version_requirements: !ruby/object:Gem::Requirement
|
73
|
+
none: false
|
74
|
+
requirements:
|
75
|
+
- - ! '>='
|
76
|
+
- !ruby/object:Gem::Version
|
77
|
+
version: '0'
|
30
78
|
description: ! "Log outgoing HTTP requests made from your application. Helpful for
|
31
79
|
tracking API calls\n of third party gems that don't provide
|
32
80
|
their own log output."
|
@@ -35,12 +83,14 @@ executables: []
|
|
35
83
|
extensions: []
|
36
84
|
extra_rdoc_files: []
|
37
85
|
files:
|
86
|
+
- lib/httplog/adapters/httpclient.rb
|
87
|
+
- lib/httplog/adapters/net_http.rb
|
38
88
|
- lib/httplog/http_log.rb
|
39
89
|
- lib/httplog/version.rb
|
40
90
|
- lib/httplog.rb
|
41
91
|
- MIT-LICENSE
|
42
92
|
- Rakefile
|
43
|
-
- README.
|
93
|
+
- README.md
|
44
94
|
homepage: http://github.com/trusche/httplog
|
45
95
|
licenses: []
|
46
96
|
post_install_message:
|
@@ -66,4 +116,3 @@ signing_key:
|
|
66
116
|
specification_version: 3
|
67
117
|
summary: Logs outgoing Net::HTTP requests.
|
68
118
|
test_files: []
|
69
|
-
has_rdoc:
|
data/README.rdoc
DELETED
@@ -1,101 +0,0 @@
|
|
1
|
-
== httplog
|
2
|
-
|
3
|
-
Log outgoing HTTP requests made from your application.
|
4
|
-
See the {blog post}[http://trusche.github.com/blog/2011/09/29/logging-outgoing-http-requests/]
|
5
|
-
for more details.
|
6
|
-
|
7
|
-
So far this gem works with the following HTTP libraries:
|
8
|
-
|
9
|
-
* Net::HTTP
|
10
|
-
* OpenURI
|
11
|
-
* HTTPClient
|
12
|
-
|
13
|
-
In theory, it should also work with any library built on top of these. But since
|
14
|
-
the difference between theory and practice is bigger in practice than in theory, YMMV.
|
15
|
-
|
16
|
-
*PLEASE NOTE* that this is very much a development tool; it is *not recommended* to
|
17
|
-
use this in a production environment.
|
18
|
-
|
19
|
-
=== Installation
|
20
|
-
|
21
|
-
gem install httplog
|
22
|
-
|
23
|
-
=== Usage
|
24
|
-
|
25
|
-
require 'httplog'
|
26
|
-
|
27
|
-
By default, this will log all outgoing HTTP requests and their responses to $stdout on DEBUG level.
|
28
|
-
|
29
|
-
=== Configuration
|
30
|
-
|
31
|
-
You can override the following default options:
|
32
|
-
|
33
|
-
HttpLog.options[:logger] = Logger.new($stdout)
|
34
|
-
HttpLog.options[:severity] = Logger::Severity::DEBUG
|
35
|
-
HttpLog.options[:log_connect] = true
|
36
|
-
HttpLog.options[:log_request] = true
|
37
|
-
HttpLog.options[:log_headers] = false
|
38
|
-
HttpLog.options[:log_data] = true
|
39
|
-
HttpLog.options[:log_status] = true
|
40
|
-
HttpLog.options[:log_response] = true
|
41
|
-
HttpLog.options[:log_benchmark] = true
|
42
|
-
HttpLog.options[:compact_log] = false # setting this to true will make all "log_*" options redundant
|
43
|
-
|
44
|
-
So if you want to use this in a Rails app:
|
45
|
-
|
46
|
-
# config/initializers/httplog.rb
|
47
|
-
HttpLog.options[:logger] = Rails.logger
|
48
|
-
|
49
|
-
=== Example
|
50
|
-
|
51
|
-
With the default configuration, the log output might look like this:
|
52
|
-
|
53
|
-
D, [2012-11-21T15:09:03.532970 #6857] DEBUG -- : [httplog] Connecting: localhost
|
54
|
-
D, [2012-11-21T15:09:03.533877 #6857] DEBUG -- : [httplog] Sending: GET http://localhost:9292/index.html
|
55
|
-
D, [2012-11-21T15:09:03.534499 #6857] DEBUG -- : [httplog] Status: 200
|
56
|
-
D, [2012-11-21T15:09:03.534544 #6857] DEBUG -- : [httplog] Benchmark: 0.00057 seconds
|
57
|
-
D, [2012-11-21T15:09:03.534578 #6857] DEBUG -- : [httplog] Response:
|
58
|
-
<html>
|
59
|
-
<head>
|
60
|
-
<title>Test Page</title>
|
61
|
-
</head>
|
62
|
-
<body>
|
63
|
-
<h1>This is the test page.</h1>
|
64
|
-
</body>
|
65
|
-
</html>
|
66
|
-
|
67
|
-
|
68
|
-
=== Known Issues
|
69
|
-
|
70
|
-
* When using open-uri, the reading of the HTTP response body is deferred,
|
71
|
-
so it is not available for logging. This will be noted in the logging statement:
|
72
|
-
|
73
|
-
D, [2012-11-21T15:09:03.547005 #6857] DEBUG -- : [httplog] Connecting: localhost
|
74
|
-
D, [2012-11-21T15:09:03.547938 #6857] DEBUG -- : [httplog] Sending: GET http://localhost:9292/index.html
|
75
|
-
D, [2012-11-21T15:09:03.548615 #6857] DEBUG -- : [httplog] Status: 200
|
76
|
-
D, [2012-11-21T15:09:03.548662 #6857] DEBUG -- : [httplog] Benchmark: 0.000617 seconds
|
77
|
-
D, [2012-11-21T15:09:03.548695 #6857] DEBUG -- : [httplog] Response: (not available yet)
|
78
|
-
|
79
|
-
* When using HTTPClient, the +:log_connect+ option has no effect.
|
80
|
-
|
81
|
-
=== Running the specs
|
82
|
-
|
83
|
-
Make sure you have the necessary dependencies installed by running +bundle install+.
|
84
|
-
Then simply run +bundle exec rspec spec+.
|
85
|
-
This will launch a simple rack server on port 9292 and run all tests locally against that server.
|
86
|
-
|
87
|
-
=== Contributing
|
88
|
-
|
89
|
-
If you have any issues with httplog,
|
90
|
-
or feature requests,
|
91
|
-
please {add an issue}[https://github.com/trusche/httplog/issues] on GitHub
|
92
|
-
or fork the project and send a pull request.
|
93
|
-
Please include passing specs with all pull requests.
|
94
|
-
|
95
|
-
=== Contributors
|
96
|
-
|
97
|
-
Thanks to these fine folks for contributing pull requests:
|
98
|
-
|
99
|
-
* {Eric Cohen}[https://github.com/eirc]
|
100
|
-
* {Nikos Dimitrakopoulos}[https://github.com/nikosd]
|
101
|
-
* {Marcos Hack}[https://github.com/marcoshack]
|