scraped 0.3.0 → 0.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/README.md +3 -6
- data/lib/scraped/response/decorator/{absolute_urls.rb → clean_urls.rb} +1 -1
- data/lib/scraped/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e86824184d28b3141f65c48a5fc51e7502c7781b
|
4
|
+
data.tar.gz: d33f15a4b1e7094b2677dbf09c9573bbeb49fcff
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 64f7fec521757f72ac2b927cddd5bf958d4d0fca01838ceb0e089bb40435e0a2d79ad6f4d737a0c83b4bba3cd26bed1dc252a2a0225169c1b382d4951157704a
|
7
|
+
data.tar.gz: 23368dd65b5773b317159343820bc9bd44525e42a38b32404616a17a3d83cb6a1fc908c2852c98607c0baf1d1f71ef5a6fc15ceaab7ce30753fbe6d74def3431
|
data/CHANGELOG.md
CHANGED
@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
|
|
5
5
|
The format is based on [Keep a Changelog](http://keepachangelog.com/)
|
6
6
|
and this project adheres to [Semantic Versioning](http://semver.org/).
|
7
7
|
|
8
|
+
## 0.4.0 - 2017-03-14
|
9
|
+
|
10
|
+
### Changed
|
11
|
+
|
12
|
+
- The AbsoluteUrls decorator has been renamed to CleanUrls.
|
13
|
+
|
8
14
|
## 0.3.0 - 2017-01-10
|
9
15
|
|
10
16
|
### Changed
|
data/README.md
CHANGED
@@ -169,12 +169,9 @@ When you inherit from a class that already has decorators the child class will a
|
|
169
169
|
|
170
170
|
### Built in decorators
|
171
171
|
|
172
|
-
####
|
172
|
+
#### Clean link and image URLs
|
173
173
|
|
174
|
-
|
175
|
-
you are scraping absolute rather than relative. Scraped comes with support for
|
176
|
-
this out of the box via the `Scraped::Response::Decorator::AbsoluteUrls`
|
177
|
-
decorator.
|
174
|
+
You will likely want to normalize link and image urls on the page you are scraping. `Scraped::Response::Decorator::CleanUrls` ensures that each link is absolute and does not contain any encoded characters.
|
178
175
|
|
179
176
|
```ruby
|
180
177
|
require 'scraped'
|
@@ -183,7 +180,7 @@ class MemberPage < Scraped::HTML
|
|
183
180
|
decorator Scraped::Response::Decorator::AbsoluteUrls
|
184
181
|
|
185
182
|
field :image do
|
186
|
-
# Image url will be absolute thanks to the decorator.
|
183
|
+
# Image url will be absolute and encoded correctly thanks to the decorator.
|
187
184
|
noko.at_css('.profile-picture/@src').text
|
188
185
|
end
|
189
186
|
end
|
data/lib/scraped/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: scraped
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- EveryPolitician
|
@@ -148,7 +148,7 @@ files:
|
|
148
148
|
- lib/scraped/request/strategy/live_request.rb
|
149
149
|
- lib/scraped/response.rb
|
150
150
|
- lib/scraped/response/decorator.rb
|
151
|
-
- lib/scraped/response/decorator/
|
151
|
+
- lib/scraped/response/decorator/clean_urls.rb
|
152
152
|
- lib/scraped/response_decorator.rb
|
153
153
|
- lib/scraped/version.rb
|
154
154
|
- scraped.gemspec
|