scraped 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/README.md +3 -6
- data/lib/scraped/response/decorator/{absolute_urls.rb → clean_urls.rb} +1 -1
- data/lib/scraped/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: e86824184d28b3141f65c48a5fc51e7502c7781b
|
4
|
+
data.tar.gz: d33f15a4b1e7094b2677dbf09c9573bbeb49fcff
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 64f7fec521757f72ac2b927cddd5bf958d4d0fca01838ceb0e089bb40435e0a2d79ad6f4d737a0c83b4bba3cd26bed1dc252a2a0225169c1b382d4951157704a
|
7
|
+
data.tar.gz: 23368dd65b5773b317159343820bc9bd44525e42a38b32404616a17a3d83cb6a1fc908c2852c98607c0baf1d1f71ef5a6fc15ceaab7ce30753fbe6d74def3431
|
data/CHANGELOG.md
CHANGED
@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
|
|
5
5
|
The format is based on [Keep a Changelog](http://keepachangelog.com/)
|
6
6
|
and this project adheres to [Semantic Versioning](http://semver.org/).
|
7
7
|
|
8
|
+
## 0.4.0 - 2017-03-14
|
9
|
+
|
10
|
+
### Changed
|
11
|
+
|
12
|
+
- The AbsoluteUrls decorator has been renamed to CleanUrls.
|
13
|
+
|
8
14
|
## 0.3.0 - 2017-01-10
|
9
15
|
|
10
16
|
### Changed
|
data/README.md
CHANGED
@@ -169,12 +169,9 @@ When you inherit from a class that already has decorators the child class will a
|
|
169
169
|
|
170
170
|
### Built in decorators
|
171
171
|
|
172
|
-
####
|
172
|
+
#### Clean link and image URLs
|
173
173
|
|
174
|
-
|
175
|
-
you are scraping absolute rather than relative. Scraped comes with support for
|
176
|
-
this out of the box via the `Scraped::Response::Decorator::AbsoluteUrls`
|
177
|
-
decorator.
|
174
|
+
You will likely want to normalize link and image urls on the page you are scraping. `Scraped::Response::Decorator::CleanUrls` ensures that each link is absolute and does not contain any encoded characters.
|
178
175
|
|
179
176
|
```ruby
|
180
177
|
require 'scraped'
|
@@ -183,7 +180,7 @@ class MemberPage < Scraped::HTML
|
|
183
180
|
decorator Scraped::Response::Decorator::AbsoluteUrls
|
184
181
|
|
185
182
|
field :image do
|
186
|
-
# Image url will be absolute thanks to the decorator.
|
183
|
+
# Image url will be absolute and encoded correctly thanks to the decorator.
|
187
184
|
noko.at_css('.profile-picture/@src').text
|
188
185
|
end
|
189
186
|
end
|
data/lib/scraped/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: scraped
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- EveryPolitician
|
@@ -148,7 +148,7 @@ files:
|
|
148
148
|
- lib/scraped/request/strategy/live_request.rb
|
149
149
|
- lib/scraped/response.rb
|
150
150
|
- lib/scraped/response/decorator.rb
|
151
|
-
- lib/scraped/response/decorator/
|
151
|
+
- lib/scraped/response/decorator/clean_urls.rb
|
152
152
|
- lib/scraped/response_decorator.rb
|
153
153
|
- lib/scraped/version.rb
|
154
154
|
- scraped.gemspec
|