hostelify 0.6.2 → 0.6.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.markdown +12 -1
- data/lib/hostelify/hostelworld.rb +1 -1
- data/lib/hostelify/version.rb +1 -1
- data/lib/hostelify.rb +1 -1
- metadata +4 -4
data/README.markdown
CHANGED
|
@@ -6,4 +6,15 @@
|
|
|
6
6
|
* Hostelbookers (back soon, use < 0.6)
|
|
7
7
|
* Gomio (removed too slow to speak of)
|
|
8
8
|
|
|
9
|
-
Web scrapping utility for publicly accessible data such as dates and prices.
|
|
9
|
+
Web scrapping utility for publicly accessible data such as dates and prices.
|
|
10
|
+
|
|
11
|
+
# Well-behaved bots Policy
|
|
12
|
+
|
|
13
|
+
This gem strictly follows the Well-behaved bots Policy, the standard policy of Google, Yahoo & many other search engines.
|
|
14
|
+
If a site owner wishes to give instructions to web robots, including this one, he/she must place a text file called robots.txt in the root of the web site hierarchy (e.g. www.example.com/robots.txt).
|
|
15
|
+
This text file should contain the instructions in a specific format (see examples below).
|
|
16
|
+
Robots that choose to follow the instructions try to fetch this file and read the instructions before fetching any other file from the web site.
|
|
17
|
+
If this file doesn’t exist web robots assume that the web owner wishes to provide no specific instructions.
|
|
18
|
+
|
|
19
|
+
>User-agent: *
|
|
20
|
+
>Disallow: /
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
class
|
|
1
|
+
class Hostelworld < Nibbler
|
|
2
2
|
element 'h1' => :name, :with => lambda { |node| node.inner_text.lstrip.rstrip }
|
|
3
3
|
element '.street-city' => :address, :with => lambda { |node| node.inner_text.gsub(/\s{6,}.*/,'').chop.lstrip }
|
|
4
4
|
element 'div.row //p[5]' => :content
|
data/lib/hostelify/version.rb
CHANGED
data/lib/hostelify.rb
CHANGED
metadata
CHANGED
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: hostelify
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
hash:
|
|
4
|
+
hash: 1
|
|
5
5
|
prerelease: false
|
|
6
6
|
segments:
|
|
7
7
|
- 0
|
|
8
8
|
- 6
|
|
9
|
-
-
|
|
10
|
-
version: 0.6.
|
|
9
|
+
- 3
|
|
10
|
+
version: 0.6.3
|
|
11
11
|
platform: ruby
|
|
12
12
|
authors:
|
|
13
13
|
- Holden Thomas
|
|
@@ -15,7 +15,7 @@ autorequire:
|
|
|
15
15
|
bindir: bin
|
|
16
16
|
cert_chain: []
|
|
17
17
|
|
|
18
|
-
date: 2011-04-
|
|
18
|
+
date: 2011-04-07 00:00:00 +02:00
|
|
19
19
|
default_executable:
|
|
20
20
|
dependencies:
|
|
21
21
|
- !ruby/object:Gem::Dependency
|