ar_book_finder 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +20 -0
- data/.travis.yml +11 -0
- data/Gemfile +15 -0
- data/LICENSE.txt +22 -0
- data/README.md +117 -0
- data/Rakefile +10 -0
- data/ar_book_finder.gemspec +27 -0
- data/lib/ar_book_finder.rb +38 -0
- data/lib/ar_book_finder/book.rb +50 -0
- data/lib/ar_book_finder/book_detail_parser.rb +66 -0
- data/lib/ar_book_finder/book_detail_processor.rb +13 -0
- data/lib/ar_book_finder/configuration.rb +29 -0
- data/lib/ar_book_finder/pagination_processor.rb +27 -0
- data/lib/ar_book_finder/publisher.rb +17 -0
- data/lib/ar_book_finder/quick_search_processor.rb +19 -0
- data/lib/ar_book_finder/scraper.rb +13 -0
- data/lib/ar_book_finder/search_results_parser.rb +38 -0
- data/lib/ar_book_finder/user_type_processor.rb +22 -0
- data/lib/ar_book_finder/version.rb +3 -0
- data/spec/.keep +0 -0
- data/spec/ar_book_finder/book_spec.rb +10 -0
- data/spec/ar_book_finder_spec.rb +15 -0
- data/spec/spec_helper.rb +18 -0
- metadata +153 -0
data/.gitignore
ADDED
data/.travis.yml
ADDED
data/Gemfile
ADDED
@@ -0,0 +1,15 @@
|
|
1
|
+
source 'https://rubygems.org'
|
2
|
+
|
3
|
+
# Specify your gem's dependencies in ar_book_finder.gemspec
|
4
|
+
gemspec
|
5
|
+
|
6
|
+
group :development do
|
7
|
+
gem 'bundler'
|
8
|
+
gem 'coveralls', require: false
|
9
|
+
gem 'rake'
|
10
|
+
end
|
11
|
+
|
12
|
+
group :test do
|
13
|
+
gem 'rspec'
|
14
|
+
gem 'simplecov'
|
15
|
+
end
|
data/LICENSE.txt
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2013 Anthony Smith
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,117 @@
|
|
1
|
+
# ARBookFinder
|
2
|
+
|
3
|
+
Retrieve book data from [arbookfind.com](http://www.arbookfind.com)
|
4
|
+
|
5
|
+
[![Build Status](https://travis-ci.org/anthonator/ar-book-finder.png?branch=master)](https://travis-ci.org/anthonator/ar-book-finder) [![Dependency Status](https://gemnasium.com/anthonator/ar-book-finder.png)](https://gemnasium.com/anthonator/ar-book-finder) [![Coverage Status](https://coveralls.io/repos/anthonator/ar-book-finder/badge.png)](https://coveralls.io/r/anthonator/ar-book-finder) [![Code Climate](https://codeclimate.com/github/anthonator/ar-book-finder.png)](https://codeclimate.com/github/anthonator/ar-book-finder)
|
6
|
+
|
7
|
+
## Installation
|
8
|
+
|
9
|
+
Add this line to your application's Gemfile:
|
10
|
+
|
11
|
+
gem 'ar_book_finder'
|
12
|
+
|
13
|
+
And then execute:
|
14
|
+
|
15
|
+
$ bundle
|
16
|
+
|
17
|
+
Or install it yourself as:
|
18
|
+
|
19
|
+
$ gem install ar_book_finder
|
20
|
+
|
21
|
+
### PhantomJS
|
22
|
+
This gem utilizes [PhantomJS](http://phantomjs.org/) for interacting with [arbookfind.com](http://www.arbookfind.com). Make sure to [install PhantomJS](http://phantomjs.org/download.html) in order to use this gem.
|
23
|
+
|
24
|
+
## Configuration
|
25
|
+
Choose what type of user to peform searches as.
|
26
|
+
|
27
|
+
Available types include:
|
28
|
+
* ```:student```
|
29
|
+
* ```:parent```
|
30
|
+
* ```:teacher```
|
31
|
+
* ```:librarian```
|
32
|
+
|
33
|
+
These types match up to the available user types [here](http://www.arbookfind.com/usertype.aspx).
|
34
|
+
```ruby
|
35
|
+
ARBookFinder.configure do |config|
|
36
|
+
config.user_type = :teacher
|
37
|
+
end
|
38
|
+
```
|
39
|
+
|
40
|
+
## What data is returned?
|
41
|
+
The ARBookFinder gem is capable or returning any data located on a book's [detail](http://www.arbookfind.com/bookdetail.aspx?q=26759&l=EN&slid=431859594) page.
|
42
|
+
|
43
|
+
This includes:
|
44
|
+
* cover
|
45
|
+
* title
|
46
|
+
* author
|
47
|
+
* summary
|
48
|
+
* ar_quiz_number
|
49
|
+
* language
|
50
|
+
* ar_quiz_availability
|
51
|
+
* atos_book_level
|
52
|
+
* interest_level
|
53
|
+
* ar_points
|
54
|
+
* rating
|
55
|
+
* word_count
|
56
|
+
* type
|
57
|
+
* topics
|
58
|
+
* series
|
59
|
+
* url
|
60
|
+
|
61
|
+
And publisher details:
|
62
|
+
* name
|
63
|
+
* lccn
|
64
|
+
* isbn
|
65
|
+
* year_published
|
66
|
+
* page_count
|
67
|
+
|
68
|
+
## Usage
|
69
|
+
|
70
|
+
### Quick Search
|
71
|
+
Quick search allows you to perform searches based on title, topics, author or ISBN.
|
72
|
+
|
73
|
+
Perform a quick search...
|
74
|
+
```ruby
|
75
|
+
results = ARBookFinder.search('harry potter')
|
76
|
+
results.page_count # Retrieve the number of pages this search returns
|
77
|
+
results.books # Retrieve the books returned on this page
|
78
|
+
```
|
79
|
+
Load book data on demand using ```#fetch```...
|
80
|
+
``` ruby
|
81
|
+
...
|
82
|
+
book = results[0]
|
83
|
+
book.fetch # Retrieve book data
|
84
|
+
book.title
|
85
|
+
book.author
|
86
|
+
...
|
87
|
+
```
|
88
|
+
Return publisher data...
|
89
|
+
```ruby
|
90
|
+
...
|
91
|
+
publisher = book.publishers
|
92
|
+
publisher.name
|
93
|
+
publisher.isbn
|
94
|
+
...
|
95
|
+
```
|
96
|
+
|
97
|
+
### Pagination and Sorting
|
98
|
+
It's also possible to paginate and sort search results.
|
99
|
+
```ruby
|
100
|
+
# Retrieve results for page 2 and sort on title
|
101
|
+
results = ARBookFinder.Search('harry potter', 2, :title)
|
102
|
+
```
|
103
|
+
|
104
|
+
## Contributing
|
105
|
+
|
106
|
+
1. Fork it
|
107
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
108
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
109
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
110
|
+
5. Create new Pull Request
|
111
|
+
|
112
|
+
## Credits
|
113
|
+
[![Sticksnleaves](http://sticksnleaves-wordpress.herokuapp.com/wp-content/themes/sticksnleaves/images/snl-logo-116x116.png)](http://www.sticksnleaves.com)
|
114
|
+
|
115
|
+
ARBookFinder is maintained and funded by [Sticksnleaves](http://www.sticksnleaves.com)
|
116
|
+
|
117
|
+
Thanks to all of our [contributors](https://github.com/anthonator/ar-book-finder/graphs/contributors)
|
data/Rakefile
ADDED
@@ -0,0 +1,27 @@
|
|
1
|
+
# coding: utf-8
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require 'ar_book_finder/version'
|
5
|
+
|
6
|
+
Gem::Specification.new do |spec|
|
7
|
+
spec.name = "ar_book_finder"
|
8
|
+
spec.version = ARBookFinder::VERSION
|
9
|
+
spec.authors = ["Anthony Smith"]
|
10
|
+
spec.email = ["anthony@sticksnleaves.com"]
|
11
|
+
spec.description = %q{Access book data on ARBookFinder}
|
12
|
+
spec.summary = %q{Crawls and parses data on arbookfind.com and returns book data in an easy to use format}
|
13
|
+
spec.homepage = "https://github.com/anthonator/ar-book-finder"
|
14
|
+
spec.license = "MIT"
|
15
|
+
|
16
|
+
spec.files = `git ls-files`.split($/)
|
17
|
+
spec.executables = spec.files.grep(%r{^bin/}) { |f| File.basename(f) }
|
18
|
+
spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
|
19
|
+
spec.require_paths = ["lib"]
|
20
|
+
|
21
|
+
spec.add_runtime_dependency 'capybara'
|
22
|
+
spec.add_runtime_dependency 'poltergeist'
|
23
|
+
spec.add_runtime_dependency 'nokogiri'
|
24
|
+
|
25
|
+
spec.add_development_dependency 'bundler', '~> 1.3'
|
26
|
+
spec.add_development_dependency 'rake'
|
27
|
+
end
|
@@ -0,0 +1,38 @@
|
|
1
|
+
require 'capybara'
|
2
|
+
require 'capybara/poltergeist'
|
3
|
+
require 'nokogiri'
|
4
|
+
|
5
|
+
require 'ar_book_finder/user_type_processor'
|
6
|
+
require 'ar_book_finder/pagination_processor'
|
7
|
+
require 'ar_book_finder/quick_search_processor'
|
8
|
+
require 'ar_book_finder/book_detail_processor'
|
9
|
+
require 'ar_book_finder/search_results_parser'
|
10
|
+
require 'ar_book_finder/book_detail_parser'
|
11
|
+
require 'ar_book_finder/configuration'
|
12
|
+
require 'ar_book_finder/scraper'
|
13
|
+
require 'ar_book_finder/book'
|
14
|
+
require 'ar_book_finder/publisher'
|
15
|
+
require 'ar_book_finder/version'
|
16
|
+
|
17
|
+
Capybara.current_driver = :poltergeist
|
18
|
+
Capybara.javascript_driver = :poltergeist
|
19
|
+
|
20
|
+
module ARBookFinder
|
21
|
+
extend Configuration
|
22
|
+
|
23
|
+
def self.scraper(user_type)
|
24
|
+
Scraper.new(user_type)
|
25
|
+
end
|
26
|
+
|
27
|
+
def self.search(query, page = 1, sort_by = 'Relevance')
|
28
|
+
scraper(options[:user_type]).search(query, page, sort_by)
|
29
|
+
end
|
30
|
+
|
31
|
+
def self.advanced_search(user_type, search_type, params)
|
32
|
+
raise 'Not yet implemented'
|
33
|
+
end
|
34
|
+
|
35
|
+
def self.collection(topic, sub_topic)
|
36
|
+
raise 'Not yet implemented'
|
37
|
+
end
|
38
|
+
end
|
@@ -0,0 +1,50 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
class Book
|
3
|
+
attr_reader :cover, :title, :author, :summary,
|
4
|
+
:ar_quiz_number, :language, :ar_quiz_availability, :atos_book_level,
|
5
|
+
:interest_level, :ar_points, :rating, :word_count,
|
6
|
+
:type, :topics, :series, :url,
|
7
|
+
:publishers
|
8
|
+
|
9
|
+
def initialize(url)
|
10
|
+
@url = url
|
11
|
+
@publishers = []
|
12
|
+
end
|
13
|
+
|
14
|
+
def fetch
|
15
|
+
BookDetailProcessor.new(@url).process
|
16
|
+
load(BookDetailParser.new(Capybara.page.html).parse)
|
17
|
+
self
|
18
|
+
end
|
19
|
+
|
20
|
+
private
|
21
|
+
def load(parsed_data)
|
22
|
+
parsed_data.keys.each do |k|
|
23
|
+
instance_variable_set("@#{k}", parsed_data[k]) unless [:ar_quiz_availability,
|
24
|
+
:topics,
|
25
|
+
:series,
|
26
|
+
:publishers].include?(k)
|
27
|
+
end
|
28
|
+
|
29
|
+
load_ar_quiz_availability(parsed_data[:ar_quiz_availability])
|
30
|
+
load_topics(parsed_data[:topics])
|
31
|
+
load_series(parsed_data[:series])
|
32
|
+
|
33
|
+
parsed_data[:publishers].each { |p| @publishers << Publisher.new(p) }
|
34
|
+
end
|
35
|
+
|
36
|
+
def load_ar_quiz_availability(ar_quiz_availability)
|
37
|
+
@ar_quiz_availability = ar_quiz_availability.split(',').collect { |v| v.strip }
|
38
|
+
end
|
39
|
+
|
40
|
+
def load_topics(topics)
|
41
|
+
@topics = topics.split(';').collect do |topic|
|
42
|
+
topic.split('-').collect { |v| v.strip }
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
def load_series(series)
|
47
|
+
@series = series.split(';').collect { |v| v.strip }
|
48
|
+
end
|
49
|
+
end
|
50
|
+
end
|
@@ -0,0 +1,66 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
class BookDetailParser
|
3
|
+
ROOT_XPATH = '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_detailTable"]'
|
4
|
+
BOOK_XPATH_NODES = {
|
5
|
+
cover: '//img[@id="ctl00_ContentPlaceHolder1_ucBookDetail_imgBookCover"]/@src',
|
6
|
+
title: 'tbody/tr/td[3]/strong',
|
7
|
+
author: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblAuthor"]',
|
8
|
+
summary: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblBookSummary"]',
|
9
|
+
ar_quiz_number: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblQuizNumber"]',
|
10
|
+
language: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblLanguageCode"]',
|
11
|
+
ar_quiz_availability: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblQuizStatusLabel"]',
|
12
|
+
atos_book_level: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblBookLevel"]',
|
13
|
+
interest_level: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblInterestLevel"]',
|
14
|
+
ar_points: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblPoints"]',
|
15
|
+
rating: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblRanking"]/img/@title',
|
16
|
+
word_count: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblWordCount"]',
|
17
|
+
type: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblFictionNonFiction"]',
|
18
|
+
topics: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblTopicLabel"]',
|
19
|
+
series: '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_lblSeriesLabel"]'
|
20
|
+
}
|
21
|
+
|
22
|
+
ROOT_PUBLISHER_XPATH = '//*[@id="ctl00_ContentPlaceHolder1_ucBookDetail_tblPublisherTable"]/tbody/tr'
|
23
|
+
PUBLISHER_XPATH_NODES = {
|
24
|
+
name: 'td[1]',
|
25
|
+
lccn: 'td[2]',
|
26
|
+
isbn: 'td[3]',
|
27
|
+
year_published: 'td[4]',
|
28
|
+
page_count: 'td[5]'
|
29
|
+
}
|
30
|
+
|
31
|
+
|
32
|
+
def initialize(html)
|
33
|
+
@doc = Nokogiri::HTML.parse(html)
|
34
|
+
@root = @doc.xpath(ROOT_XPATH)
|
35
|
+
end
|
36
|
+
|
37
|
+
def parse
|
38
|
+
hash = parse_book_nodes
|
39
|
+
hash[:publishers] = parse_publisher_nodes
|
40
|
+
hash
|
41
|
+
end
|
42
|
+
|
43
|
+
private
|
44
|
+
def parse_book_nodes
|
45
|
+
hash = {}
|
46
|
+
BOOK_XPATH_NODES.keys.each { |k| hash[k] = @root.xpath(BOOK_XPATH_NODES[k]).text.strip }
|
47
|
+
hash
|
48
|
+
end
|
49
|
+
|
50
|
+
def parse_publisher_nodes
|
51
|
+
root = @root.xpath(ROOT_PUBLISHER_XPATH)
|
52
|
+
publishers = []
|
53
|
+
root.each_with_index do |node, i|
|
54
|
+
next if i == 0
|
55
|
+
hash = {}
|
56
|
+
PUBLISHER_XPATH_NODES.keys.each do |key|
|
57
|
+
value = node.xpath(PUBLISHER_XPATH_NODES[key]).text
|
58
|
+
value = '' if value == 'Not Available'
|
59
|
+
hash[key] = value.strip
|
60
|
+
end
|
61
|
+
publishers << hash
|
62
|
+
end
|
63
|
+
publishers
|
64
|
+
end
|
65
|
+
end
|
66
|
+
end
|
@@ -0,0 +1,29 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
module Configuration
|
3
|
+
VALID_OPTION_KEYS = [
|
4
|
+
:user_type
|
5
|
+
]
|
6
|
+
|
7
|
+
DEFAULT_USER_TYPE = nil.freeze
|
8
|
+
|
9
|
+
attr_accessor(*VALID_OPTION_KEYS)
|
10
|
+
|
11
|
+
def self.extended(base)
|
12
|
+
base.reset
|
13
|
+
end
|
14
|
+
|
15
|
+
def configure
|
16
|
+
yield self
|
17
|
+
end
|
18
|
+
|
19
|
+
def options
|
20
|
+
VALID_OPTION_KEYS.inject({}) do |option, key|
|
21
|
+
option.merge!(key => send(key))
|
22
|
+
end
|
23
|
+
end
|
24
|
+
|
25
|
+
def reset
|
26
|
+
self.user_type = DEFAULT_USER_TYPE
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
@@ -0,0 +1,27 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
class PaginationProcessor
|
3
|
+
include Capybara::DSL
|
4
|
+
|
5
|
+
DEFAULT_SORT_BY = 'Relevance'
|
6
|
+
|
7
|
+
SORT_BY_TYPES = {
|
8
|
+
title: 'Title',
|
9
|
+
author: 'Author',
|
10
|
+
interest_level: 'Interest Level',
|
11
|
+
book_level: 'Book Level',
|
12
|
+
relevance: 'Relevance',
|
13
|
+
rating: 'Rating'
|
14
|
+
}
|
15
|
+
|
16
|
+
def initialize(page, sort_by)
|
17
|
+
@page = page
|
18
|
+
@sort_by = SORT_BY_TYPES[sort_by]
|
19
|
+
end
|
20
|
+
|
21
|
+
def process
|
22
|
+
fill_in('ctl00_ContentPlaceHolder1_ucSeachResults_txtPageToGoToTop', with: @page)
|
23
|
+
select(@sort_by, from: 'ctl00_ContentPlaceHolder1_ucSeachResults_lstSortOptionListTop')
|
24
|
+
click_button('ctl00_ContentPlaceHolder1_ucSeachResults_btnGoToPageTop')
|
25
|
+
end
|
26
|
+
end
|
27
|
+
end
|
@@ -0,0 +1,17 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
class Publisher
|
3
|
+
attr_reader :name, :lccn, :isbn, :year_published,
|
4
|
+
:page_count
|
5
|
+
|
6
|
+
def initialize(parsed_data)
|
7
|
+
load(parsed_data)
|
8
|
+
end
|
9
|
+
|
10
|
+
private
|
11
|
+
def load(parsed_data)
|
12
|
+
parsed_data.keys.each do |k|
|
13
|
+
instance_variable_set("@#{k}", parsed_data[k])
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
@@ -0,0 +1,19 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
class QuickSearchProcessor
|
3
|
+
include Capybara::DSL
|
4
|
+
|
5
|
+
def initialize(query, page, sort_by)
|
6
|
+
@query = query
|
7
|
+
@page = page
|
8
|
+
@sort_by = sort_by
|
9
|
+
end
|
10
|
+
|
11
|
+
def process
|
12
|
+
fill_in('ctl00_ContentPlaceHolder1_txtKeyWords', with: @query)
|
13
|
+
click_button('ctl00_ContentPlaceHolder1_btnDoIt')
|
14
|
+
if @page > 1 || @sort_by != PaginationProcessor::DEFAULT_SORT_BY
|
15
|
+
PaginationProcessor.new(@page, @sort_by).process
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
19
|
+
end
|
@@ -0,0 +1,13 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
class Scraper
|
3
|
+
def initialize(user_type)
|
4
|
+
@user_type = user_type
|
5
|
+
end
|
6
|
+
|
7
|
+
def search(query, page = 1, sort_by = 'Relevance')
|
8
|
+
UserTypeProcessor.new(@user_type).process
|
9
|
+
QuickSearchProcessor.new(query, page, sort_by).process
|
10
|
+
SearchResultsParser.new(Capybara.page.html).parse
|
11
|
+
end
|
12
|
+
end
|
13
|
+
end
|
@@ -0,0 +1,38 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
class SearchResultsParser
|
3
|
+
PAGE_COUNT_XPATH = '//*[@id="ctl00_ContentPlaceHolder1_ucSeachResults_lblResultsSummaryTop"]'
|
4
|
+
RESULTS_XPATH = '//*[@id="ctl00_ContentPlaceHolder1_ucSeachResults_lblQuizzes"]/table'
|
5
|
+
BOOK_XPATH = 'tbody/tr/td[2]'
|
6
|
+
BOOK_DETAIL_XPATH = 'table/tbody/tr/td[2]'
|
7
|
+
BOOK_URL_XPATH = 'a'
|
8
|
+
|
9
|
+
attr_reader :page_count, :books
|
10
|
+
|
11
|
+
def initialize(html)
|
12
|
+
@doc = Nokogiri::HTML.parse(html)
|
13
|
+
@books = []
|
14
|
+
end
|
15
|
+
|
16
|
+
def parse
|
17
|
+
@page_count = parse_page_count
|
18
|
+
@books = parse_results
|
19
|
+
self
|
20
|
+
end
|
21
|
+
|
22
|
+
private
|
23
|
+
def parse_page_count
|
24
|
+
@doc.xpath(PAGE_COUNT_XPATH).text.gsub(/Page \d+ of /, '')
|
25
|
+
end
|
26
|
+
|
27
|
+
def parse_results
|
28
|
+
books = []
|
29
|
+
@doc.xpath(RESULTS_XPATH).each_with_index do |result, i|
|
30
|
+
next if i.odd?
|
31
|
+
book = result.xpath(BOOK_XPATH)
|
32
|
+
book_detail = book.xpath(BOOK_DETAIL_XPATH)
|
33
|
+
books << Book.new('http://www.arbookfind.com/' + book_detail.xpath(BOOK_URL_XPATH).attribute('href').content)
|
34
|
+
end
|
35
|
+
books
|
36
|
+
end
|
37
|
+
end
|
38
|
+
end
|
@@ -0,0 +1,22 @@
|
|
1
|
+
module ARBookFinder
|
2
|
+
class UserTypeProcessor
|
3
|
+
include Capybara::DSL
|
4
|
+
|
5
|
+
USER_TYPES = {
|
6
|
+
student: 'Student',
|
7
|
+
parent: 'Parent',
|
8
|
+
teacher: 'Teacher',
|
9
|
+
librarian: 'Librarian'
|
10
|
+
}
|
11
|
+
|
12
|
+
def initialize(user_type)
|
13
|
+
@user_type = USER_TYPES[user_type]
|
14
|
+
end
|
15
|
+
|
16
|
+
def process
|
17
|
+
visit('http://www.arbookfind.com/usertype.aspx')
|
18
|
+
choose(@user_type)
|
19
|
+
click_button('Submit')
|
20
|
+
end
|
21
|
+
end
|
22
|
+
end
|
data/spec/.keep
ADDED
File without changes
|
@@ -0,0 +1,15 @@
|
|
1
|
+
require 'spec_helper'
|
2
|
+
|
3
|
+
describe ARBookFinder do
|
4
|
+
describe 'search' do
|
5
|
+
it 'should return results' do
|
6
|
+
results = ARBookFinder.search('978-0590353403').books
|
7
|
+
results.size.should > 0
|
8
|
+
end
|
9
|
+
|
10
|
+
it 'should paginate and sort results' do
|
11
|
+
results = ARBookFinder.search('harry potter', 2, :title).books
|
12
|
+
results.size.should > 0
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
data/spec/spec_helper.rb
ADDED
@@ -0,0 +1,18 @@
|
|
1
|
+
# Require supporting ruby files with custom matchers and macros, etc,
|
2
|
+
# in spec/support/ and its subdirectories.
|
3
|
+
Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each { |f| require f }
|
4
|
+
|
5
|
+
require 'simplecov'
|
6
|
+
require 'coveralls'
|
7
|
+
SimpleCov.formatter = Coveralls::SimpleCov::Formatter
|
8
|
+
SimpleCov.start
|
9
|
+
|
10
|
+
require 'ar_book_finder'
|
11
|
+
|
12
|
+
RSpec.configure do |config|
|
13
|
+
config.before(:each) do
|
14
|
+
ARBookFinder.configure do |config|
|
15
|
+
config.user_type = :teacher
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
metadata
ADDED
@@ -0,0 +1,153 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: ar_book_finder
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
prerelease:
|
5
|
+
version: 1.0.0
|
6
|
+
platform: ruby
|
7
|
+
authors:
|
8
|
+
- Anthony Smith
|
9
|
+
autorequire:
|
10
|
+
bindir: bin
|
11
|
+
cert_chain: []
|
12
|
+
date: 2013-10-26 00:00:00.000000000 Z
|
13
|
+
dependencies:
|
14
|
+
- !ruby/object:Gem::Dependency
|
15
|
+
name: capybara
|
16
|
+
version_requirements: !ruby/object:Gem::Requirement
|
17
|
+
requirements:
|
18
|
+
- - '>='
|
19
|
+
- !ruby/object:Gem::Version
|
20
|
+
version: '0'
|
21
|
+
none: false
|
22
|
+
requirement: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - '>='
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: '0'
|
27
|
+
none: false
|
28
|
+
prerelease: false
|
29
|
+
type: :runtime
|
30
|
+
- !ruby/object:Gem::Dependency
|
31
|
+
name: poltergeist
|
32
|
+
version_requirements: !ruby/object:Gem::Requirement
|
33
|
+
requirements:
|
34
|
+
- - '>='
|
35
|
+
- !ruby/object:Gem::Version
|
36
|
+
version: '0'
|
37
|
+
none: false
|
38
|
+
requirement: !ruby/object:Gem::Requirement
|
39
|
+
requirements:
|
40
|
+
- - '>='
|
41
|
+
- !ruby/object:Gem::Version
|
42
|
+
version: '0'
|
43
|
+
none: false
|
44
|
+
prerelease: false
|
45
|
+
type: :runtime
|
46
|
+
- !ruby/object:Gem::Dependency
|
47
|
+
name: nokogiri
|
48
|
+
version_requirements: !ruby/object:Gem::Requirement
|
49
|
+
requirements:
|
50
|
+
- - '>='
|
51
|
+
- !ruby/object:Gem::Version
|
52
|
+
version: '0'
|
53
|
+
none: false
|
54
|
+
requirement: !ruby/object:Gem::Requirement
|
55
|
+
requirements:
|
56
|
+
- - '>='
|
57
|
+
- !ruby/object:Gem::Version
|
58
|
+
version: '0'
|
59
|
+
none: false
|
60
|
+
prerelease: false
|
61
|
+
type: :runtime
|
62
|
+
- !ruby/object:Gem::Dependency
|
63
|
+
name: bundler
|
64
|
+
version_requirements: !ruby/object:Gem::Requirement
|
65
|
+
requirements:
|
66
|
+
- - ~>
|
67
|
+
- !ruby/object:Gem::Version
|
68
|
+
version: '1.3'
|
69
|
+
none: false
|
70
|
+
requirement: !ruby/object:Gem::Requirement
|
71
|
+
requirements:
|
72
|
+
- - ~>
|
73
|
+
- !ruby/object:Gem::Version
|
74
|
+
version: '1.3'
|
75
|
+
none: false
|
76
|
+
prerelease: false
|
77
|
+
type: :development
|
78
|
+
- !ruby/object:Gem::Dependency
|
79
|
+
name: rake
|
80
|
+
version_requirements: !ruby/object:Gem::Requirement
|
81
|
+
requirements:
|
82
|
+
- - '>='
|
83
|
+
- !ruby/object:Gem::Version
|
84
|
+
version: '0'
|
85
|
+
none: false
|
86
|
+
requirement: !ruby/object:Gem::Requirement
|
87
|
+
requirements:
|
88
|
+
- - '>='
|
89
|
+
- !ruby/object:Gem::Version
|
90
|
+
version: '0'
|
91
|
+
none: false
|
92
|
+
prerelease: false
|
93
|
+
type: :development
|
94
|
+
description: Access book data on ARBookFinder
|
95
|
+
email:
|
96
|
+
- anthony@sticksnleaves.com
|
97
|
+
executables: []
|
98
|
+
extensions: []
|
99
|
+
extra_rdoc_files: []
|
100
|
+
files:
|
101
|
+
- .gitignore
|
102
|
+
- .travis.yml
|
103
|
+
- Gemfile
|
104
|
+
- LICENSE.txt
|
105
|
+
- README.md
|
106
|
+
- Rakefile
|
107
|
+
- ar_book_finder.gemspec
|
108
|
+
- lib/ar_book_finder.rb
|
109
|
+
- lib/ar_book_finder/book.rb
|
110
|
+
- lib/ar_book_finder/book_detail_parser.rb
|
111
|
+
- lib/ar_book_finder/book_detail_processor.rb
|
112
|
+
- lib/ar_book_finder/configuration.rb
|
113
|
+
- lib/ar_book_finder/pagination_processor.rb
|
114
|
+
- lib/ar_book_finder/publisher.rb
|
115
|
+
- lib/ar_book_finder/quick_search_processor.rb
|
116
|
+
- lib/ar_book_finder/scraper.rb
|
117
|
+
- lib/ar_book_finder/search_results_parser.rb
|
118
|
+
- lib/ar_book_finder/user_type_processor.rb
|
119
|
+
- lib/ar_book_finder/version.rb
|
120
|
+
- spec/.keep
|
121
|
+
- spec/ar_book_finder/book_spec.rb
|
122
|
+
- spec/ar_book_finder_spec.rb
|
123
|
+
- spec/spec_helper.rb
|
124
|
+
homepage: https://github.com/anthonator/ar-book-finder
|
125
|
+
licenses:
|
126
|
+
- MIT
|
127
|
+
post_install_message:
|
128
|
+
rdoc_options: []
|
129
|
+
require_paths:
|
130
|
+
- lib
|
131
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
132
|
+
requirements:
|
133
|
+
- - '>='
|
134
|
+
- !ruby/object:Gem::Version
|
135
|
+
version: '0'
|
136
|
+
none: false
|
137
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
138
|
+
requirements:
|
139
|
+
- - '>='
|
140
|
+
- !ruby/object:Gem::Version
|
141
|
+
version: '0'
|
142
|
+
none: false
|
143
|
+
requirements: []
|
144
|
+
rubyforge_project:
|
145
|
+
rubygems_version: 1.8.24
|
146
|
+
signing_key:
|
147
|
+
specification_version: 3
|
148
|
+
summary: Crawls and parses data on arbookfind.com and returns book data in an easy to use format
|
149
|
+
test_files:
|
150
|
+
- spec/.keep
|
151
|
+
- spec/ar_book_finder/book_spec.rb
|
152
|
+
- spec/ar_book_finder_spec.rb
|
153
|
+
- spec/spec_helper.rb
|