bootleg 0.0.6 → 0.0.7

Sign up to get free protection for your applications and to get access to all the features.
Files changed (39) hide show
  1. checksums.yaml +7 -0
  2. data/.gitignore +1 -0
  3. data/Gemfile +0 -6
  4. data/README.md +43 -53
  5. data/bootleg.gemspec +6 -7
  6. data/lib/bootleg.rb +5 -7
  7. data/lib/bootleg/agent.rb +31 -0
  8. data/lib/bootleg/movie.rb +30 -0
  9. data/lib/bootleg/page.rb +26 -0
  10. data/lib/bootleg/theater.rb +54 -0
  11. data/lib/bootleg/version.rb +1 -1
  12. data/spec/lib/agent_spec.rb +11 -0
  13. data/spec/lib/movie_spec.rb +37 -0
  14. data/spec/lib/page_spec.rb +21 -0
  15. data/spec/lib/theater_spec.rb +52 -0
  16. metadata +31 -83
  17. data/README.rdoc +0 -3
  18. data/lib/extractor.rb +0 -32
  19. data/lib/finder.rb +0 -14
  20. data/lib/generators/bootleg/USAGE +0 -0
  21. data/lib/generators/bootleg/install_generator.rb +0 -15
  22. data/lib/generators/bootleg/movie_generator.rb +0 -24
  23. data/lib/generators/bootleg/showtime_generator.rb +0 -24
  24. data/lib/generators/bootleg/templates/movie_migration.rb +0 -10
  25. data/lib/generators/bootleg/templates/movie_model.rb +0 -10
  26. data/lib/generators/bootleg/templates/showtime_migration.rb +0 -12
  27. data/lib/generators/bootleg/templates/showtime_model.rb +0 -14
  28. data/lib/generators/bootleg/templates/theater_migration.rb +0 -10
  29. data/lib/generators/bootleg/templates/theater_model.rb +0 -10
  30. data/lib/generators/bootleg/theater_generator.rb +0 -24
  31. data/lib/manager.rb +0 -27
  32. data/lib/modules/href.rb +0 -18
  33. data/lib/modules/movie.rb +0 -24
  34. data/lib/modules/theater.rb +0 -45
  35. data/lib/modules/zipcode.rb +0 -12
  36. data/lib/presenter.rb +0 -8
  37. data/spec/extractor_spec.rb +0 -59
  38. data/spec/finder_spec.rb +0 -26
  39. data/spec/presenter_spec.rb +0 -4
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: dfffbc8d963307c1e4023281762f735b18b0b117
4
+ data.tar.gz: 0c0af543cf5c8b8c3da8fa0b2c00d5babcab9762
5
+ SHA512:
6
+ metadata.gz: 2f20a7f0caf9fdaa96fccc43dd38f85a5e8b9a29740c8aef03bfacc5e5da2b7a9ab264ca252272e567e47d4b4395a9076acb17974d9feec5b4848324e0413d3d
7
+ data.tar.gz: 74025fc09a0c95f8e7bc9900a4bdb4f8c20c5890c78cc2d6767dfa82e34ca2b64d483abf4a08fa67f08792666df628a1722c294ca254eec402524319a86abc74
data/.gitignore CHANGED
@@ -2,6 +2,7 @@
2
2
  *.rbc
3
3
  .bundle
4
4
  .config
5
+ .rspec
5
6
  .yardoc
6
7
  Gemfile.lock
7
8
  InstalledFiles
data/Gemfile CHANGED
@@ -1,10 +1,4 @@
1
1
  source 'https://rubygems.org'
2
2
 
3
- gem 'nokogiri'
4
- gem 'mechanize'
5
- gem 'rspec'
6
- gem 'activerecord'
7
- gem 'activerecord-nulldb-adapter'
8
3
  # Specify your gem's dependencies in bootleg.gemspec
9
-
10
4
  gemspec
data/README.md CHANGED
@@ -1,71 +1,61 @@
1
1
  # Bootleg
2
2
 
3
- Bootleg is a basic scraper. It scrapes movie, theater and showtime
4
- details from moviefone.com based on a certain zipcode for a 25 mile
5
- radius.
3
+ Bootleg represents my playground for scraping movies from
4
+ [moviefone.com](http://moviefone.com)
6
5
 
7
- ## Installation
6
+ ## If you want you can play with it as well
8
7
 
9
- Add this line to your application's Gemfile:
8
+ To gain access to the gem simply:
10
9
 
11
- gem 'bootleg'
10
+ ```
11
+ require 'bootleg'
12
+ ```
12
13
 
13
- And then execute:
14
+ Currently Bootleg supports search for movies and theaters within a certain
15
+ zipcode.
14
16
 
15
- $ bundle
17
+ ```
18
+ agent = Bootleg::Agent.new(zipcode: 20851)
19
+ ```
16
20
 
17
- Or install it yourself as:
21
+ The bootleg agent is responsible for initializing the enviornment and providing
22
+ a simple method ``page`` that gives us access to the first search results.
18
23
 
19
- $ gem install bootleg
24
+ ```
25
+ page = agent.page
26
+ ```
20
27
 
21
- Generate models and migrations:
28
+ A page has two handy methods. The ``theaters`` method gives you an array with
29
+ all the available theaters. You have access to a theater's title, link, price
30
+ for an adult or the price for a child as well as the address of the theater and
31
+ the movies that are currently palying at that theater. The page also has a
32
+ ``next`` method that give you access to the next page. You can keep navigating
33
+ through the search results until ``page.next`` returns _Last Page_.
22
34
 
23
- $ rails generate bootleg:install
35
+ ```
36
+ page = page.next
37
+ ```
24
38
 
25
- ## Usage
39
+ Within theaters you also have access to movies. A movie has attributes for title
40
+ link and showtimes.
26
41
 
27
- Load content for a certain zipcode:
42
+ ```
43
+ theaters = page.theaters
44
+ ```
28
45
 
29
- Example:
46
+ This gives you a list of theaters on the current page.
30
47
 
31
- $ Bootleg.load('21102')
48
+ For example if you would like access to all the movies that run in a certain
49
+ theater you could do the following.
32
50
 
33
- This will load all the movies for a 25 mile radius(including their showtimes
34
- and the theaters where they play).
51
+ ```
52
+ theaters.first.movies
53
+ ```
54
+ If you need to scrape additional information from either theaters or movies it is
55
+ fairly easy to extend either the theater class or the movies class.
35
56
 
36
- Examples:
57
+ ## TODO
37
58
 
38
- $ movie = BootlegMovie.where(name: 'Example').first
39
-
40
- Get all the theaters where the movie is played:
41
-
42
- $ theaters = movie.theaters
43
-
44
- Get all the showtimes:
45
-
46
- $ showtimes = movie.showtimes
47
-
48
- Get the theater of the showtimes:
49
-
50
- $ theater = showtimes.first.theater
51
-
52
- ## Other Details
53
-
54
- The content is stored in 3 Active Record models BootlegMovie,
55
- BootlegTheater and BootlegShowtime. After you load a zipcode just start
56
- a rails console and take a look at the models to see what information is
57
- stored inside.
58
-
59
- The zipcode is stored under BootlegShowtime.:w
60
-
61
- ## Contributing
62
-
63
- 1. Fork it
64
- 2. Create your feature branch (`git checkout -b my-new-feature`)
65
- 3. Commit your changes (`git commit -am 'Add some feature'`)
66
- 4. Push to the branch (`git push origin my-new-feature`)
67
- 5. Create new Pull Request
68
-
69
- It is relatively easy to pull other information from moviefone.com. If you
70
- need an extra feature and you would like to contribute feel free to shoot
71
- me an email at marius@mlpinit.com beforehand.
59
+ Some of the main improvements this gem could use is better error support for
60
+ different edge cases. The gem could also benefit from more tests to ilustrate
61
+ some of the patterns identified while scrapping movies and theaters.
@@ -8,17 +8,16 @@ Gem::Specification.new do |gem|
8
8
  gem.version = Bootleg::VERSION
9
9
  gem.authors = ["Marius L. Pop"]
10
10
  gem.email = ["marius@mlpinit.com"]
11
- gem.description = %q{ Scraping theaters, movies, showtimes and other relevant data from movifone.com}
12
- gem.summary = %q{ ...comming soon...}
11
+ gem.description = %q{ This gems allows you to navigate through the results }
12
+ gem.summary = %q{ Zipcode based scrapping for moviefone.com }
13
13
  gem.homepage = ""
14
14
 
15
15
  gem.files = `git ls-files`.split($/)
16
16
  gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
17
17
  gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
18
18
  gem.require_paths = ["lib"]
19
-
20
- gem.add_dependency "nokogiri"
21
- gem.add_dependency "mechanize"
22
- gem.add_dependency "activerecord"
23
- gem.add_development_dependency "rspec"
19
+
20
+ gem.add_dependency 'mechanize', '~> 2.7.3'
21
+
22
+ gem.add_development_dependency 'rspec', '3.0.0.beta2'
24
23
  end
@@ -1,8 +1,6 @@
1
+ require "mechanize"
1
2
  require "bootleg/version"
2
- require "presenter"
3
-
4
- module Bootleg
5
- def self.load(zipcode)
6
- Presenter.new(zipcode)
7
- end
8
- end
3
+ require "bootleg/agent"
4
+ require "bootleg/movie"
5
+ require "bootleg/theater"
6
+ require "bootleg/page"
@@ -0,0 +1,31 @@
1
+ module Bootleg
2
+ class Agent
3
+
4
+ attr_reader :zipcode
5
+
6
+ def initialize(args)
7
+ @zipcode = args.fetch(:zipcode)
8
+ end
9
+
10
+ def page
11
+ @page ||= Bootleg::Page.new page: mechanize.submit(search_form)
12
+ end
13
+
14
+ private
15
+
16
+ def mechanize
17
+ Mechanize.new
18
+ end
19
+
20
+ def home_page
21
+ mechanize.get('http://moviefone.com')
22
+ end
23
+
24
+ def search_form
25
+ home_page.form_with(id: 'frm-search').tap do |form|
26
+ form.fields.last.value = zipcode
27
+ end
28
+ end
29
+
30
+ end
31
+ end
@@ -0,0 +1,30 @@
1
+ module Bootleg
2
+ class Movie
3
+
4
+ attr_reader :movie
5
+ private :movie
6
+
7
+ def initialize(args)
8
+ @movie ||= args.fetch(:movie)
9
+ end
10
+
11
+ def title
12
+ movie_info.text
13
+ end
14
+
15
+ def link
16
+ movie_info.attributes['href'].value
17
+ end
18
+
19
+ def showtimes
20
+ movie.search('span.stDisplay').map { |times| times.text }
21
+ end
22
+
23
+ private
24
+
25
+ def movie_info
26
+ @movie_info ||= movie.search('div.movietitle a').last
27
+ end
28
+
29
+ end
30
+ end
@@ -0,0 +1,26 @@
1
+ module Bootleg
2
+ class Page
3
+
4
+ attr_reader :page
5
+
6
+ def initialize(args)
7
+ @page ||= args.fetch(:page)
8
+ end
9
+
10
+ def next
11
+ link ? self.class.new(page: link.click) : 'Last Page'
12
+ end
13
+
14
+ def theaters
15
+ page.search('div.theater').
16
+ map { |theater| Bootleg::Theater.new(theater: theater) }
17
+ end
18
+
19
+ private
20
+
21
+ def link
22
+ @link ||= page.link_with(class: 'next-showtime')
23
+ end
24
+
25
+ end
26
+ end
@@ -0,0 +1,54 @@
1
+ module Bootleg
2
+ class Theater
3
+
4
+ attr_reader :theater
5
+ private :theater
6
+
7
+ def initialize(args)
8
+ @theater ||= args.fetch(:theater)
9
+ end
10
+
11
+ def title
12
+ title_info.text
13
+ end
14
+
15
+ def link
16
+ title_info.attributes['href'].value
17
+ end
18
+
19
+ def address
20
+ # removes the phone number at the end of the address and the white space
21
+ theater.search('p.address').text.sub(/\|.*/,'').strip
22
+ end
23
+
24
+ def adult_price
25
+ prices.first
26
+ end
27
+
28
+ def child_price
29
+ prices.last
30
+ end
31
+
32
+ def movies
33
+ theater.search('div.movie-data-wrap').
34
+ map { |movie| Bootleg::Movie.new(movie: movie) }
35
+ end
36
+
37
+ private
38
+
39
+ def title_info
40
+ @title_link ||= theater.search('div.title a').last
41
+ end
42
+
43
+ def prices
44
+ @prices ||= theater.
45
+ search("div.prices").
46
+ first.
47
+ text.
48
+ gsub(/\s/,'').
49
+ split('|').
50
+ map { |price| $& if price.match /\$.*\d{2}/ }
51
+ end
52
+
53
+ end
54
+ end
@@ -1,3 +1,3 @@
1
1
  module Bootleg
2
- VERSION = "0.0.6"
2
+ VERSION = "0.0.7"
3
3
  end
@@ -0,0 +1,11 @@
1
+ require 'spec_helper'
2
+ require 'bootleg/agent'
3
+
4
+ describe Bootleg::Agent do
5
+
6
+ it 'raises ArgumentError if not initialized with a zipcode' do
7
+ expect{described_class.new(not_zipcode: 'hey')}.
8
+ to raise_error(KeyError, 'key not found: :zipcode')
9
+ end
10
+
11
+ end
@@ -0,0 +1,37 @@
1
+ require 'spec_helper'
2
+ require 'bootleg/movie'
3
+
4
+ describe Bootleg::Movie do
5
+
6
+ it 'raises KeyError if not initialized with a nokogiri-movie' do
7
+ expect{described_class.new(not_movie: 'not movie')}.
8
+ to raise_error(KeyError, 'key not found: :movie')
9
+ end
10
+
11
+ let(:attributes) {}
12
+ let(:nokogiri_movie) { double 'Nokogiri Movie' }
13
+ let(:href) { double( 'Href', value: 'http://moviefone.com') }
14
+ let(:movie) { double('Movie', text: 'Matrix', attributes: { 'href' => href }) }
15
+ let(:times) { [ double(text: '1am'), double(text: '2pm')] }
16
+
17
+ subject { described_class.new(movie: nokogiri_movie) }
18
+
19
+ it 'has a title' do
20
+ allow(nokogiri_movie).to receive_message_chain(:search, :last).
21
+ and_return(movie)
22
+ expect(subject.title).to eq('Matrix')
23
+ end
24
+
25
+ it 'has a link' do
26
+ allow(nokogiri_movie).to receive_message_chain(:search, :last).
27
+ and_return(movie)
28
+ expect(subject.link).to eq('http://moviefone.com')
29
+ end
30
+
31
+ it 'has showtimes' do
32
+ allow(nokogiri_movie).to receive(:search).with('span.stDisplay').
33
+ and_return(times)
34
+ expect(subject.showtimes).to include('1am', '2pm')
35
+ end
36
+
37
+ end
@@ -0,0 +1,21 @@
1
+ require 'spec_helper'
2
+ require 'bootleg/page'
3
+
4
+ describe Bootleg::Page do
5
+
6
+ it 'raises KeyError if not initialized with a nokogiri-page' do
7
+ expect{described_class.new(not_page: 'not page')}.
8
+ to raise_error(KeyError, 'key not found: :page')
9
+ end
10
+
11
+ let(:link) { double 'Link' }
12
+ let(:nokogiri_page) { double 'Nokogiri Page' }
13
+ subject { described_class.new(page: nokogiri_page) }
14
+
15
+ it 'returns "Last Page" if no more pages' do
16
+ allow(nokogiri_page).to receive(:link_with).
17
+ with(class: 'next-showtime').and_return(nil)
18
+ expect(subject.next).to eq('Last Page')
19
+ end
20
+
21
+ end
@@ -0,0 +1,52 @@
1
+ require 'spec_helper'
2
+ require 'bootleg/theater'
3
+
4
+ describe Bootleg::Theater do
5
+
6
+ it 'raises KeyError if not initialized with a nokogiri-theater' do
7
+ expect{described_class.new(not_theater: 'not theater')}.
8
+ to raise_error(KeyError, 'key not found: :theater')
9
+ end
10
+
11
+ let(:nokogiri_theater) { double 'Nokogiri Theater' }
12
+ let(:theater) { double('Theater', text: 'Royal', attributes: { 'href' => href }) }
13
+ let(:href) { double( 'Href', value: 'http://moviefone.com') }
14
+ let(:address) { double('Address', text: "\n\n\t Rockville, MD | 234-222") }
15
+ let(:prices) { double('Prices', text: 'RegularPrice:$11.50 | ChildPrice:$6.50') }
16
+
17
+ subject { described_class.new(theater: nokogiri_theater) }
18
+
19
+ it 'has a title' do
20
+ allow(nokogiri_theater).to receive_message_chain(:search, :last).
21
+ and_return(theater)
22
+ expect(subject.title).to eq('Royal')
23
+ end
24
+
25
+ it 'has a link' do
26
+ allow(nokogiri_theater).to receive_message_chain(:search, :last).
27
+ and_return(theater)
28
+ expect(subject.link).to eq('http://moviefone.com')
29
+ end
30
+
31
+ it 'has an address' do
32
+ allow(nokogiri_theater).to receive(:search).with('p.address').
33
+ and_return(address)
34
+ expect(subject.address).to eq("Rockville, MD")
35
+ end
36
+
37
+ context 'has price' do
38
+ before do
39
+ allow(nokogiri_theater).to receive_message_chain(:search, :first).
40
+ and_return(prices)
41
+ end
42
+
43
+ it 'for adult' do
44
+ expect(subject.adult_price).to eq('$11.50')
45
+ end
46
+
47
+ it 'for child' do
48
+ expect(subject.child_price).to eq('$6.50')
49
+ end
50
+ end
51
+
52
+ end
metadata CHANGED
@@ -1,148 +1,96 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: bootleg
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.6
5
- prerelease:
4
+ version: 0.0.7
6
5
  platform: ruby
7
6
  authors:
8
7
  - Marius L. Pop
9
8
  autorequire:
10
9
  bindir: bin
11
10
  cert_chain: []
12
- date: 2012-11-24 00:00:00.000000000 Z
11
+ date: 2016-05-25 00:00:00.000000000 Z
13
12
  dependencies:
14
- - !ruby/object:Gem::Dependency
15
- name: nokogiri
16
- requirement: !ruby/object:Gem::Requirement
17
- none: false
18
- requirements:
19
- - - ! '>='
20
- - !ruby/object:Gem::Version
21
- version: '0'
22
- type: :runtime
23
- prerelease: false
24
- version_requirements: !ruby/object:Gem::Requirement
25
- none: false
26
- requirements:
27
- - - ! '>='
28
- - !ruby/object:Gem::Version
29
- version: '0'
30
13
  - !ruby/object:Gem::Dependency
31
14
  name: mechanize
32
15
  requirement: !ruby/object:Gem::Requirement
33
- none: false
34
- requirements:
35
- - - ! '>='
36
- - !ruby/object:Gem::Version
37
- version: '0'
38
- type: :runtime
39
- prerelease: false
40
- version_requirements: !ruby/object:Gem::Requirement
41
- none: false
42
- requirements:
43
- - - ! '>='
44
- - !ruby/object:Gem::Version
45
- version: '0'
46
- - !ruby/object:Gem::Dependency
47
- name: activerecord
48
- requirement: !ruby/object:Gem::Requirement
49
- none: false
50
16
  requirements:
51
- - - ! '>='
17
+ - - "~>"
52
18
  - !ruby/object:Gem::Version
53
- version: '0'
19
+ version: 2.7.3
54
20
  type: :runtime
55
21
  prerelease: false
56
22
  version_requirements: !ruby/object:Gem::Requirement
57
- none: false
58
23
  requirements:
59
- - - ! '>='
24
+ - - "~>"
60
25
  - !ruby/object:Gem::Version
61
- version: '0'
26
+ version: 2.7.3
62
27
  - !ruby/object:Gem::Dependency
63
28
  name: rspec
64
29
  requirement: !ruby/object:Gem::Requirement
65
- none: false
66
30
  requirements:
67
- - - ! '>='
31
+ - - '='
68
32
  - !ruby/object:Gem::Version
69
- version: '0'
33
+ version: 3.0.0.beta2
70
34
  type: :development
71
35
  prerelease: false
72
36
  version_requirements: !ruby/object:Gem::Requirement
73
- none: false
74
37
  requirements:
75
- - - ! '>='
38
+ - - '='
76
39
  - !ruby/object:Gem::Version
77
- version: '0'
78
- description: ! ' Scraping theaters, movies, showtimes and other relevant data from
79
- movifone.com'
40
+ version: 3.0.0.beta2
41
+ description: " This gems allows you to navigate through the results "
80
42
  email:
81
43
  - marius@mlpinit.com
82
44
  executables: []
83
45
  extensions: []
84
46
  extra_rdoc_files: []
85
47
  files:
86
- - .gitignore
48
+ - ".gitignore"
87
49
  - Gemfile
88
50
  - LICENSE.txt
89
51
  - README.md
90
- - README.rdoc
91
52
  - Rakefile
92
53
  - bootleg.gemspec
93
54
  - lib/bootleg.rb
55
+ - lib/bootleg/agent.rb
56
+ - lib/bootleg/movie.rb
57
+ - lib/bootleg/page.rb
58
+ - lib/bootleg/theater.rb
94
59
  - lib/bootleg/version.rb
95
- - lib/extractor.rb
96
- - lib/finder.rb
97
- - lib/generators/bootleg/USAGE
98
- - lib/generators/bootleg/install_generator.rb
99
- - lib/generators/bootleg/movie_generator.rb
100
- - lib/generators/bootleg/showtime_generator.rb
101
- - lib/generators/bootleg/templates/movie_migration.rb
102
- - lib/generators/bootleg/templates/movie_model.rb
103
- - lib/generators/bootleg/templates/showtime_migration.rb
104
- - lib/generators/bootleg/templates/showtime_model.rb
105
- - lib/generators/bootleg/templates/theater_migration.rb
106
- - lib/generators/bootleg/templates/theater_model.rb
107
- - lib/generators/bootleg/theater_generator.rb
108
- - lib/manager.rb
109
- - lib/modules/href.rb
110
- - lib/modules/movie.rb
111
- - lib/modules/theater.rb
112
- - lib/modules/zipcode.rb
113
- - lib/presenter.rb
114
60
  - spec/.rspec
115
- - spec/extractor_spec.rb
116
- - spec/finder_spec.rb
117
- - spec/presenter_spec.rb
61
+ - spec/lib/agent_spec.rb
62
+ - spec/lib/movie_spec.rb
63
+ - spec/lib/page_spec.rb
64
+ - spec/lib/theater_spec.rb
118
65
  - spec/spec_helper.rb
119
66
  homepage: ''
120
67
  licenses: []
68
+ metadata: {}
121
69
  post_install_message:
122
70
  rdoc_options: []
123
71
  require_paths:
124
72
  - lib
125
73
  required_ruby_version: !ruby/object:Gem::Requirement
126
- none: false
127
74
  requirements:
128
- - - ! '>='
75
+ - - ">="
129
76
  - !ruby/object:Gem::Version
130
77
  version: '0'
131
78
  required_rubygems_version: !ruby/object:Gem::Requirement
132
- none: false
133
79
  requirements:
134
- - - ! '>='
80
+ - - ">="
135
81
  - !ruby/object:Gem::Version
136
82
  version: '0'
137
83
  requirements: []
138
84
  rubyforge_project:
139
- rubygems_version: 1.8.24
85
+ rubygems_version: 2.5.1
140
86
  signing_key:
141
- specification_version: 3
142
- summary: ! '...comming soon...'
87
+ specification_version: 4
88
+ summary: Zipcode based scrapping for moviefone.com
143
89
  test_files:
144
90
  - spec/.rspec
145
- - spec/extractor_spec.rb
146
- - spec/finder_spec.rb
147
- - spec/presenter_spec.rb
91
+ - spec/lib/agent_spec.rb
92
+ - spec/lib/movie_spec.rb
93
+ - spec/lib/page_spec.rb
94
+ - spec/lib/theater_spec.rb
148
95
  - spec/spec_helper.rb
96
+ has_rdoc:
@@ -1,3 +0,0 @@
1
- == Bootleg - moviefone scraper
2
-
3
- This is a gem in development. It's purpose is to scrape theaters, movies, showtimes and other details from moviefone.com .
@@ -1,32 +0,0 @@
1
- require 'mechanize'
2
- require 'nokogiri'
3
- require 'open-uri'
4
- require_relative 'finder'
5
- require_relative 'modules/theater'
6
-
7
- class Extractor
8
-
9
- attr_reader :page_theaters
10
-
11
- def initialize(page, zipcode)
12
- @page = (Nokogiri::HTML(open(page)))
13
- @page_theaters = []
14
- extract_movies
15
- @zipcode ||= zipcode
16
- end
17
-
18
- def extract_movies
19
- theaters.each do |theater|
20
- theater.extend Theater
21
- BootlegTheater.create!(name: theater.name, href: theater.link)
22
- theater_info = { name: theater.name, href: theater.link, movies: theater.movies}
23
- @page_theaters << theater_info
24
- end
25
- end
26
-
27
- private
28
-
29
- def theaters
30
- @page.css('div.theater')
31
- end
32
- end
@@ -1,14 +0,0 @@
1
- require_relative 'modules/zipcode'
2
- require_relative 'modules/href'
3
-
4
- class Finder
5
- def initialize(zipcode)
6
- zipcode.extend Zipcode
7
- @href = zipcode.search
8
- end
9
-
10
- def hrefs
11
- @href.extend Href
12
- @href.all
13
- end
14
- end
File without changes
@@ -1,15 +0,0 @@
1
- require 'rails/generators/migration'
2
-
3
- module Bootleg
4
- module Generators
5
- class InstallGenerator < Rails::Generators::Base
6
- source_root File.expand_path('../templates', __FILE__)
7
-
8
- def run_generators
9
- generate "bootleg:movie"
10
- generate "bootleg:theater"
11
- generate "bootleg:showtime"
12
- end
13
- end
14
- end
15
- end
@@ -1,24 +0,0 @@
1
- require 'rails/generators/migration'
2
-
3
- module Bootleg
4
- module Generators
5
- class MovieGenerator < Rails::Generators::Base
6
- include Rails::Generators::Migration
7
-
8
- source_root File.expand_path('../templates', __FILE__)
9
-
10
- def generate_movie_migration
11
- migration_template "movie_migration.rb", "db/migrate/create_bootleg_movies.rb"
12
- end
13
-
14
- def generate_movie_model
15
- copy_file "movie_model.rb", "app/models/bootleg_movie.rb"
16
- end
17
-
18
- def self.next_migration_number(path)
19
- @migration_number = Time.now.utc.strftime("%Y%m%d%H%M%S").to_i.to_s
20
- end
21
- end
22
- end
23
- end
24
-
@@ -1,24 +0,0 @@
1
- require 'rails/generators/migration'
2
-
3
- module Bootleg
4
- module Generators
5
- class ShowtimeGenerator < Rails::Generators::Base
6
- include Rails::Generators::Migration
7
-
8
- source_root File.expand_path('../templates', __FILE__)
9
-
10
- def generate_theater_movie_migration
11
- migration_template "showtime_migration.rb", "db/migrate/create_bootleg_showtimes.rb"
12
- end
13
-
14
- def generate_theater_movie_model
15
- copy_file "showtime_model.rb", "app/models/bootleg_showtime.rb"
16
- end
17
-
18
- def self.next_migration_number(path)
19
- @migration_number = Time.now.utc.strftime("%Y%m%d%H%M%S").to_i.to_s
20
- end
21
- end
22
- end
23
- end
24
-
@@ -1,10 +0,0 @@
1
- class CreateBootlegMovies < ActiveRecord::Migration
2
- def change
3
- create_table :bootleg_movies do |t|
4
- t.string :name
5
- t.string :href
6
-
7
- t.timestamps
8
- end
9
- end
10
- end
@@ -1,10 +0,0 @@
1
- class BootlegMovie < ActiveRecord::Base
2
- attr_accessible :name, :href, :showtimes
3
-
4
- has_many :bootleg_showtimes
5
- has_many :theaters, through: :bootleg_showtimes, source: :bootleg_theater
6
-
7
- def showtimes
8
- bootleg_showtimes
9
- end
10
- end
@@ -1,12 +0,0 @@
1
- class CreateBootlegShowtimes < ActiveRecord::Migration
2
- def change
3
- create_table :bootleg_showtimes do |t|
4
- t.integer :bootleg_movie_id
5
- t.integer :bootleg_theater_id
6
- t.string :zipcode
7
- t.string :showtimes
8
-
9
- t.timestamps
10
- end
11
- end
12
- end
@@ -1,14 +0,0 @@
1
- class BootlegShowtime < ActiveRecord::Base
2
- attr_accessible :bootleg_movie_id, :bootleg_theater_id, :zipcode, :showtimes
3
-
4
- belongs_to :bootleg_movie
5
- belongs_to :bootleg_theater
6
-
7
- def theater
8
- bootleg_theater
9
- end
10
-
11
- def movie
12
- bootleg_movie
13
- end
14
- end
@@ -1,10 +0,0 @@
1
- class CreateBootlegTheaters < ActiveRecord::Migration
2
- def change
3
- create_table :bootleg_theaters do |t|
4
- t.string :name
5
- t.string :href
6
-
7
- t.timestamps
8
- end
9
- end
10
- end
@@ -1,10 +0,0 @@
1
- class BootlegTheater < ActiveRecord::Base
2
- attr_accessible :name, :href
3
-
4
- has_many :bootleg_showtimes
5
- has_many :movies, through: :bootleg_showtimes, source: :bootleg_movie
6
-
7
- def showtimes
8
- bootleg_showtimes
9
- end
10
- end
@@ -1,24 +0,0 @@
1
- require 'rails/generators/migration'
2
-
3
- module Bootleg
4
- module Generators
5
- class TheaterGenerator < Rails::Generators::Base
6
- include Rails::Generators::Migration
7
-
8
- source_root File.expand_path('../templates', __FILE__)
9
-
10
- def generate_theater_migration
11
- migration_template "theater_migration.rb", "db/migrate/create_bootleg_theaters.rb"
12
- end
13
-
14
- def generate_theater_model
15
- copy_file "theater_model.rb", "app/models/bootleg_theater.rb"
16
- end
17
-
18
- def self.next_migration_number(path)
19
- @migration_number = Time.now.utc.strftime("%Y%m%d%H%M%S").to_i.to_s
20
- end
21
- end
22
- end
23
- end
24
-
@@ -1,27 +0,0 @@
1
- require_relative 'finder'
2
- require_relative 'extractor'
3
-
4
- class Manager
5
-
6
- class << self
7
- attr_accessor :zipcode
8
- end
9
-
10
- def initialize(zipcode)
11
- @zipcode = zipcode
12
- @pages ||= find_pages
13
- @all_theaters = []
14
- Manager.zipcode = zipcode
15
- end
16
-
17
- def find_pages
18
- Finder.new(@zipcode).hrefs
19
- end
20
-
21
- def extract_theaters
22
- @pages.each do |page|
23
- @all_theaters << Extractor.new(page, @zipcode).page_theaters
24
- end
25
- @all_theaters.flatten
26
- end
27
- end
@@ -1,18 +0,0 @@
1
- module Href
2
- def all
3
- pages = []
4
- count.times { |nr| pages << url + nr.to_s }
5
- pages
6
- end
7
-
8
- private
9
-
10
- def count
11
- self.links.select { |link| link.text.size < 3 and link.text =~ /\d/ }.last.text.to_i
12
- end
13
-
14
- def url
15
- self.uri.to_s + '?page='
16
- end
17
- end
18
-
@@ -1,24 +0,0 @@
1
- module Movie
2
- def name
3
- details.css('a').text.strip
4
- end
5
-
6
- def link
7
- "http://www.moviefone.com" + details.css('a').attribute('href').value
8
- end
9
-
10
- def showtimes
11
- values = []
12
- showtimes = self.css('a.gt').empty? ? self.css('span.stDisplay') : self.css('a.gt')
13
- showtimes.each do |time|
14
- values << time.text
15
- end
16
- values
17
- end
18
-
19
- private
20
-
21
- def details
22
- self.css('div.movietitle')
23
- end
24
- end
@@ -1,45 +0,0 @@
1
- require_relative 'movie'
2
-
3
- module Theater
4
- def name
5
- details.text.strip
6
- end
7
-
8
- def link
9
- details.attribute('href').value
10
- end
11
-
12
- def movies
13
- movies = self.css('div.movie-listing.first')
14
- values = []
15
- theater = BootlegTheater.last
16
- movies.each do |movie|
17
- movie.extend Movie
18
- movie_info = { name: movie.name, href: movie.link, showtimes: movie.showtimes }
19
- values << movie_info
20
- insert_movies(theater,movie)
21
- end
22
- values
23
- end
24
-
25
- private
26
- def details
27
- self.css('h3.title').css('a')
28
- end
29
-
30
- def insert_movies(theater, movie)
31
- existing_movie = BootlegMovie.where(name: movie.name).first
32
- if existing_movie
33
- showtime = theater.bootleg_showtimes.new
34
- showtime.bootleg_movie_id = existing_movie.id
35
- showtime.save
36
- else
37
- theater.movies.create!(name: movie.name, href: movie.link)
38
- end
39
- showtime = BootlegShowtime.last
40
- showtime.showtimes = movie.showtimes.to_s.gsub(/-/, '').gsub(/\n/,'').strip
41
- showtime.zipcode = Manager.zipcode
42
- showtime.date = Time.zone.now
43
- showtime.save
44
- end
45
- end
@@ -1,12 +0,0 @@
1
- require 'mechanize'
2
-
3
- module Zipcode
4
- def search
5
- agent = Mechanize.new
6
- page = agent.get("http://www.moviefone.com")
7
- search_form = page.form_with id: 'frm-search'
8
- search_form.fields[1].value = self
9
- agent.submit search_form
10
- end
11
- end
12
-
@@ -1,8 +0,0 @@
1
- require_relative 'manager'
2
- class Presenter
3
- attr_reader :theaters
4
-
5
- def initialize(zipcode)
6
- @theaters ||= Manager.new(zipcode).extract_theaters
7
- end
8
- end
@@ -1,59 +0,0 @@
1
- require 'spec_helper'
2
-
3
- describe Extractor do
4
- it "should raise an error withouth arguments" do
5
- expect{ Extractor.new }.to raise_error(ArgumentError)
6
- end
7
-
8
- it "should not raise error with one argument" do
9
- expect { Extractor.new("http://www.moviefone.com")}.to_not raise_error(ArgumentError)
10
- end
11
-
12
- it "should raise an error with more then one argument" do
13
- expect { Extractor.new("arg1", "arg2") }.to raise_error(ArgumentError)
14
- end
15
-
16
- before :all do
17
- @theaters = Extractor.new("http://www.moviefone.com/showtimes/manchester-md/21102/theaters").page_theaters
18
- end
19
-
20
- it "should pull out no more then 5 theaters" do
21
- @theaters.size.should eq(5)
22
- end
23
-
24
- describe Theater do
25
- before :all do
26
- @theater = @theaters[1]
27
- end
28
-
29
- it "name should match expression" do
30
- expect(@theater[:name]).to match(/(\w|\s)/)
31
- end
32
-
33
- it "href should be a link" do
34
- expect(@theater[:href]).to match(/http:\/\/www\.moviefone\.com/)
35
- end
36
-
37
- describe Movie do
38
-
39
- before :all do
40
- @movie = @theater[:movies].first
41
- end
42
- it "should have a name, href and showtimes" do
43
- expect(@movie.size).to eq(3)
44
- end
45
-
46
- it "name should match expression" do
47
- expect(@movie[:name]).to match(/(\w|\s)/)
48
- end
49
-
50
- it "href shoud mathc expression" do
51
- expect(@movie[:href]).to match(/http:\/\/www\.moviefone\.com/)
52
- end
53
-
54
- it "shotimes returns an array" do
55
- expect(@movie[:showtimes].class).to be(Array)
56
- end
57
- end
58
- end
59
- end
@@ -1,26 +0,0 @@
1
- require 'spec_helper'
2
-
3
- describe Finder do
4
- it "should raise an error with no arguments" do
5
- expect { Finder.new }.to raise_error(ArgumentError)
6
- end
7
-
8
- it "should not raise error with one argument" do
9
- expect { Finder.new("smth") }.to_not raise_error(ArgumentError)
10
- end
11
-
12
- it "should raise an error with more then one arguments" do
13
- expect { Finder.new("smth", "smthelse") }.to raise_error(ArgumentError)
14
- end
15
-
16
-
17
- describe Href do
18
- before :all do
19
- @hrefs = Finder.new('21102').hrefs
20
- end
21
-
22
- it "should have a size of 3" do
23
- expect(@hrefs.size).to eq(3)
24
- end
25
- end
26
- end
@@ -1,4 +0,0 @@
1
- require 'spec_helper'
2
-
3
- describe Presenter do
4
- end