rcarvalho-link_thumbnailer 1.0.9.1

data/.gitignore

@@ -0,0 +1,19 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+.project
+.coveralls.yml

data/.rspec

@@ -0,0 +1,2 @@
+--colour
+--format=nested

data/.travis.yml

@@ -0,0 +1,6 @@
+bundler_args: --without development
+language: ruby
+rvm:
+  - 1.9.2
+  - 1.9.3
+  - 2.0.0

data/CHANGELOG.md

@@ -0,0 +1,91 @@
+# 1.0.9
+
+- Fix issue when Location header used a relative path instead of an absolute path
+- Update gemfile to be more flexible when using Hashie gem
+
+# 1.0.8
+
+- Thanks to [juriglx](https://github.com/juriglx), support for canonical urls
+- Bug fixes
+
+# 1.0.7
+
+- Fix: Issue with preview controller
+
+# 1.0.6
+
+- Fix: Issue when setting `strict` option. Always returning OG representation.
+
+# 1.0.5
+
+- Thanks to [phlegx](https://github.com/phlegx), support for timeout http connection through configurations.
+
+# 1.0.4
+
+- Fix issue #7: nil img was returned when exception is raised. Now skiping nil images in results.
+- Thanks to [phlegx](https://github.com/phlegx), support for SSL and User Agent customization through configurations.
+
+# 1.0.3
+
+- Fix issue #5: Url was incorect in case of HTTP Redirections.
+
+# 1.0.2
+
+- Feature: User can now set options at runtime by passing valid options to ```generate``` method
+- Bug fix when doing ```rails g link_thumbnailer:install``` by explicitly specifying the scope of Rails
+
+# 1.0.1
+
+- Refactor LinkThumbnailer#generate method to have a cleaner code
+
+# 1.0.0
+
+- Update readme
+- Add PreviewController for easy integration with user's app
+- Add link_thumbnailer routes for easy integration with user's app
+- Refactor some code
+- Change 'to_a' method to 'to_hash' in object model
+
+# 0.0.6
+
+- Update readme
+- Add `to_a` to WebImage class
+- Refactor `to_json` for WebImage class
+- Add specs corresponding
+
+# 0.0.5
+
+- Bug fix
+- Remove `require 'rails'` from spec_helper.rb
+- Remove rails dependences (blank? method) in code
+- Spec fix
+
+# 0.0.4
+
+- Add specs for almost all classes
+- Add a method `to_json` for WebImage class to be able to get a usable array of images' attributes
+
+# 0.0.3
+
+- Add specs for LinkThumbnailer class
+- Refactor config system, now using dedicated configuration class
+
+# 0.0.2
+
+- Added Rspec
+- Bug fixes:
+	- Now checking if attribute is blank for LinkThumbnailer::Object.valid? method
+
+# 0.0.1
+
+- LinkThumbnailer::Object
+- LinkThumbnailer::Doc
+- LinkThumbnailer::DocParser
+- LinkThumbnailer::Fetcher
+- LinkThumbnailer::ImgComparator
+- LinkThumbnailer::ImgParser
+- LinkThumbnailer::ImgUrlFilter
+- LinkThumbnailer::Opengraph
+- LinkThumbnailer::WebImage
+- LinkThumbnailer.configure
+- LinkThumbnailer.generate

data/Gemfile

@@ -0,0 +1,12 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in link_thumbnailer.gemspec
+gemspec
+
+group :test do
+  gem 'coveralls',  require: false
+  gem 'simplecov',  require: false
+  gem 'rspec',      '~> 2.14'
+  gem 'webmock',    '~> 1.14'
+  gem 'pry',        '~> 0.9'
+end

data/LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2012 Pierre-Louis Gottfrois
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,184 @@
+# LinkThumbnailer
+
+[![Code Climate](https://codeclimate.com/github/gottfrois/link_thumbnailer.png)](https://codeclimate.com/github/gottfrois/link_thumbnailer)
+[![Coverage Status](https://coveralls.io/repos/gottfrois/link_thumbnailer/badge.png?branch=master)](https://coveralls.io/r/gottfrois/link_thumbnailer?branch=master)
+[![Build Status](https://travis-ci.org/gottfrois/link_thumbnailer.png?branch=master)](https://travis-ci.org/gottfrois/link_thumbnailer)
+
+Ruby gem generating image thumbnails from a given URL. Rank them and give you back an object containing images and website informations. Works like Facebook link previewer.
+
+Demo Application is [here](http://link-thumbnailer-demo.herokuapp.com/) !
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'link_thumbnailer'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install link_thumbnailer
+
+Run:
+
+	$ rails g link_thumbnailer:install
+
+This will add `link_thumbnailer.rb` to `config/initializers/`. See [#Configuration](https://github.com/gottfrois/link_thumbnailer#configuration) for more details.
+
+## Usage
+
+Run `irb` and require the gem:
+
+	require 'rails'
+	 => true
+
+	require 'link_thumbnailer'
+	 => true
+
+This gem can handle [Opengraph](http://ogp.me/) protocol. Here is an example with such a website:
+
+	object = LinkThumbnailer.generate('http://zerply.com')
+	 => #<LinkThumbnailer::Object description="Go beyond the résumé - showcase your work and your talent" image="http://zerply.com/img/front/facebook_icon_green.png" images=["http://zerply.com/img/front/facebook_icon_green.png"] site_name="zerply.com" title="Join Me on Zerply" url="http://zerply.com">
+
+	object.title?
+ 	=> true
+ 	object.title
+ 	=> "Join Me on Zerply"
+
+ 	object.url?
+	=> true
+	object.url
+	=> "http://zerply.com"
+
+	object.foo?
+	=> false
+	object.foo
+	=> nil
+
+Now with a regular website with no particular protocol:
+
+	object = LinkThumbnailer.generate('http://foo.com')
+	 => #<LinkThumbnailer::Object description=nil images=[[ JPEG 750x200 750x200+0+0 DirectClass 8-bit 45kb] scene=0] title="Foo.com" url="http://foo.com">
+
+	object.title
+	 => "Foo.com"
+
+	object.images
+	 => [[ JPEG 750x200 750x200+0+0 DirectClass 8-bit 45kb]
+	scene=0]
+
+	object.images.first.source_url
+	 => #<URI::HTTP:0x007ff7a923ef58 URL:http://foo.com/media/BAhbB1sHOgZmSSItMjAxMi8wNC8yNi8yMC8xMS80OS80MjYvY29yZG92YWJlYWNoLmpwZwY6BkVUWwg6BnA6CnRodW1iSSINNzUweDIwMCMGOwZU/cordovabeach.jpg>
+
+	object.to_hash
+	 => {"url"=>"http://foo.com", "images"=>[{:source_url=>"http://foo.com/media/BAhbB1sHOgZmSSItMjAxMi8wNC8yNi8yMC8xMS80OS80MjYvY29yZG92YWJlYWNoLmpwZwY6BkVUWwg6BnA6CnRodW1iSSINNzUweDIwMCMGOwZU/cordovabeach.jpg", :mime_type=>"image/jpeg", :rows=>200, :filesize=>46501, :number_colors=>9490}], "title"=>"Foo.com", "description"=>nil}
+
+	object.to_json
+	 => "{\"url\":\"http://foo.com\",\"images\":[{\"source_url\":\"http://foo.com/media/BAhbB1sHOgZmSSItMjAxMi8wNC8yNi8yMC8xMS80OS80MjYvY29yZG92YWJlYWNoLmpwZwY6BkVUWwg6BnA6CnRodW1iSSINNzUweDIwMCMGOwZU/cordovabeach.jpg\",\"mime_type\":\"image/jpeg\",\"rows\":200,\"filesize\":46501,\"number_colors\":9490}],\"title\":\"Foo.com\",\"description\":null}"
+
+You can check whether this object is valid or not (set mandatory attributes in the initializer, defaults are `[url, title, images]`)
+
+	object.valid?
+ 	=> true
+
+ You also can set options at runtime:
+
+ 	object = LinkThumbnailer.generate('http://foo.com', top: 10, limit: 20, redirect_limit: 5)
+
+## Preview Controller
+
+For an easy integration into your application, use the builtin `PreviewController`.
+
+Take a look at the demo application [here](https://github.com/gottfrois/link_thumbnailer_demo).
+
+Basically, all you have to do in your view is something like this:
+
+	<%= form_tag '/link/preview', method: :post, remote: true do %>
+		<%= text_field_tag :url %>
+		<%= submit_tag 'Preview' %>
+	<% end %>
+
+Don't forget to add this anywhere in your `routes.rb` file:
+
+	use_link_thumbnailer
+
+Note: You won't have to bother with this if you did run the installer using:
+
+	$ rails g link_thumbnailer:install
+
+The `PreviewController` will automatically respond to json calls, returning json version of the preview object. Just like in the IRB console above.
+
+## Configuration
+
+In `config/initializers/link_thumbnailer.rb`
+
+	LinkThumbnailer.configure do |config|
+	  # Set mandatory attributes require for the website to be valid.
+	  # You can set `strict` to false if you want to skip this validation.
+	  # config.mandatory_attributes = %w(url title image)
+
+	  # Whether you want to validate given website against mandatory attributes or not.
+	  # config.strict = true
+
+	  # Numbers of redirects before raising an exception when trying to parse given url.
+	  # config.redirect_limit = 3
+
+	  # List of blacklisted urls you want to skip when searching for images.
+	  # config.blacklist_urls = [
+	  #   %r{^http://ad\.doubleclick\.net/},
+	  #   %r{^http://b\.scorecardresearch\.com/},
+	  #   %r{^http://pixel\.quantserve\.com/},
+	  #   %r{^http://s7\.addthis\.com/}
+	  # ]
+
+	  # Fetch 10 images maximum.
+	  # config.limit = 10
+
+	  # Return top 5 images only.
+	  # config.top = 5
+
+	  # Set user agent
+	  # config.user_agent = 'linkthumbnailer'
+
+	  # Enable or disable SSL verification
+  	# config.verify_ssl = true
+
+	  # HTTP open_timeout: The amount of time in seconds to wait for a connection to be opened.
+	  # config.http_timeout = 5
+	end
+
+## Features
+
+Implemented:
+
+- Implements [OpenGraph](http://ogp.me/) protocol.
+- Find images and sort them according to how well they represent what the page is about (includes absolute images).
+- Sort images based on their size and color.
+- Blacklist some well known advertisings image urls.
+- Routes and Controllers to handle preview generation
+
+Coming soon:
+
+- Use the gem [ruby-readability](https://github.com/iterationlabs/ruby-readability) to parse images and website information
+- Cache results on filesystem
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Run the specs (`bundle exec rspec spec`)
+4. Commit your changes (`git commit -am 'Added some feature'`)
+5. Push to the branch (`git push origin my-new-feature`)
+6. Create new Pull Request
+
+## Contributors
+
+- [phlegx](https://github.com/phlegx)
+- [juriglx](https://github.com/juriglx)
+
+
+[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/gottfrois/link_thumbnailer/trend.png)](https://bitdeli.com/free "Bitdeli Badge")
+

data/Rakefile

@@ -0,0 +1,7 @@
+require 'bundler/gem_tasks'
+require 'rspec/core/rake_task'
+
+RSpec::Core::RakeTask.new('spec')
+
+task default: :spec
+task test:    :spec

data/app/controllers/link_thumbnailer/application_controller.rb

@@ -0,0 +1,4 @@
+module LinkThumbnailer
+  class ApplicationController < ActionController::Base
+  end
+end

data/app/controllers/link_thumbnailer/previews_controller.rb

@@ -0,0 +1,11 @@
+module LinkThumbnailer
+  class PreviewsController < ApplicationController
+
+    respond_to :json
+
+    def create
+      @preview = LinkThumbnailer.generate(params[:url])
+      render json: @preview.to_json
+    end
+  end
+end

data/lib/generators/link_thumbnailer/install_generator.rb

@@ -0,0 +1,19 @@
+module LinkThumbnailer
+  module Generators
+    class InstallGenerator < ::Rails::Generators::Base
+
+      source_root File.expand_path('../../templates', __FILE__)
+
+      desc 'Creates a LinkThumbnailer initializer for your application.'
+
+      def install
+        route "use_link_thumbnailer"
+      end
+
+      def copy_initializer
+        template 'initializer.rb', 'config/initializers/link_thumbnailer.rb'
+      end
+
+    end
+  end
+end

data/lib/generators/templates/initializer.rb

@@ -0,0 +1,41 @@
+# Use this hook to configure LinkThumbnailer bahaviors.
+LinkThumbnailer.configure do |config|
+  # Set mandatory attributes require for the website to be valid.
+  # You can set `strict` to false if you want to skip this validation.
+  # config.mandatory_attributes = %w(url title image)
+
+  # Whether you want to validate given website against mandatory attributes or not.
+  # config.strict = true
+
+  # Numbers of redirects before raising an exception when trying to parse given url.
+  # config.redirect_limit = 3
+
+  # List of blacklisted urls you want to skip when searching for images.
+  # config.blacklist_urls = [
+  #   %r{^http://ad\.doubleclick\.net/},
+  #   %r{^http://b\.scorecardresearch\.com/},
+  #   %r{^http://pixel\.quantserve\.com/},
+  #   %r{^http://s7\.addthis\.com/}
+  # ]
+
+  # Included Rmagick attributes for images. See http://www.imagemagick.org/RMagick/doc/
+  # for more details.
+  # 'source_url' is a custom attribute and should always be included since this
+  # is where you'll find the image url.
+  # config.rmagick_attributes = %w(source_url mime_type colums rows filesize number_colors)
+
+  # Fetch 10 images maximum.
+  # config.limit = 10
+
+  # Return top 5 images only.
+  # config.top = 5
+  
+  # Set user agent
+  # config.user_agent = 'linkthumbnailer'
+  
+  # Enable or disable SSL verification 
+  # config.verify_ssl = true
+  
+  # HTTP open_timeout: The amount of time in seconds to wait for a connection to be opened.
+  # config.http_timeout = 5
+end

data/lib/link_thumbnailer.rb

@@ -0,0 +1,96 @@
+require 'link_thumbnailer/engine' if defined? Rails
+require 'link_thumbnailer/configuration'
+require 'link_thumbnailer/object'
+require 'link_thumbnailer/fetcher'
+require 'link_thumbnailer/doc_parser'
+require 'link_thumbnailer/doc'
+require 'link_thumbnailer/img_url_filter'
+require 'link_thumbnailer/img_parser'
+require 'link_thumbnailer/img_comparator'
+require 'link_thumbnailer/web_image'
+require 'link_thumbnailer/opengraph'
+require 'link_thumbnailer/version'
+
+module LinkThumbnailer
+
+  module Rails
+    autoload :Routes, 'link_thumbnailer/rails/routes'
+  end
+
+  class << self
+
+    attr_accessor :configuration, :object, :fetcher, :doc_parser,
+                  :img_url_filters, :img_parser
+
+    def config
+      self.configuration ||= Configuration.new(
+        mandatory_attributes: %w(url title images),
+        strict:               true,
+        redirect_limit:       3,
+        blacklist_urls:       [
+          %r{^http://ad\.doubleclick\.net/},
+          %r{^http://b\.scorecardresearch\.com/},
+          %r{^http://pixel\.quantserve\.com/},
+          %r{^http://s7\.addthis\.com/}
+        ],
+        rmagick_attributes:   %w(source_url mime_type colums rows filesize number_colors),
+        limit:                10,
+        top:                  5,
+        user_agent:           'linkthumbnailer',
+        verify_ssl:           true,
+        http_timeout:         5
+      )
+    end
+
+    def configure
+      yield config
+    end
+
+    def generate(url, options = {})
+      set_options(options)
+      instantiate_classes
+
+      doc = self.doc_parser.parse(self.fetcher.fetch(url), url)
+
+      self.object[:url] = self.fetcher.url.to_s
+      opengraph(doc) || custom(doc)
+    end
+
+    private
+
+    def set_options(options)
+      config
+      options.each {|k, v| config[k] = v }
+    end
+
+    def instantiate_classes
+      self.object           = LinkThumbnailer::Object.new
+      self.fetcher          = LinkThumbnailer::Fetcher.new
+      self.doc_parser       = LinkThumbnailer::DocParser.new
+      self.img_url_filters  = [LinkThumbnailer::ImgUrlFilter.new]
+      self.img_parser       = LinkThumbnailer::ImgParser.new(self.fetcher, self.img_url_filters)
+    end
+
+    def opengraph(doc)
+      return nil unless opengraph?(doc)
+      self.object = LinkThumbnailer::Opengraph.parse(self.object, doc)
+      return self.object if self.object.valid?
+      nil
+    end
+
+    def custom(doc)
+      self.object[:title]       = doc.title
+      self.object[:description] = doc.description
+      self.object[:images]      = self.img_parser.parse(doc.img_abs_urls.dup)
+      self.object[:url]         = doc.canonical_url || self.object[:url]
+      return self.object if self.object.valid?
+      nil
+    end
+
+    def opengraph?(doc)
+      !doc.xpath('//meta[starts-with(@property, "og:") and @content]').empty?
+    end
+
+  end
+
+end