image_downloader 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.rdoc +73 -10
 - data/lib/image_downloader.rb +1 -1
 - metadata +35 -35
 
    
        data/README.rdoc
    CHANGED
    
    | 
         @@ -1,19 +1,82 @@ 
     | 
|
| 
       1 
     | 
    
         
            -
            =  
     | 
| 
      
 1 
     | 
    
         
            +
            = Image Downloader
         
     | 
| 
       2 
2 
     | 
    
         | 
| 
       3 
     | 
    
         
            -
             
     | 
| 
      
 3 
     | 
    
         
            +
            Quite often there is a need to collect pictures from one or another page on the Internet. This plugin solves this particular task.
         
     | 
| 
       4 
4 
     | 
    
         | 
| 
       5 
     | 
    
         
            -
            ==  
     | 
| 
      
 5 
     | 
    
         
            +
            == Installation
         
     | 
| 
      
 6 
     | 
    
         
            +
             
     | 
| 
      
 7 
     | 
    
         
            +
            sudo gem install image_downloader
         
     | 
| 
      
 8 
     | 
    
         
            +
             
     | 
| 
      
 9 
     | 
    
         
            +
            == Requirements
         
     | 
| 
      
 10 
     | 
    
         
            +
             
     | 
| 
      
 11 
     | 
    
         
            +
            * ruby 1.8 or 1.9
         
     | 
| 
      
 12 
     | 
    
         
            +
            * gem nokogiri
         
     | 
| 
      
 13 
     | 
    
         
            +
             
     | 
| 
      
 14 
     | 
    
         
            +
            == Description
         
     | 
| 
      
 15 
     | 
    
         
            +
             
     | 
| 
      
 16 
     | 
    
         
            +
            Image Downloader is a rather simple library which does the following:
         
     | 
| 
      
 17 
     | 
    
         
            +
            * get web page (with Net::HTTP)
         
     | 
| 
      
 18 
     | 
    
         
            +
            * parse html page (use regexp or nokogiri)
         
     | 
| 
      
 19 
     | 
    
         
            +
            * download images (in one or multi-threads)
         
     | 
| 
      
 20 
     | 
    
         
            +
             
     | 
| 
      
 21 
     | 
    
         
            +
            == Example usage
         
     | 
| 
      
 22 
     | 
    
         
            +
            After installation, you can use the following code as an example:
         
     | 
| 
       6 
23 
     | 
    
         | 
| 
       7 
     | 
    
         
            -
             
     | 
| 
       8 
     | 
    
         
            -
             
     | 
| 
       9 
     | 
    
         
            -
             
     | 
| 
       10 
     | 
    
         
            -
             
     | 
| 
       11 
     | 
    
         
            -
             
     | 
| 
       12 
     | 
    
         
            -
             
     | 
| 
       13 
     | 
    
         
            -
             
     | 
| 
      
 24 
     | 
    
         
            +
              require 'rubygems'
         
     | 
| 
      
 25 
     | 
    
         
            +
              require 'image_downloader'
         
     | 
| 
      
 26 
     | 
    
         
            +
              
         
     | 
| 
      
 27 
     | 
    
         
            +
              page_url = 'www.test.com'
         
     | 
| 
      
 28 
     | 
    
         
            +
              target_path = 'img_dir/'
         
     | 
| 
      
 29 
     | 
    
         
            +
              downloader = ImageDownloader::Process.new(page_url,target_path)
         
     | 
| 
      
 30 
     | 
    
         
            +
             
     | 
| 
      
 31 
     | 
    
         
            +
              #####
         
     | 
| 
      
 32 
     | 
    
         
            +
              # download all images on page in any place (by regexp, all that look like url with image)
         
     | 
| 
      
 33 
     | 
    
         
            +
              downloader.parse(:any_looks_like_image => true)
         
     | 
| 
      
 34 
     | 
    
         
            +
             
     | 
| 
      
 35 
     | 
    
         
            +
              #####
         
     | 
| 
      
 36 
     | 
    
         
            +
              # download images from all elements where usually images placed (<img...>, <a...>, ...)
         
     | 
| 
      
 37 
     | 
    
         
            +
              downloader.parse()
         
     | 
| 
      
 38 
     | 
    
         
            +
             
     | 
| 
      
 39 
     | 
    
         
            +
              #####
         
     | 
| 
      
 40 
     | 
    
         
            +
              # download image from exect places in page
         
     | 
| 
      
 41 
     | 
    
         
            +
              downloader.parse(:collect => {:link_icon => true})
         
     | 
| 
      
 42 
     | 
    
         
            +
             
     | 
| 
      
 43 
     | 
    
         
            +
              downloader.download()
         
     | 
| 
      
 44 
     | 
    
         
            +
             
     | 
| 
      
 45 
     | 
    
         
            +
            For parse method available following options:
         
     | 
| 
      
 46 
     | 
    
         
            +
            :any_looks_like_image => true
         
     | 
| 
      
 47 
     | 
    
         
            +
            (find all url which contain image extansion)
         
     | 
| 
      
 48 
     | 
    
         
            +
             
     | 
| 
      
 49 
     | 
    
         
            +
            :collect => {:all => true, :(img_src|a_href|style_url|link_icon) => true}
         
     | 
| 
      
 50 
     | 
    
         
            +
            (find images in):
         
     | 
| 
      
 51 
     | 
    
         
            +
            * img_src - <img src="url">
         
     | 
| 
      
 52 
     | 
    
         
            +
            * a_href - <a href="url">
         
     | 
| 
      
 53 
     | 
    
         
            +
            * style_url - <element style="(background|background-image): url('url')">
         
     | 
| 
      
 54 
     | 
    
         
            +
            * link_icon - <link rel="shortcut icon" href="url">
         
     | 
| 
      
 55 
     | 
    
         
            +
             
     | 
| 
      
 56 
     | 
    
         
            +
            :ignore_without => {:(extension|image_extension) => true}
         
     | 
| 
      
 57 
     | 
    
         
            +
            (ignore URLs with images according to given parameters)
         
     | 
| 
      
 58 
     | 
    
         
            +
             
     | 
| 
      
 59 
     | 
    
         
            +
            == Executables
         
     | 
| 
      
 60 
     | 
    
         
            +
            You can simply use the executed shell commands:
         
     | 
| 
      
 61 
     | 
    
         
            +
             
     | 
| 
      
 62 
     | 
    
         
            +
            For any looks like image download
         
     | 
| 
      
 63 
     | 
    
         
            +
              download_any_images url dir/
         
     | 
| 
      
 64 
     | 
    
         
            +
             
     | 
| 
      
 65 
     | 
    
         
            +
            For download favicon only
         
     | 
| 
      
 66 
     | 
    
         
            +
              download_icon url dir/
         
     | 
| 
      
 67 
     | 
    
         
            +
             
     | 
| 
      
 68 
     | 
    
         
            +
            For download all, that is located in the places for pictures
         
     | 
| 
      
 69 
     | 
    
         
            +
              download_images url dir/
         
     | 
| 
       14 
70 
     | 
    
         | 
| 
       15 
71 
     | 
    
         
             
            == Copyright
         
     | 
| 
       16 
72 
     | 
    
         | 
| 
       17 
73 
     | 
    
         
             
            Copyright (c) 2011 Malykh Oleg. See LICENSE.txt for
         
     | 
| 
       18 
74 
     | 
    
         
             
            further details.
         
     | 
| 
       19 
75 
     | 
    
         | 
| 
      
 76 
     | 
    
         
            +
            == License
         
     | 
| 
      
 77 
     | 
    
         
            +
             
     | 
| 
      
 78 
     | 
    
         
            +
            The MIT License
         
     | 
| 
      
 79 
     | 
    
         
            +
             
     | 
| 
      
 80 
     | 
    
         
            +
            == Authors
         
     | 
| 
      
 81 
     | 
    
         
            +
             
     | 
| 
      
 82 
     | 
    
         
            +
            Personal blog author: {Malykh Oleg}[http://malyh.com/] - blog in russian
         
     | 
    
        data/lib/image_downloader.rb
    CHANGED
    
    | 
         @@ -36,7 +36,7 @@ module ImageDownloader 
     | 
|
| 
       36 
36 
     | 
    
         
             
                # :any_looks_like_image => true
         
     | 
| 
       37 
37 
     | 
    
         
             
                # :ignore_without => {:(extension|image_extension) => true}
         
     | 
| 
       38 
38 
     | 
    
         
             
                # Nokogiri gem is required:
         
     | 
| 
       39 
     | 
    
         
            -
                # :collect => {:all => true, :(img_src|a_href|style_url|link_icon) => true} 
     | 
| 
      
 39 
     | 
    
         
            +
                # :collect => {:all => true, :(img_src|a_href|style_url|link_icon) => true}
         
     | 
| 
       40 
40 
     | 
    
         
             
                def parse(h={:collect => {}, :ignore_without => {}})
         
     | 
| 
       41 
41 
     | 
    
         
             
                  self.rebuild_collect_hash(h)
         
     | 
| 
       42 
42 
     | 
    
         | 
    
        metadata
    CHANGED
    
    | 
         @@ -1,13 +1,13 @@ 
     | 
|
| 
       1 
1 
     | 
    
         
             
            --- !ruby/object:Gem::Specification 
         
     | 
| 
       2 
2 
     | 
    
         
             
            name: image_downloader
         
     | 
| 
       3 
3 
     | 
    
         
             
            version: !ruby/object:Gem::Version 
         
     | 
| 
       4 
     | 
    
         
            -
              hash:  
     | 
| 
      
 4 
     | 
    
         
            +
              hash: 23
         
     | 
| 
       5 
5 
     | 
    
         
             
              prerelease: false
         
     | 
| 
       6 
6 
     | 
    
         
             
              segments: 
         
     | 
| 
       7 
7 
     | 
    
         
             
              - 0
         
     | 
| 
       8 
     | 
    
         
            -
              -  
     | 
| 
      
 8 
     | 
    
         
            +
              - 2
         
     | 
| 
       9 
9 
     | 
    
         
             
              - 0
         
     | 
| 
       10 
     | 
    
         
            -
              version: 0. 
     | 
| 
      
 10 
     | 
    
         
            +
              version: 0.2.0
         
     | 
| 
       11 
11 
     | 
    
         
             
            platform: ruby
         
     | 
| 
       12 
12 
     | 
    
         
             
            authors: 
         
     | 
| 
       13 
13 
     | 
    
         
             
            - Malykh Oleg
         
     | 
| 
         @@ -19,10 +19,25 @@ date: 2011-07-15 00:00:00 +04:00 
     | 
|
| 
       19 
19 
     | 
    
         
             
            default_executable: 
         
     | 
| 
       20 
20 
     | 
    
         
             
            dependencies: 
         
     | 
| 
       21 
21 
     | 
    
         
             
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       22 
     | 
    
         
            -
               
     | 
| 
      
 22 
     | 
    
         
            +
              name: nokogiri
         
     | 
| 
       23 
23 
     | 
    
         
             
              prerelease: false
         
     | 
| 
      
 24 
     | 
    
         
            +
              requirement: &id001 !ruby/object:Gem::Requirement 
         
     | 
| 
      
 25 
     | 
    
         
            +
                none: false
         
     | 
| 
      
 26 
     | 
    
         
            +
                requirements: 
         
     | 
| 
      
 27 
     | 
    
         
            +
                - - ">="
         
     | 
| 
      
 28 
     | 
    
         
            +
                  - !ruby/object:Gem::Version 
         
     | 
| 
      
 29 
     | 
    
         
            +
                    hash: 15
         
     | 
| 
      
 30 
     | 
    
         
            +
                    segments: 
         
     | 
| 
      
 31 
     | 
    
         
            +
                    - 1
         
     | 
| 
      
 32 
     | 
    
         
            +
                    - 4
         
     | 
| 
      
 33 
     | 
    
         
            +
                    - 4
         
     | 
| 
      
 34 
     | 
    
         
            +
                    version: 1.4.4
         
     | 
| 
      
 35 
     | 
    
         
            +
              type: :runtime
         
     | 
| 
      
 36 
     | 
    
         
            +
              version_requirements: *id001
         
     | 
| 
      
 37 
     | 
    
         
            +
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       24 
38 
     | 
    
         
             
              name: shoulda
         
     | 
| 
       25 
     | 
    
         
            -
               
     | 
| 
      
 39 
     | 
    
         
            +
              prerelease: false
         
     | 
| 
      
 40 
     | 
    
         
            +
              requirement: &id002 !ruby/object:Gem::Requirement 
         
     | 
| 
       26 
41 
     | 
    
         
             
                none: false
         
     | 
| 
       27 
42 
     | 
    
         
             
                requirements: 
         
     | 
| 
       28 
43 
     | 
    
         
             
                - - ">="
         
     | 
| 
         @@ -31,12 +46,12 @@ dependencies: 
     | 
|
| 
       31 
46 
     | 
    
         
             
                    segments: 
         
     | 
| 
       32 
47 
     | 
    
         
             
                    - 0
         
     | 
| 
       33 
48 
     | 
    
         
             
                    version: "0"
         
     | 
| 
       34 
     | 
    
         
            -
              requirement: *id001
         
     | 
| 
       35 
     | 
    
         
            -
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       36 
49 
     | 
    
         
             
              type: :development
         
     | 
| 
       37 
     | 
    
         
            -
               
     | 
| 
      
 50 
     | 
    
         
            +
              version_requirements: *id002
         
     | 
| 
      
 51 
     | 
    
         
            +
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       38 
52 
     | 
    
         
             
              name: bundler
         
     | 
| 
       39 
     | 
    
         
            -
               
     | 
| 
      
 53 
     | 
    
         
            +
              prerelease: false
         
     | 
| 
      
 54 
     | 
    
         
            +
              requirement: &id003 !ruby/object:Gem::Requirement 
         
     | 
| 
       40 
55 
     | 
    
         
             
                none: false
         
     | 
| 
       41 
56 
     | 
    
         
             
                requirements: 
         
     | 
| 
       42 
57 
     | 
    
         
             
                - - ~>
         
     | 
| 
         @@ -47,12 +62,12 @@ dependencies: 
     | 
|
| 
       47 
62 
     | 
    
         
             
                    - 0
         
     | 
| 
       48 
63 
     | 
    
         
             
                    - 0
         
     | 
| 
       49 
64 
     | 
    
         
             
                    version: 1.0.0
         
     | 
| 
       50 
     | 
    
         
            -
              requirement: *id002
         
     | 
| 
       51 
     | 
    
         
            -
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       52 
65 
     | 
    
         
             
              type: :development
         
     | 
| 
       53 
     | 
    
         
            -
               
     | 
| 
      
 66 
     | 
    
         
            +
              version_requirements: *id003
         
     | 
| 
      
 67 
     | 
    
         
            +
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       54 
68 
     | 
    
         
             
              name: jeweler
         
     | 
| 
       55 
     | 
    
         
            -
               
     | 
| 
      
 69 
     | 
    
         
            +
              prerelease: false
         
     | 
| 
      
 70 
     | 
    
         
            +
              requirement: &id004 !ruby/object:Gem::Requirement 
         
     | 
| 
       56 
71 
     | 
    
         
             
                none: false
         
     | 
| 
       57 
72 
     | 
    
         
             
                requirements: 
         
     | 
| 
       58 
73 
     | 
    
         
             
                - - ~>
         
     | 
| 
         @@ -63,12 +78,12 @@ dependencies: 
     | 
|
| 
       63 
78 
     | 
    
         
             
                    - 6
         
     | 
| 
       64 
79 
     | 
    
         
             
                    - 4
         
     | 
| 
       65 
80 
     | 
    
         
             
                    version: 1.6.4
         
     | 
| 
       66 
     | 
    
         
            -
              requirement: *id003
         
     | 
| 
       67 
     | 
    
         
            -
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       68 
81 
     | 
    
         
             
              type: :development
         
     | 
| 
       69 
     | 
    
         
            -
               
     | 
| 
      
 82 
     | 
    
         
            +
              version_requirements: *id004
         
     | 
| 
      
 83 
     | 
    
         
            +
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       70 
84 
     | 
    
         
             
              name: rcov
         
     | 
| 
       71 
     | 
    
         
            -
               
     | 
| 
      
 85 
     | 
    
         
            +
              prerelease: false
         
     | 
| 
      
 86 
     | 
    
         
            +
              requirement: &id005 !ruby/object:Gem::Requirement 
         
     | 
| 
       72 
87 
     | 
    
         
             
                none: false
         
     | 
| 
       73 
88 
     | 
    
         
             
                requirements: 
         
     | 
| 
       74 
89 
     | 
    
         
             
                - - ">="
         
     | 
| 
         @@ -77,24 +92,9 @@ dependencies: 
     | 
|
| 
       77 
92 
     | 
    
         
             
                    segments: 
         
     | 
| 
       78 
93 
     | 
    
         
             
                    - 0
         
     | 
| 
       79 
94 
     | 
    
         
             
                    version: "0"
         
     | 
| 
       80 
     | 
    
         
            -
              requirement: *id004
         
     | 
| 
       81 
     | 
    
         
            -
            - !ruby/object:Gem::Dependency 
         
     | 
| 
       82 
95 
     | 
    
         
             
              type: :development
         
     | 
| 
       83 
     | 
    
         
            -
               
     | 
| 
       84 
     | 
    
         
            -
             
     | 
| 
       85 
     | 
    
         
            -
              version_requirements: &id005 !ruby/object:Gem::Requirement 
         
     | 
| 
       86 
     | 
    
         
            -
                none: false
         
     | 
| 
       87 
     | 
    
         
            -
                requirements: 
         
     | 
| 
       88 
     | 
    
         
            -
                - - ">="
         
     | 
| 
       89 
     | 
    
         
            -
                  - !ruby/object:Gem::Version 
         
     | 
| 
       90 
     | 
    
         
            -
                    hash: 15
         
     | 
| 
       91 
     | 
    
         
            -
                    segments: 
         
     | 
| 
       92 
     | 
    
         
            -
                    - 1
         
     | 
| 
       93 
     | 
    
         
            -
                    - 4
         
     | 
| 
       94 
     | 
    
         
            -
                    - 4
         
     | 
| 
       95 
     | 
    
         
            -
                    version: 1.4.4
         
     | 
| 
       96 
     | 
    
         
            -
              requirement: *id005
         
     | 
| 
       97 
     | 
    
         
            -
            description: Detailed description for image-downloader
         
     | 
| 
      
 96 
     | 
    
         
            +
              version_requirements: *id005
         
     | 
| 
      
 97 
     | 
    
         
            +
            description: A simple lib for downloading pictures from web pages. It can get and parse page with different options and receive images from specified locations. It is possible to download images simultaneously in multiple threads or sequentially.
         
     | 
| 
       98 
98 
     | 
    
         
             
            email: malykholeg@gmail.com
         
     | 
| 
       99 
99 
     | 
    
         
             
            executables: 
         
     | 
| 
       100 
100 
     | 
    
         
             
            - download_any_images
         
     | 
| 
         @@ -150,6 +150,6 @@ rubyforge_project: 
     | 
|
| 
       150 
150 
     | 
    
         
             
            rubygems_version: 1.3.7
         
     | 
| 
       151 
151 
     | 
    
         
             
            signing_key: 
         
     | 
| 
       152 
152 
     | 
    
         
             
            specification_version: 3
         
     | 
| 
       153 
     | 
    
         
            -
            summary:  
     | 
| 
      
 153 
     | 
    
         
            +
            summary: Lib for parsing web pages to find images in specified locations and downloading them simultaneously or sequentially
         
     | 
| 
       154 
154 
     | 
    
         
             
            test_files: []
         
     | 
| 
       155 
155 
     | 
    
         |