reaver 0.16.1 → 0.17.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6f52f3ace18bfc32c02c16d50ee0e101d7ce06cdd7c721d34607d7b7d6fece7d
-  data.tar.gz: e827fa9f596d6a71f94a9176050f10047d1378fd1d02cb11aebe7d3a054bf9f2
+  metadata.gz: 149dbd0eae5bd1597a5e66059cc765c25f9f63d05d9bfdf2d7ec6484564b7bbb
+  data.tar.gz: 30e91e2d44090b1f0be9383eba41b25f854c453622de4173a910da6c12fbecf0
 SHA512:
-  metadata.gz: c854a37bc316b66a7a57630fb28d70151482fa7c099f9b5327a9c7ff8a651a472b273f266574f7d9eb575a2a07014b4fc669caecf7ddd63dd9c20f1f906b2ade
-  data.tar.gz: 8f735f0936df5ea9042be023e1ec68e755980b7564e79baeaac7e921866934e09227b908c614a7b1dd482065ab656d773ce2537ec5f1ffdc77d3b8a834666328
+  metadata.gz: 34db056ae50024e6358bf14edab543a80466485ffcc5896bc17bd462ba5b33658d50adc366e328c4f43fdb99a463423753e2e49042ca965e77dd38666eec71b1
+  data.tar.gz: 0c5147410254e9019022093668db7347c3d8c41c02c94d93ddb825ccedd6a6390674aa275c1881986ee0f8f75378af056f0b8d44348aa45fbfad3dbcd2388edf

data/CHANGELOG.md

@@ -1,48 +1,60 @@
+## 0.17.0 - Oct. 2025
+
+- Random user-agent on each download
+- Code improvement with Rubocop
+
 ### Fixes
-+ Skip in not things or things[] is nil
+
+- Skip in not things or things[] is nil
 
 ## 0.16.0, release 01/2025
-+ Reaver stop rewriting our yaml collections and erase indentation...
-+ Change the user-agent used for a more generic `Mozilla/5.0 (Windows NT 10.0; WOW64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.6556.192 Safari/537.36`
-+ Can add `keep_name = true` per file, not only globally.
+
+- Reaver stop rewriting our yaml collections and erase indentation...
+- Change the user-agent used for a more generic `Mozilla/5.0 (Windows NT 10.0; WOW64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.6556.192 Safari/537.36`
+- Can add `keep_name = true` per file, not only globally.
 
 ## 0.14.0, release 11/2024
+
 - Can `git clone` && `git pull` if use `things[].git: true`
 
 ## 0.12.0
+
 - Add `--strip-components=1` to tar, by default set to `1` may be changed by
   `things[].strip_components: <number>` variable
 
 ## 0.11.0, release 10/2024
+
 - can extract tar.xz archive.
 
 ## 0.10.1
+
 - For `all_to_dir_path` or `things[].dest_dir`, all paths are relative to `$HOME` (or
   /home/<name>), so don't include shell variable `~` or `$HOME`.
 - unzip not interactive with `-o`.
 
 ## 0.10.0, release 10/2024
+
 - New variable for collections, all_into_dir, things.dest_dir, keep_name and
   force_download.
--** Use [Marcel](https://github.com/rails/marcel) to detect file mime.
+- Use [Marcel](https://github.com/rails/marcel) to detect file mime.
 - Can decompress archive `zip` or `tar.gz (gzip)` using unzip and tar from `unix`.
 - Cam move files to specific directory.
 
 ## 0.7.0, release 12/24/22
 
-* Each collections download things in separate directory.
-* Add a `changed` boolean if a file change.
-* Stop modifying collection content, add them in a file called `metadata.yml`.
-* Improve runtime.
+- Each collections download things in separate directory.
+- Add a `changed` boolean if a file change.
+- Stop modifying collection content, add them in a file called `metadata.yml`.
+- Improve runtime.
 
 ## 0.4.0, release 12/22/22
 
-* Renaming project.
-* Add a banner.
-* Separate main.
-* Add collection yml.
-* Download stack.
+- Renaming project.
+- Add a banner.
+- Separate main.
+- Add collection yml.
+- Download stack.
 
 ## 0.0.1, release 12/21/22
-* Initial push, code freeying !
 
+- Initial push, code released!

data/README.md

@@ -1,4 +1,4 @@
-# reaver
+# Reaver
 
 [![Gem
 Version](https://badge.fury.io/rb/reaver.svg)](https://badge.fury.io/rb/reaver)
@@ -24,7 +24,7 @@ For example, your need to use Vim, vim require a lots of external plugins to be
 
     $ $EDITOR ~/.config/reaver/vim.yml
 
-In the file, we add all archives required by u projects
+In the file, we add all archives required by your projects.
 
 ```yml
 ---
@@ -42,28 +42,32 @@ all_into_dir: .config/pack/myvimpluggins/start
 keep_name: true
 ```
 
-To see more examples, go
-[here](https://github.com/szorfein/dots/tree/ansible/home/private_dot_config/reaver)
+To see more examples, look my [dotfiles repository](https://github.com/szorfein/dotfiles), each directory contain a list of tasks for Reaver.
+
+## Launch
+
+After declaring your need, start Reaver simply with:
+
+    $ reaver
+
+Reaver download all files in `~/.cache/reaver` by default.
+
+### Few rules
 
 A collection can include:
+
 - `all_into_dir: <dirname>` if all files go in a directory. Directory is created
   if not exist.
-- `keep_name: <boolean>`, if `true`, create a directory with the name of the thing, e.g, for a name `ombre.tar.gz`, the final dest will be `all_into_dir/ombre` or `dest_dir/ombre`.
+- `keep_name: <boolean>`, if `true`, create a directory with the name of the thing, e.g, for a name `ombre.tar.gz`, the final destination will be `all_into_dir/ombre` or `dest_dir/ombre`.
 - `force_download: <boolean>`, if you make change and want to download now, change to `true`.
-- `things[].dest_dir: <dirname>`, if each files go in differents directory, use this.
+- `things[].dest_dir: <dirname>`, if each files go in different directory, use this.
 - `things[].name` the new name after download, may include the file extension.
 - `things[].strip_components: <number>`, used on `tar`, default to 1, skip the first
   directory from an archive, if no subdirectory exist, you should set to 0.
 - `things[].url: <string>`
 - `things[].git: <boolean>`, If the thing need to be managed with git.
-- `time: 86000` (in second) is for search every day ( 60 * 60 * 24 ).
+- `time: 86000` (in second) is for search every day ( 60 _ 60 _ 24 ).
 
 If `all_into_dir` is defined, `things[].dest_dir` is not used.  
 All paths given are relative to `$HOME` so don't include `~` or any shell
 variables.
-
-And start reaver simply with:
-
-    $ reaver
-
-Reaver download all files in `~/.cache/reaver` by default.

data/lib/reaver/banner.rb

@@ -1,14 +1,16 @@
 # frozen_string_literal: true
 
+# Reaver.banner
 module Reaver
   def self.banner
     # Generated on https://www.coolgenerator.com/ascii-text-generator
     # font > ASCII New Roman
-    puts '
-    ____, ____, ____,__  _,____,____, 
-  (-|__)(-|_, (-/_|(-\  /(-|_,(-|__) 
-   _|  \,_|__,_/  |, _\/  _|__,_|  \,
-   (     (    (      (    (    (    
-    '
+    <<~BANNER
+       ____, ____, ____,__  _,____,____,
+      (-|__)(-|_, (-/_|(-\\  /(-|_,(-|__)
+       _|  \\,_|__,_/  |, _\\/  _|__,_|  \\,
+      (     (    (      (    (    (
+
+    BANNER
   end
 end

data/lib/reaver/collection.rb

@@ -21,7 +21,7 @@ module Reaver
                else
                  @tasks = YAML.load_file(@file)
                end
-    rescue => e
+    rescue StandardError => e
       raise e, "loading YAML fail for #{@file}: #{e.message}"
     end
 

data/lib/reaver/download.rb

@@ -5,36 +5,48 @@ require 'net/http'
 require 'tempfile'
 require 'fileutils'
 
+# Download link
 module Reaver
-  extend self
+  module_function
 
   def download(url, name)
     dest = name
     url = URI(url)
-    raise Error, 'url was invalid' if !url.respond_to?(:open)
+    raise Error, 'url was invalid' unless url.respond_to?(:open)
 
     options = {}
-    options['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; WOW64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.6556.192 Safari/537.36'
 
-    downloaded_file = ''
     Whirly.start do
       Whirly.status = "downloading #{dest}"
+      options['User-Agent'] = agent_list
       downloaded_file = URI.open(url, options)
 
-      # open-uri will return a StringIO instead of a Tempfile if the filesize
-      # is less than 10 KB, so we patch this behaviour by converting it into
-      # a Tempfile.
-      if downloaded_file.is_a?(StringIO)
-        tempfile = Tempfile.new('open-uri', binmode: true)
-        IO.copy_stream(downloaded_file, tempfile.path)
-        downloaded_file = tempfile
-        FileUtils.mv downloaded_file.path, dest
-      else
-        IO.copy_stream(downloaded_file, dest)
-      end
-
-      downloaded_file
+      get_the_link(downloaded_file, dest)
     end
   end
-end
 
+  # List from https://www.useragents.me/
+  # return a random user-agent with sample
+  def agent_list
+    ag = ['Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.10 Safari/605.1.1',
+          'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.3',
+          'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.3',
+          'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.3',
+          'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/134.0.0.0 Safari/537.36 Trailer/93.3.8652.5']
+    ag.sample
+  end
+
+  def get_the_link(file, dest)
+    # open-uri will return a StringIO instead of a Tempfile if the filesize
+    # is less than 10 KB, so we patch this behaviour by converting it into
+    # a Tempfile.
+    if file.is_a?(StringIO)
+      tempfile = Tempfile.new('open-uri', binmode: true)
+      IO.copy_stream(file, tempfile.path)
+      FileUtils.mv file.path, dest
+    else
+      IO.copy_stream(file, dest)
+    end
+    file
+  end
+end

data/lib/reaver/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Reaver
-  VERSION = '0.16.1'
+  VERSION = '0.17.0'
 end

data/lib/reaver/walk.rb

@@ -18,16 +18,11 @@ module Reaver
 
     def x
       case @extension
-      when %r{^image/(jpeg|png)}
-        copy_file
-      when %r{^application/zip}
-        extract_zip
-      when %r{^application/(gzip|x-xz)}
-        extract_gzip
-      when %r{^font/ttf}
-        copy_file
-      when %r{^application/(x-elf|x-sh)}
-        copy_file_with_x
+      when %r{^image/(jpeg|png)} then copy_file
+      when %r{^application/zip} then extract_zip
+      when %r{^application/(gzip|x-xz)} then extract_gzip
+      when %r{^font/ttf} then copy_file
+      when %r{^application/(x-elf|x-sh)} then copy_file_with_x
       else
         puts "Filetype #{@extension} not yet supported, skipping..."
       end
@@ -60,7 +55,7 @@ module Reaver
 
     def copy_file_with_x
       copy_file
-      File.chmod 0700, "#{@final_dest}/#{@filename}"
+      File.chmod 0o700, "#{@final_dest}/#{@filename}"
     end
 
     def extract_zip

data/lib/reaver.rb

@@ -11,58 +11,70 @@ require_relative 'reaver/git'
 require 'whirly'
 require 'fileutils'
 
+# Main class
 module Reaver
+  module_function
+
   # Where downloads things
   CACHE_DIR = "#{ENV['HOME']}/.cache/reaver"
 
   # Search collection paths
-  if ENV['XDG_CONFIG_HOME']
-    WORKDIR = "#{ENV['XDG_CONFIG_HOME']}/reaver"
-  else
-    WORKDIR = "#{ENV['HOME']}/.config/reaver"
-  end
+  WORKDIR = ENV['XDG_CONFIG_HOME'] ? "#{ENV['XDG_CONFIG_HOME']}/reaver" : "#{ENV['HOME']}/.config/reaver"
 
   # Configure Whirly
   Whirly.configure spinner: 'bouncingBar',
                    color: true,
                    ambiguous_characters_width: 1
 
-  def self.main
-    FileUtils.mkdir_p(WORKDIR)
-
-    #puts ">> Search collections in #{WORKDIR}"
+  FileUtils.mkdir_p(WORKDIR)
 
+  def main
     Dir.glob("#{WORKDIR}/*.yml").each do |f|
-      name = f.split('/').last
-      name = name.split('.').first
-      workdir = "#{CACHE_DIR}/#{name}"
-
-      FileUtils.mkdir_p(workdir)
-
-      collection = Collection.new(f)
-      collection.load_yaml
-
-      #puts collection.tasks
+      workdir = collection_name(f)
+      collection = load_collection(f)
 
       next unless collection.tasks
 
-      metadata = MetaData.new(workdir, collection)
-      metadata.load_yaml
-      next_download = metadata.info['next']
-      force_download = collection.tasks['force_download'] || false
-      #puts "should we force #{force_download}"
-
-      if next_download < Time.new || force_download
-          #puts ' >> Download time for ' + name
-        FileUtils.chdir(workdir)
-        #puts "  > chdir #{workdir}"
-        collection.launch(metadata)
-        #collection.save_yaml if force_download
-      else
-        puts " > Next download > #{next_download}"
+      metadata = load_metadata(workdir, collection)
+      if analyze_collection(metadata.info['next'], collection.tasks['force_download'])
+        time_to_download(workdir, collection, metadata)
       end
 
       metadata.save_yaml
     end
   end
+
+  def load_collection(pathname)
+    collection = Collection.new(pathname)
+    collection.load_yaml
+    collection
+  end
+
+  def load_metadata(workdir, collection)
+    FileUtils.mkdir_p(workdir)
+    md = MetaData.new(workdir, collection)
+    md.load_yaml
+    md
+  end
+
+  def collection_name(filename)
+    name = filename.split('/').last
+    name = name.split('.').first
+    "#{CACHE_DIR}/#{name}"
+  end
+
+  def analyze_collection(info_next, forced = nil)
+    next_download = info_next
+    force_download = forced || false
+    return true if next_download < Time.new || force_download
+
+    puts " > Next download > #{next_download}"
+    false
+  end
+
+  def time_to_download(workdir, collection, metadata)
+    FileUtils.chdir(workdir)
+    collection.launch(metadata)
+    # collection.save_yaml if force_download
+  end
 end

data/reaver.gemspec

@@ -43,7 +43,7 @@ Gem::Specification.new do |s|
 
   # git action need >=2.6
   s.required_ruby_version = '>=2.6'
-  s.add_dependency 'marcel', '~> 1.0'
+  s.add_dependency 'marcel', '~> 1.1'
   s.add_dependency 'paint', '~> 2.3'
   s.add_dependency 'whirly', '~> 0.3'
 end

metadata

@@ -1,18 +1,17 @@
 --- !ruby/object:Gem::Specification
 name: reaver
 version: !ruby/object:Gem::Version
-  version: 0.16.1
+  version: 0.17.0
 platform: ruby
 authors:
 - szorfein
-autorequire:
 bindir: bin
 cert_chain:
 - |
   -----BEGIN CERTIFICATE-----
-  MIIEPDCCAqSgAwIBAgIBAjANBgkqhkiG9w0BAQsFADBEMREwDwYDVQQDDAhzem9y
+  MIIEPDCCAqSgAwIBAgIBAzANBgkqhkiG9w0BAQsFADBEMREwDwYDVQQDDAhzem9y
   ZmVpbjEaMBgGCgmSJomT8ixkARkWCnByb3Rvbm1haWwxEzARBgoJkiaJk/IsZAEZ
-  FgNjb20wHhcNMjQxMDI1MTU0NTI4WhcNMjUxMDI1MTU0NTI4WjBEMREwDwYDVQQD
+  FgNjb20wHhcNMjUxMDI2MTQyNTU2WhcNMjYxMDI2MTQyNTU2WjBEMREwDwYDVQQD
   DAhzem9yZmVpbjEaMBgGCgmSJomT8ixkARkWCnByb3Rvbm1haWwxEzARBgoJkiaJ
   k/IsZAEZFgNjb20wggGiMA0GCSqGSIb3DQEBAQUAA4IBjwAwggGKAoIBgQCqe1yx
   EG2oM25jeHp08A8zkaDNmbI3MujjrRM/WPEYZX2dVwOxkIS20hQVuxcAsBBA4W/W
@@ -24,17 +23,17 @@ cert_chain:
   OQBB/XF1C80imZtRtdUqh6bK9WeWI4RYZ2/KwXL1AScEbXkBkkOECWoVrD18WgRm
   siuX6RkNIelhtb0En7f3bizgPqlO0qPQV+wPi9TSBxdVG12C0OmjCQYMQD0CAwEA
   AaM5MDcwCQYDVR0TBAIwADALBgNVHQ8EBAMCBLAwHQYDVR0OBBYEFOUqdDeoxQXP
-  J29lorB0/52ePa5qMA0GCSqGSIb3DQEBCwUAA4IBgQAg+ig1ZbxY3KlXD5pfn7HV
-  tl7TIkkTznhCW33HZrswRQnQ5nC/w/MClasCCDjEL6tzE4dTs/Hn/DBBFPzb0WWI
-  sUYnBaf0k4uF5mEMUXjjAsNa6P+w35VJBHSPLgsBrVX/5H9hnptw5qquxDFuSvQ3
-  MeUCnhe8+YFSQCigJRA1h3b2XMQEw7KjdJAfYx2qd7zQgfqtKp46E3Jndk7PHpfQ
-  826uy2+rqHZJtRPfh/SPUIyMjxDpW7vZd1J8uTFgVoNn8OdFv5FKsjCEP9VfxgN7
-  hO51TYmCd7cvnGrwfx6j/AqA6N2nHZNojwQ98nfXWp5tdjET+gipjZwYGmM3+7o/
-  /WIxUcce8RaxYQ9kfz9zrNKeWqlviQ6dc3tnn3Ppu/27YjyC/tqodic1pPE6US9y
-  1Eu31LqT9K75BZgLoHDybDlvVJ1B2hZI31jZOFjqpCgVxwoHmuigy2iCUcLsCTJJ
-  6ct0vZN7gCQ/YSLa3jJf1N/CH4rQokbof1fehel4SPA=
+  J29lorB0/52ePa5qMA0GCSqGSIb3DQEBCwUAA4IBgQAW22kVqz92sKW7thsKrW2i
+  BwVahsopgVLIUqp7sT+nL6Jw1QWCxolDwSToWFo0XuwbNdqCuGRBM+IzhWmCcA0e
+  WoHAPGQAhKz5jCWJ9GwgWbttYNrZ/PkyzKlT+15itMVeEEKxcqvfVFkdTXvTtliP
+  h75zR66tpFPjOejy+YD0uSPWbR/Nskf4BaMFso7mEr+5nOfzj0+Emy3MikHMm7uo
+  qZEazJO4XlmQWI2O7skt/SkE6xiZDQeCADZ4NXxBNzKkxKpozBXSeBKNUi21y9zR
+  3PGUOg6fCZt5H3kSffrvtycOnAu/FRNrQUxKRHLJKez8d7zJH64sBdldm5fZJu1O
+  pzUhdmhxa/brsIYFPPsjw0PminqSEmFahOoGyph8lw1IYu6UskIlB+J1BLZxsPkJ
+  6+fmQQAYvIhUQH+pOPELkGGo+bQMpCLb4i97kyzbOjJqAFKsgqgCOtvUC4yUZRNb
+  YulAM0cmmcfbzWUxsyAoCp9mY227qQZpGv6dDgv4IbA=
   -----END CERTIFICATE-----
-date: 2025-02-14 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: marcel
@@ -42,14 +41,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '1.1'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.0'
+        version: '1.1'
 - !ruby/object:Gem::Dependency
   name: paint
   requirement: !ruby/object:Gem::Requirement
@@ -110,7 +109,6 @@ metadata:
   source_code_uri: https://github.com/szorfein/reaver
   wiki_uri: https://github.com/szorfein/reaver/wiki
   funding_uri: https://patreon.com/szorfein
-post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -125,8 +123,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.5.16
-signing_key:
+rubygems_version: 3.6.9
 specification_version: 4
 summary: A tool to downloads and track updates of things on the Net.
 test_files: []