storazzo 0.4.2 → 0.5.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (41) hide show
  1. checksums.yaml +4 -4
  2. data/Gemfile +7 -4
  3. data/Makefile +20 -1
  4. data/README.md +23 -0
  5. data/Rakefile +35 -8
  6. data/VERSION +1 -1
  7. data/bin/hello-storazzo +5 -0
  8. data/bin/ricdisk-magic +50 -57
  9. data/bin/stats-with-md5 +268 -297
  10. data/lib/storazzo/colors.rb +43 -45
  11. data/lib/storazzo/common.rb +80 -17
  12. data/lib/storazzo/debug.rb +5 -8
  13. data/lib/storazzo/hashify.rb +44 -43
  14. data/lib/storazzo/main.rb +44 -40
  15. data/lib/storazzo/media/abstract_ric_disk.rb +163 -104
  16. data/lib/storazzo/media/gcs_bucket.rb +51 -15
  17. data/lib/storazzo/media/local_folder.rb +43 -46
  18. data/lib/storazzo/media/mount_point.rb +12 -2
  19. data/lib/storazzo/ric_disk.rb +223 -251
  20. data/lib/storazzo/ric_disk_config.rb +230 -193
  21. data/lib/storazzo/ric_disk_sample_config.rb +12 -16
  22. data/lib/storazzo/ric_disk_statsfile.rb +17 -16
  23. data/lib/storazzo/ric_disk_ugly.rb +35 -38
  24. data/lib/storazzo/version.rb +7 -7
  25. data/lib/storazzo.rb +34 -29
  26. data/storazzo.gemspec +22 -20
  27. data/test/media/test_abstract_ric_disk.rb +19 -0
  28. data/test/media/test_gcs_bucket.rb +58 -0
  29. data/test/media/test_local_folder.rb +145 -0
  30. data/test/media/test_mount_point.rb +25 -0
  31. data/test/test_ric_disk.rb +16 -0
  32. data/test/test_ric_disk_config.rb +20 -29
  33. data/test/test_ric_disk_stats_file.rb +13 -14
  34. data/test/test_storazzo.rb +26 -26
  35. data/var/dumps/file_stat.linux.yaml +15 -0
  36. data/var/dumps/file_stat.macosx.yaml +15 -0
  37. data/var/test/disks/disk02-full/Rakefile +13 -0
  38. data/var/test/disks/ricdisk_stats_v11.rds +11 -0
  39. metadata +38 -10
  40. data/test/test_gcs_bucket.rb +0 -70
  41. data/test/test_local_folder.rb +0 -121
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 6a489f7c8e545a123a224850369894e939e8bc541ca2b00ce4f2306493144ec2
4
- data.tar.gz: 35f76e4a92849cb0547910ac8f240b38dda4cecaf70ceec0b6d665d6d5d3ce5f
3
+ metadata.gz: aa13f66b7bfcdf56a065b666c4a256be432bfa8dab74e39eca9b3d6029a7a4a9
4
+ data.tar.gz: 4efc1c0086cf9a9ad710e47d1ee9e0559d0d3597a79ca21381687f15d0abe163
5
5
  SHA512:
6
- metadata.gz: 3d69dbf50b340e06833ba28cf4219ee6aca33b5466e442377597c825ae0c2adaef2d8be3b4330adee8dde52d693e84f3f69c5208df767e988268f45359e36f27
7
- data.tar.gz: ef9ce00d9af13c84aa2db03996ea383af7494007a2bdd94f799d4720d2cc2aae63106e7bbf74db555c6574180f3948305712d115f757f07a196d96958da74ef5
6
+ metadata.gz: 02fa0fcdbad9284bb5c26dd1fd284c0c32aee90c317148c981cb8c983346f4e4e093d9a941145a6717f27df23681566c9435e0f5a9a237d3b5e3e5e221a35d44
7
+ data.tar.gz: 43825ffe96f30ec8e9581f7ef25fc08d36e9636d0cf013bfec7611727d19dd699920a12cf068e516f9eaef78826984310072977af01f740d28b6810dd44cd734
data/Gemfile CHANGED
@@ -1,7 +1,10 @@
1
- # useless now
2
-
1
+ source "https://rubygems.org"
3
2
 
4
3
  # to troubleshoot/debug things
5
- #gem 'rubocop'
4
+ gem 'rubocop'
5
+ gem "ruby-lsp", "~> 0.0.4", :group => :development
6
6
  gem 'rake'
7
- gem 'pry'
7
+ # gem 'pry'
8
+ gem 'pry', :group => :development
9
+ # gem 'require_all' # yup I'm lazy: https://stackoverflow.com/questions/735073/best-way-to-require-all-files-from-a-directory-in-ruby
10
+ # gem 'after_do' # To allow AFTER calling instance to do LOAD() automagically. Naah I can do without :)
data/Makefile CHANGED
@@ -51,4 +51,23 @@ watch-test:
51
51
  watch -c make test
52
52
 
53
53
  test-gcs-bucket:
54
- ruby -I test test/test_gcs_bucket.rb
54
+ #echo "Warning this uses the INCLUDED gem not the latest one in development'
55
+ #ruby -I test test/media/test_gcs_bucket.rb
56
+ rake test TEST=test/media/test_gcs_bucket.rb
57
+
58
+ test-local-folder:
59
+ echo "Warning this uses the INCLUDED gem not the latest one in development'
60
+ ruby -I test test/media/test_local_folder.rb
61
+ # https://medium.com/@tegon/quick-guide-to-minitest-arguments-745bf9fe4b3
62
+ test-media-subfolder:
63
+ rake test TEST="test/media/*.rb"
64
+ test-verbose:
65
+ rake test:verbose --verbose
66
+ test-single-file-continuously:
67
+ \watch -n 5 --color rake test TEST="test/media/test_local_folder.rb"
68
+
69
+ test-binary-with-local-gem:
70
+ ruby -Ilib bin/hello-storazzo
71
+ test-binary-with-system-gem:
72
+ bin/hello-storazzo
73
+
data/README.md CHANGED
@@ -9,6 +9,29 @@
9
9
 
10
10
  (Latest version is hosted in https://rubygems.org/gems/storazzo)
11
11
 
12
+ # Tests
13
+
14
+ I still struggle to enforce the include of LOCAL unchecked code rather than latest required system gem (cmon Ruby!)
15
+ but I found loads of interesting ways to test my code by googling and StoackOverflowing:
16
+
17
+ * `rake test TEST="test/sum_test.rb"`
18
+ * test-gcs-bucket: `ruby -I test test/test_gcs_bucket.rb` (meh - see below)
19
+ * test-media-subfolder: `rake test TEST="test/media/*.rb"`
20
+
21
+ Single test in single file:
22
+
23
+ * `rake test TEST="test/sum_test.rb" TESTOPTS="--name=test_returns_two"` (sample)
24
+ * `rake test TEST="test/media/test_local_folder.rb" TESTOPTS="--name=test_1_first_directory_parsing_actually_works"`
25
+ * `ruby -I test test/test_local_folder.rb -n test_first_directory_parsing_actually_works` (note this includes `storazzo` latest gem
26
+ and doesnt benefit from LATEST code so its NOT good for testing: use RAKE for that).
27
+
28
+ **Testing binary files** is hard: by default they 'require storazzo' and use the GEM INSTALLed version which is a few versions away, usually.
29
+ So while developing you might want to include the lib/ folder, like this:
30
+
31
+ * Use local gem (super latest) for checking latest code: `ruby -Ilib bin/hello-storazzo`
32
+ * This will use the gem installed a few days ago, likely so wont do you any good to test latest code: `bin/hello-storazzo`
33
+
34
+ Now to toggle verbosity I believe I need to go into Rakefile (bummer)
12
35
  # Thanks
13
36
 
14
37
  Inspiration from:
data/Rakefile CHANGED
@@ -1,17 +1,44 @@
1
- #require 'rake'
1
+ # require 'rake'
2
2
 
3
3
  # from hola: https://guides.rubygems.org/make-your-own-gem/#adding-an-executable
4
4
  require "rake/testtask"
5
5
 
6
- Rake::TestTask.new do |t|
6
+ desc "Run Unit tests"
7
+ Rake::TestTask.new(:test) do |t|
7
8
  t.libs << "test"
9
+ t.libs << "test/media"
10
+ # note this is only useful for this: https://chriskottom.com/articles/command-line-flags-for-minitest-in-the-raw/
8
11
  t.verbose = false
9
- t.warning = false
12
+ t.warning = false
13
+ # puts "[RiccardoOnly]: t.pattern: #{t.pattern}"
14
+ t.pattern = 'test/**/test_*.rb'
10
15
  end
11
16
 
12
- desc "Run tests"
17
+ desc "By default, Run Unit tests"
13
18
  task default: :test
14
19
 
20
+ # Adding test/media directory to rake test.
21
+ desc "Test tests/media/* code sobenem"
22
+ namespace :test do
23
+ desc "Test Verbosely by default since I'm too stupid to toggle via ENV var dammit"
24
+ Rake::TestTask.new(:verbose) do |t|
25
+ t.libs << "test"
26
+ t.libs << "test/media"
27
+ t.verbose = true
28
+ t.warning = true
29
+ t.pattern = 'test/**/test_*.rb'
30
+ $DEBUG = true
31
+ end
32
+ end
33
+ # namespace :verbose_test do
34
+ # desc "Test tests/media/* code"
35
+ # Rake::TestTask.new do |t|
36
+ # t.libs << "test/media"
37
+ # # Rails::TestTask.new(media: 'test:prepare') do |t|
38
+ # t.pattern = 'test/**/test_*.rb'
39
+ # end
40
+ # end
41
+ # Rake::Task['test:run'].enhance ["test:media"]
15
42
 
16
43
  # begin
17
44
  # require 'bundler/setup'
@@ -19,14 +46,14 @@ task default: :test
19
46
  # rescue LoadError
20
47
  # puts 'although not required, bundler is recommended for running the tests'
21
48
  # end
22
-
49
+
23
50
  # task default: :spec
24
-
51
+
25
52
  # require 'rspec/core/rake_task'
26
53
  # RSpec::Core::RakeTask.new(:spec)
27
-
54
+
28
55
  # require 'rubocop/rake_task'
29
56
  # RuboCop::RakeTask.new do |task|
30
57
  # task.requires << 'rubocop-performance'
31
58
  # task.requires << 'rubocop-rspec'
32
- # end
59
+ # end
data/VERSION CHANGED
@@ -1 +1 @@
1
- 0.4.2
1
+ 0.5.0
@@ -0,0 +1,5 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require 'storazzo'
4
+
5
+ Storazzo::Main.say_hi(ARGV[0])
data/bin/ricdisk-magic CHANGED
@@ -1,39 +1,35 @@
1
1
  #!/usr/bin/env ruby
2
2
 
3
-
4
-
5
- ############# ############# ############# ############# ############# ############# ############# #############
3
+ ############# ############# ############# ############# ############# ############# ############# #############
6
4
  # placeholder until i make it work..
7
5
  # https://guides.rubygems.org/make-your-own-gem/#adding-an-executable
8
- #require 'storazzo'
9
- #require_relative '../lib/storazzo'
6
+ # require 'storazzo'
7
+ # require_relative '../lib/storazzo'
10
8
  require 'fileutils'
11
9
  require 'yaml'
12
10
  require 'socket'
13
- require 'optparse' # http://ruby.about.com/od/advancedruby/a/optionparser.htm
11
+ require 'optparse' # http://ruby.about.com/od/advancedruby/a/optionparser.htm
14
12
 
15
13
  # Including local gem.
16
- #puts File.expand_path(FileUtils.pwd, "/../")
17
- #local_gem_path = File.expand_path(FileUtils.pwd, "/../")
18
- #gem 'storazzo', path: local_gem_path
14
+ # puts File.expand_path(FileUtils.pwd, "/../")
15
+ # local_gem_path = File.expand_path(FileUtils.pwd, "/../")
16
+ # gem 'storazzo', path: local_gem_path
19
17
  require 'storazzo'
20
- #include Storazzo
21
- #include Storazzo::Colors
18
+ # include Storazzo
19
+ # include Storazzo::Colors
22
20
  extend Storazzo::Colors
23
- include Storazzo::Common #instead
24
-
25
-
26
- #puts "First I need to figure out how to bring in all the libraries in here.."
27
- #Storazzo::Main.say_hi("ARGV is: #{ ARGV.join ', '}")
28
- #puts Storazzo.version
29
- ############# ############# ############# ############# ############# ############# ############# #############
21
+ include Storazzo::Common # instead
30
22
 
23
+ # puts "First I need to figure out how to bring in all the libraries in here.."
24
+ # Storazzo::Main.say_hi("ARGV is: #{ ARGV.join ', '}")
25
+ # puts Storazzo.version
26
+ ############# ############# ############# ############# ############# ############# ############# #############
31
27
 
32
28
  # I believe this is wrong
33
- #require 'storazzo'
29
+ # require 'storazzo'
34
30
  # required to have methods wiothout self.
35
- #include 'lib/ric_disk'
36
- #extend Storazzo::Colors
31
+ # include 'lib/ric_disk'
32
+ # extend Storazzo::Colors
37
33
 
38
34
  if RUBY_VERSION.split('.')[0] == 1
39
35
  puts "Refusing to launch a script form Ruby 1. Sorry Ric, its 2020 damn it!"
@@ -41,7 +37,7 @@ if RUBY_VERSION.split('.')[0] == 1
41
37
  end
42
38
 
43
39
  $PROG_VER = '0.4'
44
- $DEBUG = ENV['DEBUG'] == 'true' # (true/false)
40
+ $DEBUG = ENV['DEBUG'] == 'true' # (true/false)
45
41
  # p ENV['DEBUG']
46
42
  # p $DEBUG
47
43
 
@@ -65,24 +61,24 @@ BIG_LONG_MULTILINE
65
61
 
66
62
  =end
67
63
 
68
-
69
64
  $myconf = {
70
- :app_name => "RicDisk Magic should be sth like #{$0}",
71
- :description => "
65
+ :app_name => "RicDisk Magic should be sth like #{$0}",
66
+ :description => "
72
67
  This program is loosely inspired to ricdisk-magic.sh but its much better.
73
- The idea is: iterate through all ARGV elements and build a DB of all files with length and MD5.
68
+ The idea is: iterate through all ARGV elements and build a DB of all files with length and MD5.
74
69
  Then if folder is writeable, put this list INSIDE IT as its so conveniente.
75
- If not, put in a ENV-var-defined folder which defaults to ./.storazzo/ :)
76
-
70
+ If not, put in a ENV-var-defined folder which defaults to ./.storazzo/ :)
71
+
77
72
  And maybe someday - add a nice protobuf and an entity in lib/.
78
73
 
79
74
  Note. This software is part of the Storazzo gem ('sai chi ti saluta un casino?'):
80
75
 
81
76
  $ gem install storazzo
82
77
  ".strip.gsub(/^\s+/, "").gsub(/\s+$/, ""),
83
- # TODO move to some class default
84
- :media_dirs => %w{ /media/riccardo/ /Volumes/ /mnt/ ~/git/storazzo/var/test/ /sobenme/giusto/per/imparare/ad/ammutolire/gli/errori/ },
85
- :mount_types => %w{ vfat ntfs },
78
+ # TODO move to some class default
79
+ :media_dirs => %w{/media/riccardo/ /Volumes/ /mnt/ ~/git/storazzo/var/test/
80
+ /sobenme/giusto/per/imparare/ad/ammutolire/gli/errori/},
81
+ :mount_types => %w{vfat ntfs},
86
82
  }
87
83
  $stats_file = "ricdisk_stats_v11.rds" # => RicDiskStatsFile
88
84
  $gcs_bucket = 'TODO-CHANGE-CONFIG-palladius'
@@ -92,18 +88,17 @@ $gcs_bucket = 'TODO-CHANGE-CONFIG-palladius'
92
88
  # 2022-04-26 2.1.0 Historical momemnt: removed gem 'ric' dependency
93
89
  $TEMPLATE_VER = '2.1.1'
94
90
 
95
-
96
- def usage(comment=nil)
91
+ def usage(comment = nil)
97
92
  puts white($optparse.banner)
98
93
  puts($optparse.summarize)
99
94
  puts("Description: " + gray($myconf[:description]))
100
95
  puts red(comment) if comment
101
- #puts "Description: #{ $myconf[:description] }"
96
+ # puts "Description: #{ $myconf[:description] }"
102
97
  exit 13
103
98
  end
104
99
 
105
100
  # include it in main if you want a custome one
106
- def init() # see lib_autoinit in lib/util.rb
101
+ def init() # see lib_autoinit in lib/util.rb
107
102
  $opts = {}
108
103
  # setting defaults
109
104
  $opts[:verbose] = false
@@ -113,19 +108,19 @@ def init() # see lib_autoinit in lib/util.rb
113
108
 
114
109
  $optparse = OptionParser.new do |opts|
115
110
  opts.banner = "#{$0} v.#{$PROG_VER}\n Usage: #{File.basename $0} [options] file1 file2 ..."
116
- opts.on( '-d', '--debug', 'enables debug (DFLT=false)' ) { $opts[:debug] = true ; $DEBUG = true }
117
- opts.on( '-f', '--force', 'force stuff (DFLT=false)' ) { $opts[:force] = true }
118
- opts.on( '-h', '--help', 'Display this screen' ) { usage }
119
- #opts.on( '-j', '--jabba', 'Activates my Jabber powerful CLI' ) { $opts[:jabba] = true }
120
- opts.on( '-n', '--dryrun', "Don't really execute code" ) { $opts[:dryrun] = true }
121
- opts.on( '-l', '--logfile FILE', 'Write log to FILE' ) {|file| $opts[:logfile] = file }
122
- opts.on( '-v', '--verbose', 'Output more information' ) { $opts[:verbose] = true}
111
+ opts.on('-d', '--debug', 'enables debug (DFLT=false)') { $opts[:debug] = true; $DEBUG = true }
112
+ opts.on('-f', '--force', 'force stuff (DFLT=false)') { $opts[:force] = true }
113
+ opts.on('-h', '--help', 'Display this screen') { usage }
114
+ # opts.on( '-j', '--jabba', 'Activates my Jabber powerful CLI' ) { $opts[:jabba] = true }
115
+ opts.on('-n', '--dryrun', "Don't really execute code") { $opts[:dryrun] = true }
116
+ opts.on('-l', '--logfile FILE', 'Write log to FILE') { |file| $opts[:logfile] = file }
117
+ opts.on('-v', '--verbose', 'Output more information') { $opts[:verbose] = true }
123
118
  end
124
119
  $optparse.parse!
125
120
  end
126
121
 
127
122
  def real_program
128
- deb("Hello world from a templated '#{yellow $0 }'")
123
+ deb("Hello world from a templated '#{yellow $0}'")
129
124
  deb "+ Options are: #{gray $opts}"
130
125
  deb "+ Depured args: #{azure ARGV}"
131
126
  deb "+ Script-specifig super-cool conf: #{green $prog_conf_d}"
@@ -133,51 +128,49 @@ def real_program
133
128
 
134
129
  # Your code goes here...
135
130
  puts white("Hello world from #{$myconf[:app_name]}!")
136
- puts "Description: '''#{white $myconf[:description] }'''"
131
+ puts "Description: '''#{white $myconf[:description]}'''"
137
132
 
138
- config = Storazzo::RicDiskConfig.instance()
133
+ config = Storazzo::RicDiskConfig.instance()
139
134
  config.load
140
135
  puts "StorazzoConfig: #{config}"
141
- #config.load # auto_sbrodola(ARGV)
136
+ # config.load # auto_sbrodola(ARGV)
142
137
  puts yellow("ARGV: #{ARGV}")
143
138
  config.iterate_through_file_list_for_disks(ARGV)
144
139
 
145
140
  # if ARGV == [] # empty -> ALL
146
141
  # dirs = RicDisk.find_active_dirs()
147
- # dirs.each {|dir|
142
+ # dirs.each {|dir|
148
143
  # RicDisk.sbrodola_ricdisk(dir)
149
144
  # RicDisk.calculate_stats_files(dir) # dir is inutile
150
145
  # } # TODO refactor in option sbrodola_afterwards=true. :)
151
146
  # else
152
147
  # deb "I consider ARGV come la lista di directories da parsare :)"
153
148
  # dirs = RicDisk.find_active_dirs()
154
- # ARGV.each{ |dir|
149
+ # ARGV.each{ |dir|
155
150
  # dir = File.expand_path(dir)
156
151
  # if dirs.include?(dir)
157
152
  # deb "Legit dir: #{green dir}"
158
153
  # RicDisk.sbrodola_ricdisk(dir)
159
154
  # RicDisk.calculate_stats_files(dir) # dir is inutile
160
155
  # else
161
- # deb "Figghiu ri buttana: doesnt exist #{red dir}"
156
+ # deb "Figghiu ri buttana: doesnt exist #{red dir}"
162
157
  # end
163
158
  # }
164
159
  # end #/if aRGV
165
-
166
160
  end
167
161
 
168
162
  def main(filename)
169
163
  deb "I'm called by #{white filename}"
170
164
  deb "HISTORY: #{gray HISTORY}"
171
- #deb "To remove this shit, just set $DEBUG=false :)"
172
- init # Enable this to have command line parsing capabilities!
173
- puts white("$DEBUG is #{$DEBUG }. Tu turn on, call me with -d") unless $DEBUG
174
- #warn "[warn] template v#{$TEMPLATE_VER }: proviamo il warn che magari depreca il DEB"
165
+ init # Enable this to have command line parsing capabilities!
166
+ puts white("$DEBUG is #{$DEBUG}. Tu turn on, call me with -d") unless $DEBUG
167
+ # warn "[warn] template v#{$TEMPLATE_VER }: proviamo il warn che magari depreca il DEB"
175
168
  real_program
176
169
 
177
- if ($DEBUG)
170
+ if ($DEBUG)
178
171
  puts "First I need to figure out how to bring in all the libraries in here.."
179
- Storazzo::Main.say_hi("ARGV is: #{ ARGV.join ', '}")
180
- puts Storazzo.version
172
+ Storazzo::Main.say_hi("ARGV is: #{ARGV.join ', '}")
173
+ puts Storazzo.version
181
174
  end
182
175
  end
183
176