nzbgetpp 0.1.1rc0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/.document ADDED
@@ -0,0 +1,5 @@
1
+ lib/**/*.rb
2
+ bin/*
3
+ -
4
+ features/**/*.feature
5
+ LICENSE.txt
data/.gitignore ADDED
@@ -0,0 +1,15 @@
1
+ # rcov generated
2
+ coverage
3
+
4
+ # rdoc generated
5
+ rdoc
6
+
7
+ # yard generated
8
+ doc
9
+ .yardoc
10
+
11
+ # bundler
12
+ .bundle
13
+
14
+ # jeweler generated
15
+ pkg
data/.rspec ADDED
@@ -0,0 +1 @@
1
+ --color
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source "http://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in nzbgetpp.gemspec
4
+ gemspec
data/Gemfile.lock ADDED
@@ -0,0 +1,28 @@
1
+ PATH
2
+ remote: .
3
+ specs:
4
+ nzbgetpp (0.1.1rc0)
5
+
6
+ GEM
7
+ remote: http://rubygems.org/
8
+ specs:
9
+ diff-lcs (1.1.2)
10
+ rake (0.9.2.2)
11
+ rcov (0.9.9)
12
+ rspec (2.5.0)
13
+ rspec-core (~> 2.5.0)
14
+ rspec-expectations (~> 2.5.0)
15
+ rspec-mocks (~> 2.5.0)
16
+ rspec-core (2.5.1)
17
+ rspec-expectations (2.5.0)
18
+ diff-lcs (~> 1.1.2)
19
+ rspec-mocks (2.5.0)
20
+
21
+ PLATFORMS
22
+ ruby
23
+
24
+ DEPENDENCIES
25
+ nzbgetpp!
26
+ rake
27
+ rcov
28
+ rspec
data/LICENSE.txt ADDED
@@ -0,0 +1,20 @@
1
+ Copyright (c) 2010 Marc Bowes
2
+
3
+ Permission is hereby granted, free of charge, to any person obtaining
4
+ a copy of this software and associated documentation files (the
5
+ "Software"), to deal in the Software without restriction, including
6
+ without limitation the rights to use, copy, modify, merge, publish,
7
+ distribute, sublicense, and/or sell copies of the Software, and to
8
+ permit persons to whom the Software is furnished to do so, subject to
9
+ the following conditions:
10
+
11
+ The above copyright notice and this permission notice shall be
12
+ included in all copies or substantial portions of the Software.
13
+
14
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,103 @@
1
+ # nzbgetpp #
2
+
3
+ This is a postprocessing script for
4
+ [nzbget](http://nzbget.sf.net). That is, after nzbget is done
5
+ downloading things, this is a script that can be called to tidy up
6
+ your download. Currently, we support:
7
+
8
+ * unrarring files
9
+ * removing unneeded files
10
+ * categories
11
+ * callbacks
12
+ * logging
13
+ * tests
14
+
15
+ NzbGetPP has a pre-defined callback for
16
+ [dewey](http://github.com/timsjoberg/dewey), which will automatically
17
+ place TV shows in the right place (and can be configured to rename
18
+ things).
19
+
20
+ Other callbacks can include things like telling some other client to
21
+ download the now-available file (more on this coming Real Soon Now) or
22
+ updating a database collection.
23
+
24
+ ## Design ##
25
+
26
+ One of the major painpoints I've found with writing nzbget
27
+ postprocessing scripts is they tend to be monolithic (bash) scripts
28
+ and become really hard to reason about and extend. Testing them is
29
+ also painful. This project has been designed to make it real easy to
30
+ reason about it (OO design), see what is happening (logging) and test
31
+ it works without actually downloading things (rspec).
32
+
33
+ ## Installing ##
34
+
35
+ Install the project via a gem or git/hub:
36
+
37
+ `gem install nzbgetpp`
38
+
39
+ .. or if that doesn't work, or you want the Git version:
40
+
41
+ NZBGETPP_INSTALL_PATH=~/src/nzbgetpp
42
+ git clone git://github.com/marcbowes/nzbgetpp $NZBGETPP_INSTALL_PATH
43
+ # or:
44
+ # curl https://github.com/marcbowes/nzbgetpp/tarball/master > $NZBGETPP_INSTALL_PATH
45
+ pushd $NZBGETPP_INSTALL_PATH
46
+ sudo rake install
47
+
48
+ ## Configuring ##
49
+
50
+ We look in two places for a `config.rb` file: either the default one
51
+ we ship in `support/`, or in `$HOME/.nzbgetpp/config.rb`. This is a
52
+ Ruby script that essentially lets you write code into
53
+ `lib/nzbgetpp.rb`. We provide convenience `configure do` and
54
+ `install_callback(name) do` methods, both of which have examples in
55
+ the factory edition of the config.
56
+
57
+ The shipped version should "Just Work (TM)" in that it tries to pick
58
+ the various binaries out of your environment. We make the assumption
59
+ that you want to store stuff in `$HOME/download/complete`, but you can
60
+ change this by setting `storage_directory` in the configure block.
61
+
62
+ The only possibly mysterious value is the `scratch_directory`. This is
63
+ the directory we use to work on the nzb. It means that we don't step
64
+ on anyone's toes as it is neither the initial destination, nor the
65
+ final. That is, if you have some other script running (e.g. scanning
66
+ for new files), this prevents race conditions (we do a `mv` at the
67
+ end, which is atomic on the filesystem). That said, please make sure
68
+ that `storage_directory` is on the same drive as `scratch_directory`,
69
+ else you will incur a penalty when moving the files at the end.
70
+
71
+ ## Alternatives ##
72
+
73
+ Have a look at the
74
+ [nzbget page on post-processing scripts](http://nzbget.sourceforge.net/Postprocessing_scripts). At the time of writing, the two options are:
75
+
76
+ * [PPWeb](http://dalrun.com/Linux/Software/Nzbget/PPWeb/) is a Perl
77
+ based Web solution for managing nzbget and comes with post-processing scripts.
78
+ * [Oversight](http://code.google.com/p/oversight/wiki/UnpackingScriptsOnly)
79
+ is a full system but they provide their unpacking scripts in
80
+ isolation. I was using this for a while and it works OK, but I found
81
+ it slow and tricky to extend.
82
+
83
+ ## Contributing to nzbgetpp ##
84
+
85
+ * Check out the latest master to make sure the feature hasn't been
86
+ implemented or the bug hasn't been fixed yet
87
+ * Check out the issue tracker to make sure someone already hasn't
88
+ requested it and/or contributed it
89
+ * Fork the project
90
+ * Start a feature/bugfix branch
91
+ * Commit and push until you are happy with your contribution
92
+ * Make sure to add tests for it. This is important so I don't break it
93
+ in a future version unintentionally.
94
+ * Please try not to mess with the Rakefile, version, or history. If
95
+ you want to have your own version, or is otherwise necessary, that
96
+ is fine, but please isolate to its own commit so I can cherry-pick
97
+ around it.
98
+
99
+ ## Copyright ##
100
+
101
+ Copyright (c) 2011 Marc Bowes. I don't care what you do with it. There
102
+ are no guarentees.
103
+
data/Rakefile ADDED
@@ -0,0 +1,17 @@
1
+ require 'bundler'
2
+ Bundler::GemHelper.install_tasks
3
+
4
+ require 'rspec/core/rake_task'
5
+
6
+ RSpec::Core::RakeTask.new do |t|
7
+ t.rspec_opts = %w(-fs --color)
8
+ # If you're pedantic ..
9
+ # t.ruby_opts = %w(-w)
10
+ end
11
+
12
+ RSpec::Core::RakeTask.new(:rcov) do |spec|
13
+ spec.pattern = 'spec/**/*_spec.rb'
14
+ spec.rcov = true
15
+ end
16
+
17
+ task :default => :spec
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 0.1.0
data/bin/nzbgetpp ADDED
@@ -0,0 +1,82 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ # Reference: http://nzbget.svn.sourceforge.net/viewvc/nzbget/trunk/postprocess-example.sh?revision=359&content-type=text%2Fplain
4
+ #
5
+ # NZBGet passes following arguments to postprocess-programm as environment
6
+ # variables:
7
+ # NZBPP_DIRECTORY - path to destination dir for downloaded files;
8
+ # NZBPP_NZBFILENAME - name of processed nzb-file;
9
+ # NZBPP_PARFILENAME - name of par-file or empty string (if no collections were
10
+ # found);
11
+ # NZBPP_PARSTATUS - result of par-check:
12
+ # 0 = not checked: par-check disabled or nzb-file does
13
+ # not contain any par-files;
14
+ # 1 = checked and failed to repair;
15
+ # 2 = checked and successfully repaired;
16
+ # 3 = checked and can be repaired but repair is disabled;
17
+ # NZBPP_NZBCOMPLETED - state of nzb-job:
18
+ # 0 = there are more collections in this nzb-file queued;
19
+ # 1 = this was the last collection in nzb-file;
20
+ # NZBPP_PARFAILED - indication of failed par-jobs for current nzb-file:
21
+ # 0 = no failed par-jobs;
22
+ # 1 = current par-job or any of the previous par-jobs for
23
+ # the same nzb-files failed;
24
+ # NZBPP_CATEGORY - category assigned to nzb-file (can be empty string).
25
+ #
26
+
27
+ # ARGV looks something like this
28
+ # [
29
+ # "/path/to/dst/name-of-folder", # 0. NZBPP_DIRECTORY
30
+ # "/path/to/nzb/name-of-nzb", # 1. NZBPP_NZBFILENAME
31
+ # "/path/to/dst/name-of-folder/name-of-par2", # 2. NZBPP_PARFILENAME
32
+ # "2", # 3. NZBPP_PARSTATUS
33
+ # "1", # 4. NZBPP_NZBCOMPLETED
34
+ # "0", # 5. NZBPP_PARFAILED
35
+ # "" # 6. NZBPP_CATEGORY
36
+ # ]
37
+
38
+ # Return value: nzbget processes the exit code returned by the script:
39
+ # 91 - request nzbget to do par-check/repair for current collection in the
40
+ # current nzb-file;
41
+ # 92 - request nzbget to do par-check/repair for all collections in the
42
+ # current nzb-file;
43
+ # 93 - post-process successful (status = SUCCESS);
44
+ # 94 - post-process failed (status = FAILURE);
45
+ # 95 - post-process skipped (status = NONE);
46
+ # All other return codes are interpreted as "status unknown".
47
+ RC_POSTPROCESS_PARCHECK_CURRENT = 91
48
+ RC_POSTPROCESS_PARCHECK_ALL = 92
49
+ RC_POSTPROCESS_SUCCESS = 93
50
+ RC_POSTPROCESS_ERROR = 94
51
+ RC_POSTPROCESS_NONE = 95
52
+
53
+ begin
54
+ require 'nzbgetpp'
55
+
56
+ pp = NzbGetPP::PostProcessor.new(ARGV[0],
57
+ NzbGetPP::Nzb.new(ARGV[1],
58
+ ARGV[4]),
59
+ NzbGetPP::Par.new(ARGV[2],
60
+ ARGV[3],
61
+ ARGV[5]),
62
+ ARGV[6])
63
+ pp.postprocess!
64
+
65
+ Kernel.exit! RC_POSTPROCESS_SUCCESS
66
+ rescue NzbGetPP::PostProcessor::InflationError
67
+ Kernel.exit! RC_POSTPROCESS_PARCHECK_ALL
68
+ rescue Exception => e
69
+ STDERR.puts("[ERROR] (#{e.class.name}) #{e.message}")
70
+ e.backtrace.each do |line|
71
+ STDERR.puts("[DETAIL] #{line}")
72
+ end
73
+
74
+ File.open("/tmp/nzbgetpp.stderr", "a") do |f|
75
+ f.puts("An exception occurred running nzbgetpp at " + Time.now.to_s)
76
+ f.puts([[e.class.name, e.message].join(": "),
77
+ e.backtrace.join("\n")].join("\n"))
78
+ f.puts()
79
+ end
80
+
81
+ Kernel.exit! RC_POSTPROCESS_ERROR
82
+ end
@@ -0,0 +1,90 @@
1
+ module NzbGetPP
2
+ class Log
3
+
4
+ require "time"
5
+
6
+ LEVELS = {
7
+ :debug => 0,
8
+ :detail => 1,
9
+ :info => 2,
10
+ :warning => 3,
11
+ :error => 4,
12
+ :fatal => 5,
13
+ }
14
+
15
+ attr_accessor :log_fn
16
+ attr_reader :level
17
+
18
+ def initialize(level = :debug,
19
+ log_fn = nil)
20
+ self.level = level
21
+ self.log_fn = log_fn
22
+ open_log_fd()
23
+ end
24
+
25
+ def open_log_fd
26
+ @log_fd.close() if @log_fd and not @log_fd.closed?
27
+ if self.log_fn
28
+ @log_fd = File.open(self.log_fn, "a")
29
+ end
30
+ end
31
+
32
+ def close_log_fd
33
+ if @log_fd and not @log_fd.closed?
34
+ @log_fd.flush()
35
+ @log_fd.close()
36
+ end
37
+ end
38
+
39
+ LEVELS.each_key do |level|
40
+ define_method(level) do |message|
41
+ write_to_log(level, message)
42
+ end
43
+ end
44
+
45
+ def level=(level)
46
+ @level = level
47
+ @_enum_level = LEVELS[level]
48
+ end
49
+
50
+ def write_to_log(level, message)
51
+ if LEVELS[level] >= @_enum_level
52
+ do_write_to_log(level, message)
53
+ else
54
+ # Toss it
55
+ end
56
+ end
57
+
58
+ def do_write_to_log(level, message)
59
+ puts("[%s] %s" % [level.to_s.upcase, message])
60
+
61
+ if @log_fd and not @log_fd.closed?
62
+ log_for_humans(Time.now.iso8601(), level, message)
63
+ end
64
+ end
65
+
66
+ # Map the log level (LOG_LEVEL_*) to an array containing
67
+ # [human_name:String, colour:String(ANSI escape sequence)]
68
+ ANSI_RED = "\033[0;31m"
69
+ ANSI_RED_INVERTED = "\033[7;31m"
70
+ ANSI_BROWN = "\033[0;33m"
71
+ ANSI_MAGENTA = "\033[0;35m"
72
+ ANSI_GREEN = "\033[0;32m"
73
+ ANSI_BOLD_WHITE = "\033[0;37m"
74
+ ANSI_NORMAL = "\033[0m"
75
+ HUMAN_LOG_LEVELS = {
76
+ :fatal => ["FATAL", ANSI_RED_INVERTED],
77
+ :error => ["ERROR", ANSI_RED],
78
+ :warning => ["WARN", ANSI_MAGENTA],
79
+ :info => ["INFO", ANSI_GREEN],
80
+ :detail => ["DEBUG", ANSI_NORMAL],
81
+ :debug => ["DEBUG", ANSI_BROWN],
82
+ }
83
+ def log_for_humans(iso8601_timestr, log_level, log_msg)
84
+ level_str, level_colour = HUMAN_LOG_LEVELS[log_level]
85
+ @log_fd.puts("#{ANSI_BOLD_WHITE}#{iso8601_timestr}#{ANSI_NORMAL} "\
86
+ "#{level_colour}#{"%5.5s" % level_str}#{ANSI_NORMAL} #{log_msg}")
87
+ @log_fd.flush()
88
+ end
89
+ end
90
+ end
@@ -0,0 +1,26 @@
1
+ module NzbGetPP
2
+ class Nzb
3
+
4
+ COMPLETED = {
5
+ "0" => false,
6
+ "1" => true,
7
+ }
8
+
9
+ attr_reader :filename
10
+
11
+
12
+ def initialize(filename, completed)
13
+ @filename = filename
14
+ self.completed = completed
15
+ end
16
+
17
+
18
+ def completed=(nzbpp_nzbcompleted)
19
+ @completed = COMPLETED[nzbpp_nzbcompleted]
20
+ end
21
+
22
+ def completed?
23
+ !!@completed
24
+ end
25
+ end
26
+ end
@@ -0,0 +1,38 @@
1
+ module NzbGetPP
2
+ class Par
3
+
4
+ STATUSES = {
5
+ "0" => :not_checked,
6
+ "1" => :cannot_repair,
7
+ "2" => :repaired,
8
+ "3" => :repairable,
9
+ }
10
+
11
+ FAILED = {
12
+ "0" => false,
13
+ "1" => true,
14
+ }
15
+
16
+ attr_reader :filename, :status
17
+
18
+
19
+ def initialize(filename, status, failed)
20
+ @filename = filename
21
+ self.status = status
22
+ self.failed = failed
23
+ end
24
+
25
+
26
+ def status=(nzbpp_parstatus)
27
+ @status = STATUSES[nzbpp_parstatus]
28
+ end
29
+
30
+ def failed=(nzbpp_parfailed)
31
+ @failed = FAILED[nzbpp_parfailed]
32
+ end
33
+
34
+ def failed?
35
+ !!@failed
36
+ end
37
+ end
38
+ end
@@ -0,0 +1,27 @@
1
+ module NzbGetPP
2
+ module Shellout
3
+ # FIXME: probably want to implement timeouts here..
4
+ def shellout(cmd)
5
+ stdout_rd, stdout_wr = IO.pipe
6
+ stderr_rd, stderr_wr = IO.pipe
7
+ child_pid, child_status = nil
8
+ child_pid = Kernel.fork
9
+
10
+ if child_pid
11
+ stdout_wr.close
12
+ stderr_wr.close
13
+ return [child_pid, stdout_wr, stderr_wr]
14
+ else
15
+ Process.setsid
16
+ STDIN.close
17
+ STDOUT.reopen(stdout_wr)
18
+ STDERR.reopen(stderr_wr)
19
+ 3.upto(256) { |fd| IO.new(fd).close rescue nil }
20
+
21
+ Kernel.exec(cmd)
22
+ # Never reached.
23
+ Kernel.exit!
24
+ end
25
+ end
26
+ end
27
+ end
@@ -0,0 +1,3 @@
1
+ module NzbGetPP
2
+ VERSION = "0.1.1rc0"
3
+ end
data/lib/nzbgetpp.rb ADDED
@@ -0,0 +1,244 @@
1
+ module NzbGetPP
2
+
3
+ require "nzbgetpp/version"
4
+
5
+ class PostProcessor
6
+
7
+ require 'fileutils'
8
+ require 'pathname'
9
+ require 'ostruct'
10
+ require 'nzbgetpp/log'
11
+ require 'nzbgetpp/nzb'
12
+ require 'nzbgetpp/par'
13
+ require 'nzbgetpp/shellout'
14
+
15
+ include Shellout
16
+
17
+ class Error < RuntimeError; end
18
+ class ConfigError < Error; end
19
+ class InflationError < Error; end
20
+
21
+
22
+ attr_accessor :category
23
+ attr_accessor :config
24
+ attr_accessor :log
25
+ attr_accessor :nzb
26
+ attr_accessor :par
27
+ attr_reader :path
28
+
29
+ def initialize(path,
30
+ nzb,
31
+ par,
32
+ category)
33
+ self.path = path
34
+ self.nzb = nzb
35
+ self.par = par
36
+ self.category = category
37
+
38
+ clear_callbacks()
39
+ load_config()
40
+ self.log = mk_log(config.log_level, config.log_fn)
41
+ end
42
+
43
+
44
+ def load_config()
45
+ @config = OpenStruct.new()
46
+ config_fn = figure_config_fn()
47
+ ruby = File.read(config_fn)
48
+ Kernel.eval(ruby, binding(), config_fn, 0)
49
+ @config
50
+ end
51
+
52
+ def configure
53
+ yield(@config)
54
+ end
55
+
56
+ def install_callback(name, &block)
57
+ @callbacks.push([name, block])
58
+ end
59
+
60
+ def clear_callbacks
61
+ @callbacks = Array.new()
62
+ end
63
+
64
+ def run_callbacks
65
+ log.info("Running callbacks")
66
+ log.detail("Callbacks #=> #{@callbacks.inspect()}")
67
+
68
+ @callbacks.each do |name, block|
69
+ log.debug("Running callback #{name}")
70
+ block.call()
71
+ end
72
+ end
73
+
74
+ def postprocess!
75
+ unless nzb.completed?
76
+ log.info("Called prematurely - this nzb ain't ready for me")
77
+ return true
78
+ end
79
+
80
+ unless Dir[path.join("*.rar").to_s].empty?
81
+ inflate_rar_files
82
+ else
83
+ log.info("Nothing to inflate")
84
+ end
85
+
86
+ remove_unneeded_files(path)
87
+ merge_directory_tree(path, scratch_directory)
88
+ remove_download_directory(path)
89
+ remove_unneeded_files(scratch_directory)
90
+
91
+ log.info("Moving to final resting place")
92
+ storage_dir = storage_directory.dirname
93
+ unless storage_dir.exist?
94
+ log.debug("Storage dir #{storage_dir} will be created")
95
+ FileUtils.mkdir_p(storage_dir.to_s)
96
+ end
97
+ FileUtils.mv(scratch_directory, storage_directory)
98
+ run_callbacks()
99
+
100
+ log.info("Postprocessing complete")
101
+ log.close_log_fd()
102
+ return true
103
+ rescue PostProcessor::Error => e
104
+ log.error("#{e.class.name}: #{e.message}")
105
+ e.backtrace.each do |line|
106
+ log.detail(line)
107
+ end
108
+ raise(e)
109
+ end
110
+
111
+ # We shellout to unrar as a demonstration of our lack of faith for
112
+ # Ruby-based unraring to handle large files efficiently, fast or
113
+ # even at all.
114
+ def inflate_rar_files
115
+ log.info("Inflating rar files")
116
+
117
+ unrar_cmd = mk_unrar_cmd
118
+ log.debug(unrar_cmd)
119
+ FileUtils.mkdir_p(scratch_directory)
120
+
121
+ child_pid, stdout, stderr = shellout(unrar_cmd)
122
+
123
+ # Will raise Errno::ECHILD if the pid doesn't exist
124
+ until Process.wait(child_pid, Process::WNOHANG)
125
+ readable, _ = IO.select([stdout, stderr], [], [], 1)
126
+ readable.each do |io|
127
+ line = io.readline
128
+ case io
129
+ when stdout
130
+ log.detail(line)
131
+ else
132
+ log.error(line)
133
+ end
134
+ end
135
+ end
136
+
137
+ if $?.success?
138
+ log.info("Inflation succeeded")
139
+ else
140
+ raise(InflationError, child_status.inspect)
141
+ end
142
+ end
143
+
144
+ def merge_directory_tree(source, destination)
145
+ Dir.glob(File.join(source, "**/*")).each do |f|
146
+ # Skip directories. If they're empty, we don't want to create
147
+ # them. If they have content, we'll create them via the
148
+ # mkdir_p().
149
+ next if File.directory?(f)
150
+
151
+ rel_dir = File.dirname(f).gsub(/^#{Regexp.escape(source.to_s)}\/?/, "")
152
+ target_dir = File.join(destination,
153
+ rel_dir)
154
+
155
+ unless File.exist?(target_dir)
156
+ log.debug("Making %s" % target_dir)
157
+ FileUtils.mkdir_p(target_dir)
158
+ end
159
+
160
+ log.debug("Will move #{f} to #{target_dir}")
161
+ FileUtils.mv(f, target_dir)
162
+ end
163
+ end
164
+
165
+ def remove_download_directory(where)
166
+ FileUtils.rm_r(where)
167
+ end
168
+
169
+ def remove_unneeded_files(where)
170
+ log.info("Cleaning up unnecessary files in %s" %
171
+ where.to_s)
172
+
173
+ config.discard_files.each do |glob|
174
+ log.debug("Will remove %s #=> %s" %
175
+ [
176
+ where.join(glob),
177
+ Dir.glob(where.join(glob)).inspect()
178
+ ])
179
+ Dir.glob(where.join(glob)).each do |file|
180
+ FileUtils.rm_f(file)
181
+ end
182
+ end
183
+
184
+ Dir.glob(where.join("*")) do |glob|
185
+ remove_unneeded_files(Pathname.new(glob)) if File.directory? glob
186
+ end
187
+ end
188
+
189
+ def path=(path)
190
+ @path = Pathname.new(File.expand_path(path))
191
+ raise ArgumentError unless @path.exist?
192
+ rescue => e
193
+ raise(ArgumentError,
194
+ "#{path.inspect} is not a valid download destination")
195
+ end
196
+
197
+ def mk_log(log_level, log_fn)
198
+ Log.new(log_level, log_fn)
199
+ end
200
+
201
+ def mk_unrar_cmd
202
+ [
203
+ config.unrar_bin,
204
+ config.unrar_flags.join(" "),
205
+ "\"#{unrar_targets.to_s}\"",
206
+ "\"#{scratch_directory.to_s}\"",
207
+ ].join(" ")
208
+ end
209
+
210
+ def unrar_targets
211
+ @unrar_targets ||= Pathname.new(File.join(path,
212
+ config.unrar_targets)).expand_path
213
+ end
214
+
215
+ def scratch_directory
216
+ @scratch_directory ||= Pathname.new(File.join(*[
217
+ config.scratch_directory,
218
+ self.category,
219
+ path.basename,
220
+ ].compact)).expand_path
221
+ end
222
+
223
+ def storage_directory
224
+ @storage_directory ||= Pathname.new(File.join(*[
225
+ config.storage_directory,
226
+ self.category,
227
+ path.basename,
228
+ ].compact)).expand_path
229
+ end
230
+
231
+ def figure_config_fn
232
+ fn = "config.rb"
233
+
234
+ custom = File.join(ENV["HOME"], ".nzbgetpp", fn)
235
+ return custom if File.exist? custom
236
+
237
+ default = File.expand_path(File.join(File.dirname(__FILE__), "../support", fn))
238
+ return default if File.exist? default
239
+
240
+ raise RuntimeError,
241
+ "Unable to find config in #{[custom, default].inspect}"
242
+ end
243
+ end
244
+ end
data/nzbgetpp.gemspec ADDED
@@ -0,0 +1,25 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "nzbgetpp/version"
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "nzbgetpp"
7
+ s.version = NzbGetPP::VERSION
8
+ s.platform = Gem::Platform::RUBY
9
+ s.authors = ["Marc Bowes"]
10
+ s.email = ["marcbowes+nzbgetpp@gmail.com"]
11
+ s.homepage = ""
12
+ s.summary = %q{NzbGet Postprocessor}
13
+ s.description = %q{A postprocessing script for NzbGet, written in Ruby.}
14
+
15
+ s.rubyforge_project = "nzbgetpp"
16
+
17
+ s.files = `git ls-files`.split("\n")
18
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
19
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
20
+ s.require_paths = ["lib"]
21
+
22
+ s.add_development_dependency(%q<rake>, [">= 0"])
23
+ s.add_development_dependency(%q<rspec>, [">= 0"])
24
+ s.add_development_dependency(%q<rcov>, [">= 0"])
25
+ end
data/spec/log_spec.rb ADDED
@@ -0,0 +1,22 @@
1
+ require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
2
+
3
+ require 'nzbgetpp/log'
4
+
5
+ describe NzbGetPP::Log do
6
+ before do
7
+ @log = NzbGetPP::MockLog.new
8
+ end
9
+
10
+ it "should write to the log when the level is permitting" do
11
+ @log.debug("foo")
12
+ @log.store.should == [
13
+ [:debug, "foo"]
14
+ ]
15
+ end
16
+
17
+ it "should toss unprivileged messages" do
18
+ @log.level = :error
19
+ @log.debug("foo")
20
+ @log.store.should be_empty
21
+ end
22
+ end
data/spec/nzb_spec.rb ADDED
@@ -0,0 +1,17 @@
1
+ require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
2
+
3
+ require 'nzbgetpp/nzb'
4
+
5
+ describe NzbGetPP::Nzb do
6
+ before do
7
+ @nzb = NzbGetPP::Nzb.new("name.nzb", "1")
8
+ end
9
+
10
+ it "should correctly report completed?" do
11
+ @nzb.completed?.should be_true
12
+ @nzb.completed = "0"
13
+ @nzb.completed?.should be_false
14
+ @nzb.completed = "1"
15
+ @nzb.completed?.should be_true
16
+ end
17
+ end
@@ -0,0 +1,106 @@
1
+ require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
2
+
3
+ require 'fileutils'
4
+
5
+ describe NzbGetPP do
6
+ before do
7
+ @workdir = "/tmp/nzbgetpp-spec-#{$$}"
8
+ FileUtils.mkdir_p(@workdir)
9
+
10
+ FileUtils.cp_r(File.join(File.dirname(__FILE__),
11
+ "support/sample-dst"),
12
+ File.join(@workdir,
13
+ "dst"))
14
+
15
+ @pp = NzbGetPP::PostProcessor.new(File.join(@workdir, "dst/example job"),
16
+ NzbGetPP::Nzb.new("example.nzb", 1),
17
+ NzbGetPP::Par.new("example.par2", 2, 0),
18
+ nil)
19
+ @pp.config.scratch_directory = File.join(@workdir, "scratch")
20
+
21
+ # Comment me out if you want logging for debug purposes
22
+ # @pp.log.log_fn = "/tmp/nzbgetpp.log"
23
+ # @pp.log.open_log_fd()
24
+ @pp.log = NzbGetPP::MockLog.new()
25
+ end
26
+
27
+ it "should correctly determine unrar targets" do
28
+ @pp.unrar_targets.to_s.should == File.join(@workdir, "dst/example job/*.rar")
29
+ end
30
+
31
+ it "should correctly determine scratch directory" do
32
+ @pp.scratch_directory.to_s.should == File.join(@pp.config.scratch_directory, "example job")
33
+ end
34
+
35
+ it "should correctly determine scratch directory with categories" do
36
+ @pp.category = "category"
37
+ @pp.scratch_directory.to_s.should == File.join(@pp.config.scratch_directory, "category/example job")
38
+ end
39
+
40
+ it "should correctly determine storage directory with categories" do
41
+ @pp.category = "category"
42
+ @pp.storage_directory.to_s.should == File.join(@pp.config.storage_directory, "category/example job")
43
+ end
44
+
45
+ # Disabled: requires unrar to be installed
46
+ #
47
+ # it "should inflate rars" do
48
+ # @pp.inflate_rar_files
49
+ # @pp.scratch_directory.join("example.file").exist?.should be_true
50
+ # end
51
+
52
+ it "should move survivors to the scratch dir" do
53
+ @pp.merge_directory_tree(@pp.path.to_s, @pp.scratch_directory)
54
+ @pp.scratch_directory.join("survivor").exist?.should be_true
55
+ end
56
+
57
+ it "should remove unneeded files" do
58
+ # Make some junk
59
+ junk_dir = Pathname.new(File.join(@workdir, "junk"))
60
+ FileUtils.mkdir_p(junk_dir)
61
+ junk_files = %w[
62
+ junk.rar
63
+ junk.r00
64
+ junk.r01
65
+ junk.s00
66
+ junk.s01
67
+ junk.nzb
68
+ junk.1
69
+ junk.sfv
70
+ junk.srr
71
+ junk.sample.file
72
+ _brokenlog.txt
73
+ survivor
74
+ ].each do |junk_file|
75
+ FileUtils.touch(junk_dir.join(junk_file))
76
+ end
77
+ Dir[junk_dir.join("*").to_s].size.should == junk_files.size
78
+
79
+ @pp.remove_unneeded_files(junk_dir)
80
+
81
+ # Check we completed our list
82
+ @pp.config.discard_files.each do |glob|
83
+ Dir[junk_dir.join(glob).to_s].should be_empty
84
+ end
85
+
86
+ # Safety net
87
+ survivor = junk_dir.join("survivor")
88
+ survivor.exist?.should be_true
89
+ FileUtils.rm(survivor.to_s)
90
+ Dir[junk_dir.join("*").to_s].should be_empty
91
+ end
92
+
93
+ it "should run callbacks" do
94
+ @pp.clear_callbacks
95
+ touched = false
96
+ @pp.install_callback(:toucher) do
97
+ touched = true
98
+ end
99
+ @pp.run_callbacks
100
+ touched.should == true
101
+ end
102
+
103
+ after do
104
+ FileUtils.rm_r(@workdir)
105
+ end
106
+ end
data/spec/par_spec.rb ADDED
@@ -0,0 +1,29 @@
1
+ require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
2
+
3
+ require 'nzbgetpp/par'
4
+
5
+ describe NzbGetPP::Par do
6
+ before do
7
+ @par = NzbGetPP::Par.new("name.par2", "2", "0")
8
+ end
9
+
10
+ it "should correctly report status" do
11
+ @par.status.should == :repaired
12
+ @par.status = "0"
13
+ @par.status.should == :not_checked
14
+ @par.status = "1"
15
+ @par.status.should == :cannot_repair
16
+ @par.status = "2"
17
+ @par.status.should == :repaired
18
+ @par.status = "3"
19
+ @par.status.should == :repairable
20
+ end
21
+
22
+ it "should correctly report failed?" do
23
+ @par.failed?.should be_false
24
+ @par.failed = "1"
25
+ @par.failed?.should be_true
26
+ @par.failed = "0"
27
+ @par.failed?.should be_false
28
+ end
29
+ end
@@ -0,0 +1,19 @@
1
+ require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
2
+
3
+ require 'nzbgetpp/shellout'
4
+
5
+ include NzbGetPP::Shellout
6
+
7
+ describe NzbGetPP::Shellout do
8
+ it "should return 0 for true" do
9
+ pid, _ = shellout("/bin/true")
10
+ Process.wait(pid)
11
+ $?.success?.should be_true
12
+ end
13
+
14
+ it "should return 1 for false" do
15
+ pid, _ = shellout("/bin/false")
16
+ Process.wait(pid)
17
+ $?.success?.should be_false
18
+ end
19
+ end
@@ -0,0 +1,12 @@
1
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
2
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
3
+ require 'rspec'
4
+ require 'nzbgetpp'
5
+
6
+ # Requires supporting files with custom matchers and macros, etc,
7
+ # in ./support/ and its subdirectories.
8
+ Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f}
9
+
10
+ RSpec.configure do |config|
11
+
12
+ end
@@ -0,0 +1,16 @@
1
+ module NzbGetPP
2
+ require 'nzbgetpp/log'
3
+
4
+ class MockLog < Log
5
+ attr_reader :store
6
+
7
+ def initialize
8
+ @store = []
9
+ super
10
+ end
11
+
12
+ def do_write_to_log(level, message)
13
+ @store << [level, message]
14
+ end
15
+ end
16
+ end
File without changes
@@ -0,0 +1,47 @@
1
+ require "spec_helper"
2
+
3
+ require 'nzbgetpp/shellout'
4
+
5
+ include NzbGetPP::Shellout
6
+
7
+ describe NzbGetPP do
8
+ it "should parse * as a * not as an expanded list of arguments" do
9
+ sample_prog_fpath = File.join("/tmp/nzbgetpp-spec", Process.pid.to_s, "arg_writer.sh")
10
+ dir = File.dirname(sample_prog_fpath)
11
+ FileUtils.mkdir_p(dir)
12
+ File.open(sample_prog_fpath, "w") do |f|
13
+ f.puts <<SCRIPT
14
+ #!/usr bin/env/ruby
15
+
16
+ argc_fpath = File.expand_path("argc", File.dirname(__FILE__))
17
+ argv_fpath = File.expand_path("argv", File.dirname(__FILE__))
18
+ File.open(argc_fpath, "w") do |f|
19
+ f.puts(ARGV.size)
20
+ end
21
+ File.open(argv_fpath, "w") do |f|
22
+ f.puts(ARGV.inspect)
23
+ end
24
+ SCRIPT
25
+ end
26
+ FileUtils.touch(File.join(dir, "foo"))
27
+ FileUtils.touch(File.join(dir, "bar"))
28
+
29
+ arg = File.join(dir, "*").inspect
30
+ cmd = [
31
+ "ruby",
32
+ sample_prog_fpath,
33
+ arg,
34
+ ].join(" ")
35
+ child_pid, _ = shellout(cmd)
36
+ Process.wait(child_pid)
37
+ $?.success?.should be_true
38
+
39
+ File.read(File.join(dir, "argc")).strip.
40
+ should == "1"
41
+
42
+ File.read(File.join(dir, "argv")).strip.
43
+ should == "[#{arg}]"
44
+
45
+ FileUtils.rm_r(dir)
46
+ end
47
+ end
data/support/config.rb ADDED
@@ -0,0 +1,98 @@
1
+ def which(bin)
2
+ candidates = ENV["PATH"].split(":").map do |dir|
3
+ File.expand_path(bin, dir)
4
+ end
5
+
6
+ candidates.detect do |candidate|
7
+ File.exist?(candidate)
8
+ end
9
+ end
10
+
11
+ configure do |c|
12
+ c.unrar_targets = "*.rar"
13
+ c.discard_files = [
14
+ "*.rar",
15
+ "*.r[0-9][0-9]",
16
+ "*.s[0-9][0-9]",
17
+ "*.nzb",
18
+ "*.par2",
19
+ "*.1",
20
+ "*.sfv",
21
+ "*.srr",
22
+ "*sample*",
23
+ "_brokenlog.txt",
24
+ ]
25
+ c.scratch_directory = File.join(ENV["HOME"], "download/scratch")
26
+ c.storage_directory = File.join(ENV["HOME"], "download/complete")
27
+ c.unrar_bin = which("unrar")
28
+ c.unrar_flags = [
29
+ "x",
30
+ "-y",
31
+ "-p-",
32
+ "-o+",
33
+ ]
34
+
35
+ c.log_level = :debug
36
+ log_dir = File.join(ENV["HOME"], ".nzbgetpp")
37
+ c.log_fn = File.join(log_dir, "log") if File.exist?(log_dir)
38
+ end
39
+
40
+ #
41
+ # Dewey support
42
+ #
43
+ configure do |c|
44
+ c.tv_dir = File.join(config.storage_directory, "archive/tv.series")
45
+ c.dewey_bin = which("dewey")
46
+ end
47
+
48
+ def move_tv_shows_to_the_archive
49
+ if category.nil? ||
50
+ category !~ /tv/i
51
+ # Not TV, so move along
52
+ log.detail("This isn't a TV show, so dewey won't be invoked")
53
+ return
54
+ end
55
+
56
+ if config.dewey_bin.nil? ||
57
+ config.dewey_bin.empty? ||
58
+ !File.exist?(config.dewey_bin)
59
+ # Dewey not available, so we won't archive this tv show
60
+ log.warning("This is a TV show but dewey isn't available :-(")
61
+ return
62
+ end
63
+
64
+ # Support for appending extra information into your tv-dir. For
65
+ # example, this allows us to set the category to 'tv/720p' and call
66
+ # dewey with '--tv-dir=archive/tv.series/720p'.
67
+ tv_dir = config.tv_dir
68
+ if (m = category.match(/tv\/(\w+)/i))
69
+ tv_dir = File.join(tv_dir, m[1])
70
+ end
71
+
72
+ dewey_exec = [
73
+ config.dewey_bin,
74
+ [
75
+ ["--tv-dir", tv_dir.inspect()],
76
+ ].map { |args| args.join("=") },
77
+ storage_directory.to_s.inspect(),
78
+ ].join(" ")
79
+ log.detail("Will exec(): " + dewey_exec.inspect())
80
+
81
+ rd, wr = IO.pipe()
82
+ rc = shellout(dewey_exec, nil, wr, wr)
83
+ wr.close() unless wr.closed?
84
+ rd.each_line do |line|
85
+ log.detail(line)
86
+ end
87
+ rd.close() unless rd.closed?
88
+
89
+ if rc
90
+ log.info("Dewey seems happy :-)")
91
+ else
92
+ log.error("Dewey failed with a rc #{rc.inspect()} :-(")
93
+ end
94
+ end
95
+
96
+ install_callback(:dewey) do
97
+ move_tv_shows_to_the_archive()
98
+ end
metadata ADDED
@@ -0,0 +1,111 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: nzbgetpp
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.1rc0
5
+ prerelease: 5
6
+ platform: ruby
7
+ authors:
8
+ - Marc Bowes
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2012-05-09 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: rake
16
+ requirement: &18907960 !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - ! '>='
20
+ - !ruby/object:Gem::Version
21
+ version: '0'
22
+ type: :development
23
+ prerelease: false
24
+ version_requirements: *18907960
25
+ - !ruby/object:Gem::Dependency
26
+ name: rspec
27
+ requirement: &18905760 !ruby/object:Gem::Requirement
28
+ none: false
29
+ requirements:
30
+ - - ! '>='
31
+ - !ruby/object:Gem::Version
32
+ version: '0'
33
+ type: :development
34
+ prerelease: false
35
+ version_requirements: *18905760
36
+ - !ruby/object:Gem::Dependency
37
+ name: rcov
38
+ requirement: &18904720 !ruby/object:Gem::Requirement
39
+ none: false
40
+ requirements:
41
+ - - ! '>='
42
+ - !ruby/object:Gem::Version
43
+ version: '0'
44
+ type: :development
45
+ prerelease: false
46
+ version_requirements: *18904720
47
+ description: A postprocessing script for NzbGet, written in Ruby.
48
+ email:
49
+ - marcbowes+nzbgetpp@gmail.com
50
+ executables:
51
+ - nzbgetpp
52
+ extensions: []
53
+ extra_rdoc_files: []
54
+ files:
55
+ - .document
56
+ - .gitignore
57
+ - .rspec
58
+ - Gemfile
59
+ - Gemfile.lock
60
+ - LICENSE.txt
61
+ - README.md
62
+ - Rakefile
63
+ - VERSION
64
+ - bin/nzbgetpp
65
+ - lib/nzbgetpp.rb
66
+ - lib/nzbgetpp/log.rb
67
+ - lib/nzbgetpp/nzb.rb
68
+ - lib/nzbgetpp/par.rb
69
+ - lib/nzbgetpp/shellout.rb
70
+ - lib/nzbgetpp/version.rb
71
+ - nzbgetpp.gemspec
72
+ - spec/log_spec.rb
73
+ - spec/nzb_spec.rb
74
+ - spec/nzbgetpp_spec.rb
75
+ - spec/par_spec.rb
76
+ - spec/shellout_spec.rb
77
+ - spec/spec_helper.rb
78
+ - spec/support/log.rb
79
+ - spec/support/sample-dst/example job/_brokenlog.txt
80
+ - spec/support/sample-dst/example job/example.rar
81
+ - spec/support/sample-dst/example job/survivor
82
+ - spec/unrar_spec.rb
83
+ - support/config.rb
84
+ homepage: ''
85
+ licenses: []
86
+ post_install_message:
87
+ rdoc_options: []
88
+ require_paths:
89
+ - lib
90
+ required_ruby_version: !ruby/object:Gem::Requirement
91
+ none: false
92
+ requirements:
93
+ - - ! '>='
94
+ - !ruby/object:Gem::Version
95
+ version: '0'
96
+ segments:
97
+ - 0
98
+ hash: 3546351088323564806
99
+ required_rubygems_version: !ruby/object:Gem::Requirement
100
+ none: false
101
+ requirements:
102
+ - - ! '>'
103
+ - !ruby/object:Gem::Version
104
+ version: 1.3.1
105
+ requirements: []
106
+ rubyforge_project: nzbgetpp
107
+ rubygems_version: 1.8.11
108
+ signing_key:
109
+ specification_version: 3
110
+ summary: NzbGet Postprocessor
111
+ test_files: []