file_pipeline 0.0.6 → 0.1.1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 51b495ebce21752d34f27eef33a350539f8918076774063323e603cde82a7726
4
- data.tar.gz: f6bd55e7129fe2308193af2e12630a867cca866ae6ce71dd177a9deea3e2aa00
3
+ metadata.gz: 2669b6e193cbdf383b4db7bf1fe9e16a4519cefb152b394e34310ea9690d4ed6
4
+ data.tar.gz: 5c07a106071954420545dfec51b1110f71027614384e77d553f4132322848faa
5
5
  SHA512:
6
- metadata.gz: 47077cb546d41be57b9c662718710c815dda1c3d77b1834946800d2ddf5472abfde8ecf9e107f58b69f6eb573ea536464a316c5d60a5ae97e94a4d5deb6fbc01
7
- data.tar.gz: 25b006ee9e71a989eea5f22eece86918feabe4f7579cf96425110cf64efe5f0e00ab40335b6002298d552f979894bbcd210869ce8e724c35628306d8da701e1e
6
+ metadata.gz: b193249acfb972b2a0bc167728a2147d9e7c39e4469a4f3808ed9ff23b3db0cdbfa06000e3477a83058918a31174bad854821a95287abfb964965e6db02421d4
7
+ data.tar.gz: 265d513deaf12846e01c767c71a63c481bb0f21a8bc81d7b00e7ba2ea2291afb0c3d99cd9c3830fd37715772a8c807e3455fdfb3c4f14d558e5df5cc9ba52183
data/README.rdoc CHANGED
@@ -54,7 +54,7 @@ instructions on how to create custom operations).
54
54
  ==== Basic set up with default operations
55
55
 
56
56
  To define an operation, pass the class name of the operation in underscore
57
- notation and without the containing module name, and any options to
57
+ notation without the containing module name, and any options to
58
58
  {#define_operation}[rdoc-ref:FilePipeline::Pipeline#define_operation].
59
59
 
60
60
  The example below adds an instance of
@@ -87,8 +87,9 @@ call <tt>#define_operation</tt> with the desired operations and options.
87
87
 
88
88
  When file operations are to be used that are not included in the gem, place
89
89
  the source files for the class definitions in one or more directories and
90
- initialize the Pipeline object with the directory paths. The directories will
91
- be added to the {source directories}[rdoc-ref:FilePipeline.source_directories].
90
+ initialize the Pipeline object with the paths to those directories. The
91
+ directories will be added to the
92
+ {source directories}[rdoc-ref:FilePipeline.source_directories].
92
93
 
93
94
  Directories are added to the source directories in reverse order, so that
94
95
  directories added later will have precedence when searching source files. The
@@ -103,7 +104,7 @@ finally in the included default operations.
103
104
 
104
105
  The basename for source files _must_ be the class name in underscore notation
105
106
  without the containing module name. If, for example, the operation is
106
- <tt>FileOperations::MyOperation</tt>, the source file basename should be
107
+ <tt>FileOperations::MyOperation</tt>, the source file basename has to be
107
108
  <tt>'my_operation.rb'</tt>
108
109
 
109
110
  my_pipeline = FilePipeline::Pipeline.new('~/custom_operations',
@@ -146,7 +147,7 @@ VersionedFile provides access to a files metadata via the
146
147
  {#metadata}[rdoc-ref:FilePipeline::VersionedFile#metadata] method of the
147
148
  versioned file instance.
148
149
 
149
- Both the metadata for the original file and the current (latest) version can
150
+ Metadata for the original file, the current (latest) or an arbitrary version can
150
151
  be accessed:
151
152
 
152
153
  image = FilePipeline::VersionedFile.new('~/image.jpg')
@@ -167,12 +168,13 @@ versions available, pass the <tt>:for_version</tt> option with the symbol
167
168
 
168
169
  Some file operations can comprise metadata; many image processing libraries
169
170
  will not preserve all _Exif_ tags and their values when converting images to
170
- a different format, but only write a small subset of tags to the file they
171
- create. In these cases, the
171
+ a different format, but only write a subset of tags to the file they create.
172
+ In these cases, the
172
173
  {ExifRestoration}[rdoc-ref:FilePipeline::FileOperations::ExifRestoration]
173
- operation can be used to try to restore the tags that have been discarded, but
174
- it can not write all tags. It will store all tags and their values that it could
175
- not write back to the file and return them as captured data.
174
+ operation can be used to try to restore the tags that have been discarded. The
175
+ operation uses Exiftool to write tags, and Exiftool will not write all tags.
176
+ It will store any tags and their values that it could not write back to the file
177
+ and return them as captured data.
176
178
 
177
179
  Likewise, if the
178
180
  {ExifRedaction}[rdoc-ref:FilePipeline::FileOperations::ExifRedaction] is applied
@@ -246,8 +248,7 @@ The <tt>#initialize</tt> method _must_ take an +options+ argument (a hash
246
248
  with a default value, or a <em>double splat</em>) and _must_ be exposed
247
249
  through an <tt>#options</tt> getter method.
248
250
 
249
- The options passed can be any for the file operation to properly configure
250
- a specific instance of a method.
251
+ The options passed can be any to properly configure an instance of the class.
251
252
 
252
253
  This requirement is imposed by the
253
254
  {#define_operation}[rdoc-ref:FilePipeline::Pipeline#define_operation] instance
@@ -294,20 +295,26 @@ The three arguments required for implementations of <tt>#run</tt> are:
294
295
  succession of modified versions has been created.
295
296
 
296
297
  The <em>original file</em> will only be used by file operations that require
297
- it for reference, e.g. to restore file metadata that was compromised by
298
- other file operations.
298
+ it for reference, e.g. to restore or recover file metadata that was compromised
299
+ by other file operations.
299
300
 
300
301
  ===== Return value
301
302
 
302
- The method _must_ return the path to the file that was created by the
303
- operation (perferrably in the _directory_). It _may_ also return a
304
- {Results}[rdoc-ref:FilePipeline::FileOperations::Results] object, containing the
305
- operation itself, a _success_ flag (+true+ or +false+), and any logs or data
306
- returned by the operation.
303
+ If the operation modifies the file (i.e. creates a new version), the +run+
304
+ method _must_ return the path to the file that was created (perferrably in the
305
+ _directory_). If it does not modify and no results are returned, it _must_
306
+ return +nil+.
307
+
308
+ The method _may_ return a
309
+ {Results}[rdoc-ref:FilePipeline::FileOperations::Results] object along with the
310
+ path or +nil+. The results object should contain the operation itself, a
311
+ _success_ flag (+true+ or +false+), and any logs or data returned by the
312
+ operation.
307
313
 
308
314
  If results are returned with the path to the created file, both values must
309
315
  be wrapped in an array, with the path as the first element, the results as
310
- the second.
316
+ the second. If the operation does not modify and therefore not return a path,
317
+ the first element of the array must be +nil+.
311
318
 
312
319
  ===== Example
313
320
 
@@ -365,15 +372,21 @@ logic to perform the actual file operation, but will call an
365
372
  {#operation method}[rdoc-label:label-The+operation+method] that _must_ be
366
373
  defined in the subclass unless the subclass overrides the <tt>#run</tt> method.
367
374
 
368
- The <tt>#run</tt> method will generate the new path that is passed to the
369
- <tt>#operation</tt> method, and to which the latter will write the new
370
- version of the file. The new file path will need an appropriate file type
371
- extension. The default behavior is to assume that the extension will be the
372
- same as for the file that was passed in as the basis from which the new
373
- version will be created. If the operation will result in a different file
374
- type, the subclass _should_ define a <tt>#target_extension</tt> method that
375
- returns the appropriate file extension (see
376
- {Target file extensions}[rdoc-label:label-Target+file+extensions]).
375
+ If the operation is modifying (creates a new version), the <tt>#run</tt> method
376
+ will generate the new path that is passed to the <tt>#operation</tt> method,
377
+ and to which the latter will write the new version of the file. The new file
378
+ path will need an appropriate file type extension. The default behavior is to
379
+ assume that the extension will be the same as for the file that was passed in as
380
+ the basis from which the new version will be created. If the operation will
381
+ result in a different file type, the subclass _should_ define a
382
+ <tt>#target_extension</tt> method that returns the appropriate file extension
383
+ (see {Target file extensions}[rdoc-label:label-Target+file+extensions]).
384
+
385
+ Subclasses of FileOperation are by default modifying. If the operation is not
386
+ modifying (does not create a new version of the file), the subclass _must_
387
+ override the <tt>#modiies?</tt> method or override the <tt>#run</tt> method to
388
+ ensure it does not return a file path (see
389
+ {Non-modifying operations}[rdoc-label:label-Non-modifying+operations]).
377
390
 
378
391
  ==== Initializer
379
392
 
@@ -488,6 +501,16 @@ return the appropriate
488
501
  return
489
502
  end
490
503
 
504
+ ==== Non-modifying operations
505
+
506
+ If the operation will not create a new version, the class _must_ redefine the
507
+ <tt>#modifies?</tt> method to return +false+:
508
+
509
+ # non-modiyfing operation
510
+ def modifies?
511
+ false
512
+ end
513
+
491
514
  ==== Target file extensions
492
515
 
493
516
  If the file that the operation creates is of a different type than the file
data/lib/file_pipeline.rb CHANGED
@@ -4,6 +4,7 @@ require 'securerandom'
4
4
 
5
5
  require_relative 'file_pipeline/errors'
6
6
  require_relative 'file_pipeline/file_operations'
7
+ require_relative 'file_pipeline/versions'
7
8
  require_relative 'file_pipeline/versioned_file'
8
9
  require_relative 'file_pipeline/pipeline'
9
10
 
@@ -22,7 +23,7 @@ module FilePipeline
22
23
  return source_directories if source_directories.include? directory_path
23
24
 
24
25
  no_dir = !File.directory?(directory_path)
25
- raise Errors::SourceDirectoryError, dir: directory if no_dir
26
+ raise Errors::SourceDirectoryError.new dir: directory if no_dir
26
27
 
27
28
  @src_directories.prepend directory_path
28
29
  end
@@ -32,7 +33,7 @@ module FilePipeline
32
33
  def self.load(file_operation)
33
34
  const = file_operation.split('_').map(&:capitalize).join
34
35
  FilePipeline.load_file(file_operation) unless const_defined? const
35
- const_get 'FileOperations::' + const
36
+ const_get "FileOperations::#{const}"
36
37
  rescue NameError
37
38
  # TODO: implement autogenerating module names from file_operation src path
38
39
  const_get const
@@ -44,9 +45,10 @@ module FilePipeline
44
45
  src_file += '.rb' unless src_file.end_with? '.rb'
45
46
  src_path = FilePipeline.source_path src_file
46
47
  if src_path.nil?
47
- raise Errors::SourceFileError,
48
- file: src_file,
49
- directories: FilePipeline.source_directories
48
+ raise Errors::SourceFileError.new(
49
+ file: src_file,
50
+ directories: FilePipeline.source_directories
51
+ )
50
52
  end
51
53
  require src_path
52
54
  end
@@ -1,6 +1,7 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  require_relative 'errors/failed_modification_error'
4
+ require_relative 'errors/misplaced_version_file_error'
4
5
  require_relative 'errors/missing_version_file_error'
5
6
  require_relative 'errors/source_directory_error'
6
7
  require_relative 'errors/source_file_error'
@@ -8,32 +8,64 @@ module FilePipeline
8
8
  # The file opration that caused the error.
9
9
  attr_reader :info
10
10
 
11
- # FIXME: should contain original file name file name!
11
+ # Returns a new instance.
12
+ #
13
+ # ===== Arguments
14
+ #
15
+ # * +msg+ - error message for the exception. If none provided, the
16
+ # instance will be initialized with the #default_message.
17
+ #
18
+ # ===== Options
19
+ #
20
+ # * <tt>info</tt> - a FileOperations::Results object or an object.
21
+ # * <tt>file</tt> - path to the file thas was being processed.
12
22
  def initialize(msg = nil, info: nil, file: nil)
13
23
  @file = file
14
24
  @info = info
15
-
16
- if info.respond_to?(:operation) && info.respond_to?(:log)
17
- msg ||= "#{@info.operation&.name} with options"\
18
- " #{@info.operation&.options} failed on #{file}."
19
- if original_error
20
- msg += "\nException raised by the operation:"\
21
- " #{original_error.inspect}. Backtrace:\n"
22
- msg += original_backtrace if original_backtrace
23
- end
24
- else
25
- msg ||= 'Operation failed' unless info
26
- end
25
+ msg ||= default_message
27
26
  super msg
28
27
  end
29
28
 
29
+ # Returns the backtrace of the error that caused the exception.
30
30
  def original_backtrace
31
31
  original_error&.backtrace&.join("\n")
32
32
  end
33
33
 
34
+ # Returns the error that caused the exception.
34
35
  def original_error
35
36
  @info.log.find { |item| item.is_a? Exception }
36
37
  end
38
+
39
+ private
40
+
41
+ # Appends the backtrace of the error that caused the exception to the
42
+ # #default_message.
43
+ def append_backtrace(str)
44
+ return "#{str}\n" unless original_backtrace
45
+
46
+ "#{str} Backtrace:\n#{original_backtrace}"
47
+ end
48
+
49
+ # Appends the message of the error that caused the exception to the
50
+ # #default_message.
51
+ def append_error(str)
52
+ return str unless original_error
53
+
54
+ str += "\nException raised by the operation:"\
55
+ " #{original_error.inspect}."
56
+ append_backtrace str
57
+ end
58
+
59
+ # Returns a String with the #message for +self+.
60
+ def default_message
61
+ if info.respond_to?(:operation) && info.respond_to?(:log)
62
+ msg = "#{info.operation&.name} with options"\
63
+ " #{info.operation&.options} failed on #{@file}."
64
+ append_error msg
65
+ else
66
+ 'Operation failed'
67
+ end
68
+ end
37
69
  end
38
70
  end
39
71
  end
@@ -0,0 +1,30 @@
1
+ # frozen_string_literal: true
2
+
3
+ module FilePipeline
4
+ module Errors
5
+ # Error class for exceptions that are raised when a new version is added,
6
+ # but the file is not in the VersionedFile's working directory.
7
+ class MisplacedVersionFileError < StandardError
8
+ # Path for of the misplaced file for the version.
9
+ attr_reader :file
10
+
11
+ # Path for the directory where the file should have been (the
12
+ # VersionedFile's working directory).
13
+ attr_reader :directory
14
+
15
+ def initialize(msg = nil, file: nil, directory: nil)
16
+ @file = file
17
+ @directory = directory
18
+ msg ||= default_message
19
+ super msg
20
+ end
21
+
22
+ private
23
+
24
+ def default_message
25
+ "File #{File.basename @file} was expected in #{@directory},"\
26
+ " but was in #{File.dirname @file}."
27
+ end
28
+ end
29
+ end
30
+ end
@@ -0,0 +1,53 @@
1
+ # frozen_string_literal: true
2
+
3
+ module FilePipeline
4
+ module FileOperations
5
+ # A non-modifying FileOperation that compares a file's _Exif_ Metadata with
6
+ # that of a reference file and returns tags missing in the working file as
7
+ # captured data.
8
+ #
9
+ # Used to recover _Exif_ tags that were not preserved during e.g. a file
10
+ # conversion.
11
+ class ExifRecovery < FileOperation
12
+ include ExifManipulable
13
+
14
+ # :args: options
15
+ #
16
+ # Returns a new instance.
17
+ #
18
+ # ===== Options
19
+ #
20
+ # * <tt>skip_tags</tt> - _Exif_ tags to be ignored during comparison.
21
+ #
22
+ # The ExifManipulable mixin defines a set of _Exif_
23
+ # {tags}[rdoc-ref:FilePipeline::FileOperations::ExifManipulable.file_tags]
24
+ # that will always be ignored.
25
+ def initialize(**opts)
26
+ defaults = { skip_tags: [] }
27
+ super(opts, defaults)
28
+ @options[:skip_tags] += ExifManipulable.file_tags
29
+ end
30
+
31
+ # Returns the DROPPED_EXIF_DATA tag defined in CapturedDataTags.
32
+ #
33
+ # Instances of ExifRecovery will capture any _Exif_ tags and their values
34
+ # that are present in the reference file but missing in the working file.
35
+ def captured_data_tag
36
+ CapturedDataTags::DROPPED_EXIF_DATA
37
+ end
38
+
39
+ # Instances of ExifRecovery do not modify the working file.
40
+ def modifies?
41
+ false
42
+ end
43
+
44
+ # Compares the _Exif_ metadata of <tt>src_file</tt> with that of
45
+ # +original+ and returns all tags that are present in +original+ but
46
+ # missing in <tt>src_file</tt>.
47
+ def operation(src_file, _, original)
48
+ original_exif, src_file_exif = read_exif original, src_file
49
+ missing_exif_fields(src_file_exif, original_exif)
50
+ end
51
+ end
52
+ end
53
+ end
@@ -2,9 +2,12 @@
2
2
 
3
3
  module FilePipeline
4
4
  module FileOperations
5
- # A FileOperation that compares Exif Metadata in two files and copies tags
6
- # missing in one from the other. Used to restore Exif tags that were not
7
- # preserved during e.g. a file conversion.
5
+ # A modifying FileOperation that compares a file's Exif Metadata with that
6
+ # of a reference file and attempts to copy tags missing in the working file
7
+ # from the reference file.
8
+ #
9
+ # Used to restore Exif tags that were not preserved during e.g. a file
10
+ # conversion.
8
11
  #
9
12
  # *Caveat:* if this operation is applied to a file together with
10
13
  # ExifRedaction, it should be applied _before_ the latter, to avoid
@@ -20,10 +23,9 @@ module FilePipeline
20
23
  #
21
24
  # * <tt>skip_tags</tt> - _Exif_ tags to be ignored during restoration.
22
25
  #
23
- # The ExifManipulable mixin defines a set of _Exif_ tags that will always
24
- # be ignored. These are tags relating to the file properties (e.g.
25
- # filesize, MIME-type) that will have been altered by any prior operation,
26
- # such as file format conversions.
26
+ # The ExifManipulable mixin defines a set of _Exif_
27
+ # {tags}[rdoc-ref:FilePipeline::FileOperations::ExifManipulable.file_tags]
28
+ # that will always be ignored.
27
29
  def initialize(**opts)
28
30
  defaults = { skip_tags: [] }
29
31
  super(opts, defaults)
@@ -38,8 +40,6 @@ module FilePipeline
38
40
  CapturedDataTags::DROPPED_EXIF_DATA
39
41
  end
40
42
 
41
- # :args: src_file, out_file
42
- #
43
43
  # Writes a new version of <tt>src_file</tt> to <tt>out_file</tt> with all
44
44
  # writable _Exif_ tags from +original+ restored.
45
45
  #
@@ -34,7 +34,7 @@ module FilePipeline
34
34
  def operation(*args)
35
35
  src_file, out_file = args
36
36
  image = Vips::Image.new_from_file src_file
37
- image.tiffsave(out_file, options)
37
+ image.tiffsave(out_file, **options)
38
38
  # Return lof if any
39
39
  end
40
40
 
@@ -9,6 +9,10 @@ module FilePipeline
9
9
  # Returns an Array of tags to be ignored during comparison. These can
10
10
  # be merged with an ExifManipulable including FileOperation's options
11
11
  # to skip tags (e.g. the <tt>skip_tags</tt> option in ExifRestoration).
12
+ #
13
+ # The included tags relate to the file properties (e.g. filesize,
14
+ # MIME-type) that will have been altered by any prior operation, such as
15
+ # file format conversions.
12
16
  def self.file_tags
13
17
  %w[FileSize FileModifyDate FileAccessDate FileInodeChangeDate
14
18
  FilePermissions FileType FileTypeExtension MIMEType]
@@ -20,7 +24,7 @@ module FilePipeline
20
24
  end
21
25
 
22
26
  def self.strip_path(str) # :nodoc:
23
- str.sub(%r{ - \/?(\/|[-:.]+|\w+)+\.\w+$}, '')
27
+ str.sub(%r{ - /?(/|[-:.]+|\w+)+\.\w+$}, '')
24
28
  end
25
29
 
26
30
  # Redacts (deletes) all <tt>tags_to_delete</tt> in <tt>out_file</tt>.
@@ -18,6 +18,9 @@ module FilePipeline
18
18
  # that is passed to #run or #operation as <tt>src_file</tt>, the subclass
19
19
  # must have a #target_extension method that returns the appropriate
20
20
  # extension.
21
+ #
22
+ # If the operation is non-modifying, the subclass must redefine the
23
+ # #modifies? methods to return +false+.
21
24
  class FileOperation
22
25
  # A Hash; any options used when performing #operation.
23
26
  attr_reader :options
@@ -62,6 +65,12 @@ module FilePipeline
62
65
  results false, log_data
63
66
  end
64
67
 
68
+ # Returns +true+ if the FIleOperation will create a new version.
69
+ # _Default:_ +true+.
70
+ def modifies?
71
+ true
72
+ end
73
+
65
74
  # Returns the class name (string) of +self+ _without_ the names of the
66
75
  # modules that the class is nested in.
67
76
  def name
@@ -133,11 +142,11 @@ module FilePipeline
133
142
  # e.g. when exif metadata tags missing in the <tt>src_file</tt> are to
134
143
  # be copied over from another file.
135
144
  def run(src_file, directory, original = nil)
136
- out_file = target directory, extension(src_file)
145
+ out_file = target directory, extension(src_file) if modifies?
137
146
  log_data = operation src_file, out_file, original
138
147
  [out_file, success(log_data)]
139
148
  rescue StandardError => e
140
- FileUtils.rm out_file if File.exist? out_file
149
+ FileUtils.rm out_file if out_file && File.exist?(out_file)
141
150
  [out_file, failure(e)]
142
151
  end
143
152
 
@@ -41,10 +41,7 @@ module FilePipeline
41
41
  # Adds a file operation object #file_operations. The object must implement
42
42
  # a _run_ method (see FileOperations::FileOperation#run for details).
43
43
  def <<(file_operation_instance)
44
- unless file_operation_instance.respond_to? :run
45
- raise TypeError, 'File operations must implement a #run method'
46
- end
47
-
44
+ validate file_operation_instance
48
45
  @file_operations << file_operation_instance
49
46
  end
50
47
 
@@ -88,7 +85,7 @@ module FilePipeline
88
85
  #
89
86
  def define_operation(file_operation, options = {})
90
87
  operation = FilePipeline.load file_operation
91
- self << operation.new(options)
88
+ self << operation.new(**options)
92
89
  self
93
90
  end
94
91
 
@@ -106,5 +103,15 @@ module FilePipeline
106
103
  operation.run version, directory, original
107
104
  end
108
105
  end
106
+
107
+ private
108
+
109
+ # Raises TypeError if <tt>file_operation_instance</tt> does not implement a
110
+ # #run method.
111
+ def validate(file_operation_instance)
112
+ return file_operation_instance if file_operation_instance.respond_to? :run
113
+
114
+ raise TypeError, 'File operations must implement a #run method'
115
+ end
109
116
  end
110
117
  end
@@ -1,5 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require_relative 'versions/validator'
3
4
  module FilePipeline
4
5
  # VersionedFile creates a directory where it stores any versions of _file_.
5
6
  class VersionedFile
@@ -19,6 +20,35 @@ module FilePipeline
19
20
  # by #finalize is not replacing the original.
20
21
  attr_reader :target_suffix
21
22
 
23
+ extend Forwardable
24
+
25
+ # Returns a two-dimesnional array, where each nested array has two items;
26
+ # the file operation object and data captured by the operartion (if any).
27
+ #
28
+ # <tt>[[description_object, data_or_nil], ...]</tt>
29
+ delegate captured_data: :history
30
+
31
+ # Returns any data captured by <tt>operation_name</tt>.
32
+ #
33
+ # If multiple instances of one operation class have modified the file,
34
+ # pass any +options+ the specific instance of the operation was initialized
35
+ # with as the optional second argument.
36
+ delegate captured_data_for: :history
37
+
38
+ # Returns an array with all data captured by operations with +tag+.
39
+ #
40
+ # Tags are defined in FileOperations::CapturedDataTags
41
+ delegate captured_data_with: :history
42
+
43
+ # Returns an array of triplets (arryas with three items each): the name of
44
+ # the file operation class (a string), options (a hash), and the actual log
45
+ # (an array).
46
+ delegate log: :history
47
+
48
+ # Returns an array with paths to the version files of +self+ (excluding
49
+ # #original).
50
+ delegate versions: :history
51
+
22
52
  # Returns a new instance with +file+ as the #original.
23
53
  #
24
54
  # ===== Arguments
@@ -41,23 +71,10 @@ module FilePipeline
41
71
 
42
72
  @original = file
43
73
  @basename = File.basename(file, '.*')
44
- @history = {}
74
+ @history = Versions::History.new
45
75
  @directory = nil
46
76
  @target_suffix = target_suffix
47
- end
48
-
49
- # Copies the file with path _src_ to <em>/dir/filename</em>.
50
- def self.copy(src, dir, filename)
51
- dest = FilePipeline.path(dir, filename)
52
- FileUtils.cp src, dest
53
- dest
54
- end
55
-
56
- # Moves the file with path _src_ to <em>/dir/filename</em>.
57
- def self.move(src, dir, filename)
58
- dest = FilePipeline.path(dir, filename)
59
- FileUtils.mv src, dest
60
- dest
77
+ history[original] = nil
61
78
  end
62
79
 
63
80
  # Adds a new version to #history and returns _self_.
@@ -65,14 +82,10 @@ module FilePipeline
65
82
  # <tt>version_info</tt> must be a path to an existing file or an array with
66
83
  # the path and optionally a FileOperations::Results instance:
67
84
  # <tt>['path/to/file', results_object]</tt>.
68
- # Will move the file to #directory if it is in another directory.
85
+ # Will raise MisplacedVersionFileError if it is in another directory.
69
86
  def <<(version_info)
70
- file, info = version_info
71
- if info&.failure
72
- raise Errors::FailedModificationError, info: info, file: original
73
- end
74
-
75
- version = validate(file)
87
+ version, info = Versions::Validator[version_info, self]
88
+ version ||= @current
76
89
  @history[version] = info
77
90
  self
78
91
  rescue StandardError => e
@@ -80,37 +93,6 @@ module FilePipeline
80
93
  raise e
81
94
  end
82
95
 
83
- # Returns a two-dimesnional array, where each nested array has two items;
84
- # the file operation object and data captured by the operartion (if any).
85
- #
86
- # <tt>[[description_object, data_or_nil], ...]</tt>
87
- def captured_data
88
- filter_history :data
89
- end
90
-
91
- # Returns any data captured by <tt>operation_name</tt>.
92
- #
93
- # If multiple instances of one operation class have modified the file,
94
- # pass any +options+ the specific instance of the operation was initialized
95
- # with as the optional second argument.
96
- def captured_data_for(operation_name, **options)
97
- raw_data = captured_data.filter do |operation, _|
98
- operation.name == operation_name &&
99
- options.all? { |k, v| operation.options[k] == v }
100
- end
101
- raw_data.map(&:last)
102
- end
103
-
104
- # Returns an array with all data captured by operations with +tag+ has.
105
- #
106
- # Tags are defined in FileOperations::CapturedDataTags
107
- def captured_data_with(tag)
108
- return unless changed?
109
-
110
- captured_data.select { |operation, _| operation.captured_data_tag == tag }
111
- .map(&:last)
112
- end
113
-
114
96
  # Returns +true+ if there are #versions (file has been modified).
115
97
  #
116
98
  # *Warning:* It will also return +true+ if the file has been cloned.
@@ -122,7 +104,7 @@ module FilePipeline
122
104
  # the file to history, but no FileOperations::Results.
123
105
  def clone
124
106
  filename = FilePipeline.new_basename + current_extension
125
- clone_file = VersionedFile.copy(current, directory, filename)
107
+ clone_file = Versions.copy(current, directory, filename)
126
108
  self << clone_file
127
109
  end
128
110
 
@@ -159,21 +141,15 @@ module FilePipeline
159
141
  # * +true+ - The finalized version will replace the #original.
160
142
  def finalize(overwrite: false)
161
143
  yield(self) if block_given?
162
- filename = overwrite ? replacing_trarget : preserving_taget
144
+ return original unless changed?
145
+
146
+ filename = overwrite ? replacing_target : preserving_target
163
147
  FileUtils.rm original if overwrite
164
- @original = VersionedFile.copy(current, original_dir, filename)
148
+ @original = Versions.copy(current, original_dir, filename)
165
149
  ensure
166
150
  reset
167
151
  end
168
152
 
169
- # Returns an array of triplets (arryas with three items each): the name of
170
- # the file operation class (a string), options (a hash), and the actual log
171
- # (an array).
172
- def log
173
- filter_history(:log)
174
- .map { |operation, info| [operation.name, operation.options, info] }
175
- end
176
-
177
153
  # Returns the Exif metadata
178
154
  #
179
155
  # ===== Options
@@ -186,10 +162,14 @@ module FilePipeline
186
162
  #--
187
163
  # TODO: when file is not an image file, this should return other metadata
188
164
  # than exif.
189
- # TODO: implement the option to return metadata for a specif version index
190
165
  #++
191
166
  def metadata(for_version: :current)
192
- file = public_send for_version
167
+ file = case for_version
168
+ when :current, :original
169
+ public_send for_version
170
+ else
171
+ for_version
172
+ end
193
173
  read_exif(file).first
194
174
  end
195
175
 
@@ -218,54 +198,32 @@ module FilePipeline
218
198
  # Returns a hash into which all captured data from file operations with the
219
199
  # FileOperations::CapturedDataTags::DROPPED_EXIF_DATA has been merged.
220
200
  def recovered_metadata
201
+ return unless changed?
202
+
221
203
  captured_data_with(FileOperations::CapturedDataTags::DROPPED_EXIF_DATA)
222
204
  &.reduce({}) { |recovered, data| recovered.merge data }
223
205
  end
224
206
 
225
- # Returns an array with paths to the version files of +self+ (excluding
226
- # #original).
227
- def versions
228
- history.keys
229
- end
230
-
231
207
  alias touch clone
232
208
 
233
209
  private
234
210
 
235
- # item = :data or :log
236
- def filter_history(item)
237
- history.inject([]) do |results, (_, info)|
238
- next results unless info.respond_to?(item) && info.public_send(item)
239
-
240
- results << [info.operation, info.public_send(item)]
241
- end
242
- end
243
-
244
211
  # Returns the filename for a target file that will not overwrite the
245
212
  # original.
246
- def preserving_taget
247
- basename + '_' + target_suffix + current_extension
213
+ def preserving_target
214
+ "#{basename}_#{target_suffix}#{current_extension}"
248
215
  end
249
216
 
250
217
  # Returns the filename for a target file that will overwrite the
251
218
  # original.
252
- def replacing_trarget
219
+ def replacing_target
253
220
  basename + current_extension
254
221
  end
255
222
 
256
223
  # Deletes the work directory and resets #versions
257
224
  def reset
258
225
  FileUtils.rm_r directory, force: true
259
- @history = {}
260
- end
261
-
262
- # Validates if file exists and has been stored in #directory.
263
- def validate(file)
264
- raise Errors::MissingVersionFileError, file: file unless File.exist? file
265
-
266
- return file if File.dirname(file) == directory
267
-
268
- VersionedFile.move file, directory, File.basename(file)
226
+ history.clear!
269
227
  end
270
228
 
271
229
  # Creates the directory containing all version files. Directory name is
@@ -273,9 +231,8 @@ module FilePipeline
273
231
  #
274
232
  # Raises SystemCallError if the directory already exists.
275
233
  def workdir
276
- subdir = basename + '_versions'
277
234
  filedir = File.dirname(original)
278
- dirname = File.join filedir, subdir
235
+ dirname = File.join filedir, "#{basename}_versions"
279
236
  FileUtils.mkdir(dirname)
280
237
  File.path dirname
281
238
  end
@@ -0,0 +1,22 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'versions/history'
4
+
5
+ module FilePipeline
6
+ # Module that contains classes to work with VersionedFile.
7
+ module Versions
8
+ # Copies the file with path _src_ to <em>/dir/filename</em>.
9
+ def self.copy(src, dir, filename)
10
+ dest = FilePipeline.path(dir, filename)
11
+ FileUtils.cp src, dest
12
+ dest
13
+ end
14
+
15
+ # Moves the file with path _src_ to <em>/dir/filename</em>.
16
+ def self.move(src, dir, filename)
17
+ dest = FilePipeline.path(dir, filename)
18
+ FileUtils.mv src, dest
19
+ dest
20
+ end
21
+ end
22
+ end
@@ -0,0 +1,106 @@
1
+ # frozen_string_literal: true
2
+
3
+ module FilePipeline
4
+ module Versions
5
+ # History objects keep track of a VersionedFile instances versions names and
6
+ # any associated logs or data for each version.
7
+ class History
8
+ # Returns a new instance.
9
+ def initialize
10
+ @entries = {}
11
+ end
12
+
13
+ # Retrieves the _results_ object for the <tt>version_name</tt>.
14
+ def [](version_name)
15
+ @entries[version_name]
16
+ end
17
+
18
+ # Associates the +results+ with the <tt>version_name</tt>.
19
+ def []=(version_name, results)
20
+ entry = @entries.fetch version_name, []
21
+ entry << results
22
+ @entries[version_name] = entry.compact
23
+ end
24
+
25
+ # Returns a two-dimensional array, where each nested array has two items:
26
+ # * the file operation object
27
+ # * data captured by the operartion (if any).
28
+ #
29
+ # <tt>[[file_operation_object, data_or_nil], ...]</tt>
30
+ def captured_data
31
+ filter :data
32
+ end
33
+
34
+ # Returns any data captured by <tt>operation_name</tt>.
35
+ #
36
+ # If multiple instances of one operation class have modified the file,
37
+ # pass any +options+ the specific instance of the operation was
38
+ # initialized with as the optional second argument.
39
+ def captured_data_for(operation_name, **options)
40
+ return if empty?
41
+
42
+ captured_data.filter { |op, _| matches? op, operation_name, options }
43
+ .map(&:last)
44
+ end
45
+
46
+ # Returns an array with all data captured by operations with +tag+.
47
+ # Returns an empty array if there is no data for +tag+.
48
+ #
49
+ # Tags are defined in FileOperations::CapturedDataTags
50
+ def captured_data_with(tag)
51
+ captured_data.filter { |op, _| op.captured_data_tag == tag }
52
+ .map(&:last)
53
+ end
54
+
55
+ # Clears all history entries (version names and associated results).
56
+ def clear!
57
+ @entries.clear
58
+ end
59
+
60
+ # Returns +true+ if +self+ has no entries (version names and associated
61
+ # results), +true+ otherwise.
62
+ def empty?
63
+ @entries.empty?
64
+ end
65
+
66
+ # Returns an array of triplets (arryas with three items each):
67
+ # * Name of the file operation class (String).
68
+ # * Options for the file operation instance (Hash).
69
+ # * The log (Array).
70
+ def log
71
+ filter(:log).map { |op, results| [op.name, op.options, results] }
72
+ end
73
+
74
+ # Returns a two-dimensional Array where every nested Array will consist
75
+ # of the version name (file path) at index +0+ and +nil+ or an Array with
76
+ # all _results_ objects for the version at index +1+:
77
+ #
78
+ # <tt>[version_name, [results1, ...]]</tt>
79
+ def to_a
80
+ @entries.to_a
81
+ end
82
+
83
+ # Returns an array with paths to the version files of +self+ (excluding
84
+ # #original).
85
+ def versions
86
+ @entries.keys
87
+ end
88
+
89
+ private
90
+
91
+ # Filters entries in self by +item+ (<tt>:log</tt> or <tt>:data</tt>).
92
+ def filter(item)
93
+ @entries.values.flatten.select(&item).map do |results|
94
+ [results.operation, results.public_send(item)]
95
+ end
96
+ end
97
+
98
+ # Returns +true+ if +name+ matches the _name_ attribute of +operation+ and
99
+ # +options+ matches the options the operation instance is initialized
100
+ # with.
101
+ def matches?(operation, name, opts)
102
+ operation.name == name && opts.all? { |k, v| operation.options[k] == v }
103
+ end
104
+ end
105
+ end
106
+ end
@@ -0,0 +1,85 @@
1
+ # frozen_string_literal: true
2
+
3
+ module FilePipeline
4
+ module Versions
5
+ # Validator objects verify the version file and results returned by a
6
+ # FileOperation.
7
+ #
8
+ # They will validate:
9
+ # - that the version file existst
10
+ # - that it is in the correct directory
11
+ # - that the file operation has not returned any failures
12
+ class Validator
13
+ extend Forwardable
14
+
15
+ # File for the version that resulted from a FileOperation.
16
+ attr_reader :file
17
+
18
+ # FileOperation::Results object.
19
+ attr_reader :info
20
+
21
+ # Returns a new instance.
22
+ #
23
+ # ===== Arguments
24
+ #
25
+ # * <tt>version_info</tt> - path to an existing file or an array with the
26
+ # path and optionally a FileOperations::Results instance.
27
+ # * +directory+ - directory where the file is expected (the working
28
+ # directory of a VersionedFile).
29
+ # * +filename+ - name of the file to be returned if the file operation was
30
+ # was non-modifying (usually the VersionedFile#original).
31
+ def initialize(version_info, directory, filename)
32
+ @file, @info = [version_info].flatten
33
+ @directory = directory
34
+ @filename = filename
35
+ end
36
+
37
+ # Validates file, directory, and info for <tt>version_info</tt> in the
38
+ # context of <tt>versioned_file</tt>.
39
+ #
40
+ # ===== Arguments
41
+ #
42
+ # * <tt>version_info</tt> - path to an existing file or an array with the
43
+ # path and optionally a FileOperations::Results instance.
44
+ # * <tt>versioned_file</tt> - an object that responds to #original and
45
+ # returns a file path, and #directory and returns a directory path.
46
+ def self.[](version_info, versioned_file)
47
+ new(version_info, versioned_file.directory, versioned_file.original)
48
+ .validate_info
49
+ .validate_file
50
+ .validate_directory
51
+ .then { |validator| [validator.file, validator.info] }
52
+ end
53
+
54
+ # Returns +true+ when there is no file for the version (result of a
55
+ # non-modifying file operation), +false+ otherwise.
56
+ def unmodified?
57
+ @file.nil?
58
+ end
59
+
60
+ # Raises MisplacedVersionFileError if #file is not in #directory.
61
+ def validate_directory
62
+ return self if unmodified? || File.dirname(@file) == @directory
63
+
64
+ raise Errors::MisplacedVersionFileError.new file: @file,
65
+ directory: @directory
66
+ end
67
+
68
+ # Raises MissingVersionFileError if #file does not exist on the file
69
+ # system.
70
+ def validate_file
71
+ return self if unmodified? || File.exist?(@file)
72
+
73
+ raise Errors::MissingVersionFileError.new file: @file
74
+ end
75
+
76
+ # Raises FailedModificationError if the file operation generatint the
77
+ # #info failed.
78
+ def validate_info
79
+ return self unless @info&.failure
80
+
81
+ raise Errors::FailedModificationError.new info: @info, file: @filename
82
+ end
83
+ end
84
+ end
85
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: file_pipeline
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.6
4
+ version: 0.1.1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Martin Stein
8
- autorequire:
8
+ autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-11-19 00:00:00.000000000 Z
11
+ date: 2021-02-16 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: multi_exiftool
@@ -16,29 +16,29 @@ dependencies:
16
16
  requirements:
17
17
  - - "~>"
18
18
  - !ruby/object:Gem::Version
19
- version: 0.11.0
19
+ version: 0.16.0
20
20
  type: :runtime
21
21
  prerelease: false
22
22
  version_requirements: !ruby/object:Gem::Requirement
23
23
  requirements:
24
24
  - - "~>"
25
25
  - !ruby/object:Gem::Version
26
- version: 0.11.0
26
+ version: 0.16.0
27
27
  - !ruby/object:Gem::Dependency
28
28
  name: ruby-vips
29
29
  requirement: !ruby/object:Gem::Requirement
30
30
  requirements:
31
31
  - - "~>"
32
32
  - !ruby/object:Gem::Version
33
- version: 2.0.16
33
+ version: 2.0.17
34
34
  type: :runtime
35
35
  prerelease: false
36
36
  version_requirements: !ruby/object:Gem::Requirement
37
37
  requirements:
38
38
  - - "~>"
39
39
  - !ruby/object:Gem::Version
40
- version: 2.0.16
41
- description: The file_pipeline gem provides a framework fornondestructive application
40
+ version: 2.0.17
41
+ description: The file_pipeline gem provides a framework for nondestructive application
42
42
  of file operation batches to files.
43
43
  email: loveablelobster@fastmail.fm
44
44
  executables: []
@@ -50,11 +50,13 @@ files:
50
50
  - lib/file_pipeline.rb
51
51
  - lib/file_pipeline/errors.rb
52
52
  - lib/file_pipeline/errors/failed_modification_error.rb
53
+ - lib/file_pipeline/errors/misplaced_version_file_error.rb
53
54
  - lib/file_pipeline/errors/missing_version_file_error.rb
54
55
  - lib/file_pipeline/errors/source_directory_error.rb
55
56
  - lib/file_pipeline/errors/source_file_error.rb
56
57
  - lib/file_pipeline/file_operations.rb
57
58
  - lib/file_pipeline/file_operations/captured_data_tags.rb
59
+ - lib/file_pipeline/file_operations/default_operations/exif_recovery.rb
58
60
  - lib/file_pipeline/file_operations/default_operations/exif_redaction.rb
59
61
  - lib/file_pipeline/file_operations/default_operations/exif_restoration.rb
60
62
  - lib/file_pipeline/file_operations/default_operations/ptiff_conversion.rb
@@ -65,11 +67,14 @@ files:
65
67
  - lib/file_pipeline/file_operations/results.rb
66
68
  - lib/file_pipeline/pipeline.rb
67
69
  - lib/file_pipeline/versioned_file.rb
70
+ - lib/file_pipeline/versions.rb
71
+ - lib/file_pipeline/versions/history.rb
72
+ - lib/file_pipeline/versions/validator.rb
68
73
  homepage: https://github.com/loveablelobster/file_pipeline
69
74
  licenses:
70
75
  - MIT
71
76
  metadata: {}
72
- post_install_message:
77
+ post_install_message:
73
78
  rdoc_options: []
74
79
  require_paths:
75
80
  - lib
@@ -77,15 +82,15 @@ required_ruby_version: !ruby/object:Gem::Requirement
77
82
  requirements:
78
83
  - - ">="
79
84
  - !ruby/object:Gem::Version
80
- version: '0'
85
+ version: '3.0'
81
86
  required_rubygems_version: !ruby/object:Gem::Requirement
82
87
  requirements:
83
88
  - - ">="
84
89
  - !ruby/object:Gem::Version
85
90
  version: '0'
86
91
  requirements: []
87
- rubygems_version: 3.0.6
88
- signing_key:
92
+ rubygems_version: 3.2.3
93
+ signing_key:
89
94
  specification_version: 4
90
95
  summary: Nondestructive file processing with a defined batch
91
96
  test_files: []