bitferry 0.0.1 → 0.0.2

Sign up to get free protection for your applications and to get access to all the features.
Files changed (6) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGES.md +8 -0
  3. data/README.md +3 -5
  4. data/lib/bitferry/cli.rb +14 -14
  5. data/lib/bitferry.rb +392 -389
  6. metadata +4 -3
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 07536a8e52281f9dbe3a085aa9bedff6164f116babcb163149e8a13e2214f771
4
- data.tar.gz: a81dbb898d67e9c9e2083871c69bfd6396e56caa12904b085cf8e7c5e85248d5
3
+ metadata.gz: '08ca0d191eb9905a93c35437fba08ccd3ef79ec0f04d1b64b9ac8e48582fd83a'
4
+ data.tar.gz: f7fdb4592f72f72a532dfbf48f362b7fa8d3b4141a11cd8483d2e98c56a9f1bd
5
5
  SHA512:
6
- metadata.gz: c8025822e4520ec87254036acd3b7c6f3933949b3a0737cc9d12d91d5ae4d5fa5d98487a9f2d60a18c15cfb9d4009212b35ead99ce415ececdc42b48c03bcd7f
7
- data.tar.gz: 1af8e4d44dd64818f78eadb27b62233e33bd4f7f1cc493dfafbcb7ca4defd77e5ea868ad5060ee28e6eefc3f4c122842e7774b4627506305ee553e12157ea3c0
6
+ metadata.gz: 3fb6e3700029f5ca720228aa3dd6d61b21949322b4ee3d497597b586715962f9e46ebda45d2eec07d1782dbccd7a5b3c3a6b444f832d7060e88f74301748b6db
7
+ data.tar.gz: 75fcc7c1724b5219eecc3a421a5ff4bcbd90018f5e74eb7491be260faebf9e5686021768be785bfe5f59a5aa517fd766688f443083adf7c8dfc8daeaecbbea43
data/CHANGES.md ADDED
@@ -0,0 +1,8 @@
1
+ ## 0.0.2
2
+
3
+ - Fix infinite nesting of directories during GEM installation
4
+ - Doc updates
5
+
6
+ ## 0.0.1
7
+
8
+ - Initial release
data/README.md CHANGED
@@ -1,4 +1,4 @@
1
- # Bitferry - file synchronization/backup automation tool
1
+ # Bitferry - file synchronization/backup automation
2
2
 
3
3
  <div align="right"><i>Ein Backup ist kein Backup</i></div><br><br>
4
4
 
@@ -6,8 +6,9 @@ The [Bitferry](https://github.com/okhlybov/bitferry) is aimed at establishing th
6
6
 
7
7
  The intended usage ranges from maintaining simple directory copy to another location (disk, mount point) to complex many-to-many (online/offline) data replication/backup solution employing portable media as additional data storage and a means of data propagation between the offsites.
8
8
 
9
- Bitferry is a frontend to the [Rclone](https://rclone.org) and [Restic](https://restic.net) utilities.
9
+ The core idea that drives Bitferry is the conversion of full (absolute) endpoint's paths into the volume-relative ones, where the volume is a data file which is put along the endpoint's data and denotes the root of the directory hierarchy. This leads to the important location independence property meaning that Bitferry is then able to restore the tasks' source-destination endpoint connections in spite of the volume location changes, which is a likely scenario in case of portable storage (different UNIX mount points, Windows drives etc.).
10
10
 
11
+ Bitferry is effectively a frontend to the [Rclone](https://rclone.org) and [Restic](https://restic.net) utilities.
11
12
 
12
13
  ## Features
13
14
 
@@ -27,7 +28,6 @@ Bitferry is a frontend to the [Rclone](https://rclone.org) and [Restic](https://
27
28
 
28
29
  * Offline portable storage (USB flash, HDDs, SSDs etc.) relay
29
30
 
30
-
31
31
  ## Use cases
32
32
 
33
33
  * Maintain an update-only files copy in a separate location on the same site
@@ -36,7 +36,6 @@ Bitferry is a frontend to the [Rclone](https://rclone.org) and [Restic](https://
36
36
 
37
37
  * Maintain an incremental files backup on a portable medium with multiple offsite copies of the repository
38
38
 
39
-
40
39
  ## Implementation
41
40
 
42
41
  The Bitferry itself is written in [Ruby](https://www.ruby-lang.org) programming language. Being a Ruby code, the Bitferry requires the platform-specific Ruby runtime, version 3.0 or higher.
@@ -45,7 +44,6 @@ The source code is hosted on [GitHub](https://github.com/okhlybov/bitferry) and
45
44
 
46
45
  In addition, the platform-specific [Rclone](https://github.com/rclone/rclone/releases) and [Restic](https://github.com/restic/restic/releases) executables are required to be accessible through the `PATH` directory list or through the respective `RCLONE` and `RESTIC` environment variables.
47
46
 
48
-
49
47
  ## Kickstart
50
48
 
51
49
  Install Bitferry
data/lib/bitferry/cli.rb CHANGED
@@ -165,14 +165,14 @@ Clamp do
165
165
  subcommand ['copy', 'c'], 'Create copy task' do
166
166
  banner %{
167
167
  Create source --> destination file copy task.
168
-
168
+
169
169
  The task operates recursively on two specified endpoints.
170
170
  This task unconditionally copies all source files overwriting existing files in destination.
171
-
171
+
172
172
  #{Endpoint}
173
-
173
+
174
174
  #{Encryption}
175
-
175
+
176
176
  This task employs the Rclone worker.
177
177
  }
178
178
  setup_rclone_task(self)
@@ -185,12 +185,12 @@ Clamp do
185
185
  subcommand ['update', 'u'], 'Create update task' do
186
186
  banner %{
187
187
  Create source --> destination file update (freshen) task.
188
-
188
+
189
189
  The task operates recursively on two specified endpoints.
190
190
  This task copies newer source files while skipping unchanged files in destination.
191
-
191
+
192
192
  #{Endpoint}
193
-
193
+
194
194
  #{Encryption}
195
195
 
196
196
  This task employs the Rclone worker.
@@ -205,15 +205,15 @@ Clamp do
205
205
  subcommand ['synchronize', 'sync', 's'], 'Create one way sync task' do
206
206
  banner %{
207
207
  Create source --> destination one way file synchronization task.
208
-
208
+
209
209
  The task operates recursively on two specified endpoints.
210
210
  This task copies newer source files while skipping unchanged files in destination.
211
211
  Also, it deletes destination files which are non-existent in source.
212
-
212
+
213
213
  #{Endpoint}
214
214
 
215
215
  #{Encryption}
216
-
216
+
217
217
  This task employs the Rclone worker.
218
218
  }
219
219
  setup_rclone_task(self)
@@ -226,15 +226,15 @@ Clamp do
226
226
  subcommand ['equalize', 'bisync', 'e'], 'Create two way sync task' do
227
227
  banner %{
228
228
  Create source <-> destination two way file synchronization task.
229
-
229
+
230
230
  The task operates recursively on two specified endpoints.
231
231
  This task retains only the most recent versions of files on both endpoints.
232
232
  Opon execution both endpoints are left identical.
233
-
233
+
234
234
  #{Endpoint}
235
235
 
236
236
  #{Encryption}
237
-
237
+
238
238
  This task employs the Rclone worker.
239
239
  }
240
240
  setup_rclone_task(self)
@@ -341,4 +341,4 @@ Clamp do
341
341
  end
342
342
 
343
343
 
344
- end
344
+ end
data/lib/bitferry.rb CHANGED
@@ -12,10 +12,9 @@ require 'shellwords'
12
12
  module Bitferry
13
13
 
14
14
 
15
- VERSION = '0.0.1'
15
+ VERSION = '0.0.2'
16
16
 
17
17
 
18
- # :nodoc:
19
18
  module Logging
20
19
  def self.log
21
20
  unless @log
@@ -488,7 +487,7 @@ module Bitferry
488
487
 
489
488
  include Logging
490
489
  extend Logging
491
-
490
+
492
491
 
493
492
  attr_reader :tag
494
493
 
@@ -627,7 +626,7 @@ module Bitferry
627
626
 
628
627
  include Logging
629
628
  extend Logging
630
-
629
+
631
630
 
632
631
  def self.executable = @executable ||= (rclone = ENV['RCLONE']).nil? ? 'rclone' : rclone
633
632
 
@@ -651,300 +650,301 @@ module Bitferry
651
650
  def self.reveal(token) = exec('reveal', '--', token)
652
651
 
653
652
 
654
- end
653
+ class Encryption
655
654
 
656
655
 
657
- class Rclone::Encryption
656
+ PROCESS = {
657
+ default: ['--crypt-filename-encoding', :base32, '--crypt-filename-encryption', :standard],
658
+ extended: ['--crypt-filename-encoding', :base32768, '--crypt-filename-encryption', :standard]
659
+ }
660
+ PROCESS[nil] = PROCESS[:default]
658
661
 
659
662
 
660
- PROCESS = {
661
- default: ['--crypt-filename-encoding', :base32, '--crypt-filename-encryption', :standard],
662
- extended: ['--crypt-filename-encoding', :base32768, '--crypt-filename-encryption', :standard]
663
- }
664
- PROCESS[nil] = PROCESS[:default]
663
+ def process_options = @process_options.nil? ? [] : @process_options # As a mandatory option it should never be nil
665
664
 
666
665
 
667
- def process_options = @process_options.nil? ? [] : @process_options # As a mandatory option it should never be nil
666
+ def initialize(token, process: nil)
667
+ @process_options = Bitferry.optional(process, PROCESS)
668
+ @token = token
669
+ end
668
670
 
669
671
 
670
- def initialize(token, process: nil)
671
- @process_options = Bitferry.optional(process, PROCESS)
672
- @token = token
673
- end
672
+ def create(password, **opts) = initialize(Rclone.obscure(password), **opts)
674
673
 
675
674
 
676
- def create(password, **opts) = initialize(Rclone.obscure(password), **opts)
675
+ def restore(hash) = @process_options = hash[:rclone]
677
676
 
678
677
 
679
- def restore(hash) = @process_options = hash[:rclone]
678
+ def externalize = process_options.empty? ? {} : { rclone: process_options }
680
679
 
681
680
 
682
- def externalize = process_options.empty? ? {} : { rclone: process_options }
681
+ def configure(task) = install_token(task)
683
682
 
684
683
 
685
- def configure(task) = install_token(task)
684
+ def process(task) = ENV['RCLONE_CRYPT_PASSWORD'] = obtain_token(task)
686
685
 
687
686
 
688
- def process(task) = ENV['RCLONE_CRYPT_PASSWORD'] = obtain_token(task)
687
+ def arguments(task) = process_options + ['--crypt-remote', encrypted(task).root.to_s]
689
688
 
690
689
 
691
- def arguments(task) = process_options + ['--crypt-remote', encrypted(task).root.to_s]
690
+ def install_token(task)
691
+ x = decrypted(task)
692
+ raise TypeError, 'unsupported unencrypted endpoint type' unless x.is_a?(Endpoint::Bitferry)
693
+ Volume[x.volume_tag].vault[task.tag] = @token # Token is stored on the decrypted end only
694
+ end
692
695
 
693
696
 
694
- def install_token(task)
695
- x = decrypted(task)
696
- raise TypeError, 'unsupported unencrypted endpoint type' unless x.is_a?(Endpoint::Bitferry)
697
- Volume[x.volume_tag].vault[task.tag] = @token # Token is stored on the decrypted end only
698
- end
697
+ def obtain_token(task) = Volume[decrypted(task).volume_tag].vault.fetch(task.tag)
699
698
 
700
699
 
701
- def obtain_token(task) = Volume[decrypted(task).volume_tag].vault.fetch(task.tag)
700
+ def self.new(*args, **opts)
701
+ obj = allocate
702
+ obj.send(:create, *args, **opts)
703
+ obj
704
+ end
702
705
 
703
706
 
704
- def self.new(*args, **opts)
705
- obj = allocate
706
- obj.send(:create, *args, **opts)
707
- obj
708
- end
707
+ def self.restore(hash)
708
+ obj = ROUTE.fetch(hash.fetch(:operation).intern).allocate
709
+ obj.send(:restore, hash)
710
+ obj
711
+ end
709
712
 
710
713
 
711
- def self.restore(hash)
712
- obj = ROUTE.fetch(hash.fetch(:operation).intern).allocate
713
- obj.send(:restore, hash)
714
- obj
715
714
  end
716
715
 
717
716
 
718
- end
717
+ class Encrypt < Encryption
719
718
 
720
719
 
721
- class Rclone::Encrypt < Rclone::Encryption
720
+ def encrypted(task) = task.destination
722
721
 
723
722
 
724
- def encrypted(task) = task.destination
723
+ def decrypted(task) = task.source
725
724
 
726
725
 
727
- def decrypted(task) = task.source
726
+ def externalize = super.merge(operation: :encrypt)
728
727
 
729
728
 
730
- def externalize = super.merge(operation: :encrypt)
729
+ def show_operation = 'encrypt+'
731
730
 
732
731
 
733
- def show_operation = 'encrypt+'
732
+ def arguments(task) = super + [decrypted(task).root.to_s, ':crypt:']
734
733
 
735
734
 
736
- def arguments(task) = super + [decrypted(task).root.to_s, ':crypt:']
735
+ end
737
736
 
738
737
 
739
- end
738
+ class Decrypt < Encryption
740
739
 
741
740
 
742
- class Rclone::Decrypt < Rclone::Encryption
741
+ def encrypted(task) = task.source
743
742
 
744
743
 
745
- def encrypted(task) = task.source
744
+ def decrypted(task) = task.destination
746
745
 
747
746
 
748
- def decrypted(task) = task.destination
747
+ def externalize = super.merge(operation: :decrypt)
749
748
 
750
749
 
751
- def externalize = super.merge(operation: :decrypt)
750
+ def show_operation = 'decrypt+'
752
751
 
753
752
 
754
- def show_operation = 'decrypt+'
753
+ def arguments(task) = super + [':crypt:', decrypted(task).root.to_s]
755
754
 
756
755
 
757
- def arguments(task) = super + [':crypt:', decrypted(task).root.to_s]
756
+ end
758
757
 
759
- end
760
758
 
759
+ ROUTE = {
760
+ encrypt: Encrypt,
761
+ decrypt: Decrypt
762
+ }
761
763
 
762
- Rclone::Encryption::ROUTE = {
763
- encrypt: Rclone::Encrypt,
764
- decrypt: Rclone::Decrypt
765
- }
766
764
 
765
+ class Task < Bitferry::Task
767
766
 
768
- class Rclone::Task < Task
769
767
 
768
+ attr_reader :source, :destination
770
769
 
771
- attr_reader :source, :destination
772
770
 
771
+ attr_reader :encryption
773
772
 
774
- attr_reader :encryption
775
773
 
774
+ attr_reader :token
776
775
 
777
- attr_reader :token
778
776
 
777
+ PROCESS = {
778
+ default: ['--metadata']
779
+ }
780
+ PROCESS[nil] = PROCESS[:default]
779
781
 
780
- PROCESS = {
781
- default: ['--metadata']
782
- }
783
- PROCESS[nil] = PROCESS[:default]
784
782
 
783
+ def initialize(source, destination, encryption: nil, process: nil, **opts)
784
+ super(**opts)
785
+ @process_options = Bitferry.optional(process, PROCESS)
786
+ @source = source.is_a?(Endpoint) ? source : Bitferry.endpoint(source)
787
+ @destination = destination.is_a?(Endpoint) ? destination : Bitferry.endpoint(destination)
788
+ @encryption = encryption
789
+ end
785
790
 
786
- def initialize(source, destination, encryption: nil, process: nil, **opts)
787
- super(**opts)
788
- @process_options = Bitferry.optional(process, PROCESS)
789
- @source = source.is_a?(Endpoint) ? source : Bitferry.endpoint(source)
790
- @destination = destination.is_a?(Endpoint) ? destination : Bitferry.endpoint(destination)
791
- @encryption = encryption
792
- end
793
791
 
792
+ def create(*args, process: nil, **opts)
793
+ super(*args, process: process, **opts)
794
+ encryption.configure(self) unless encryption.nil?
795
+ end
794
796
 
795
- def create(*args, process: nil, **opts)
796
- super(*args, process: process, **opts)
797
- encryption.configure(self) unless encryption.nil?
798
- end
799
797
 
798
+ def show_status = "#{show_operation} #{source.show_status} #{show_direction} #{destination.show_status}"
800
799
 
801
- def show_status = "#{show_operation} #{source.show_status} #{show_direction} #{destination.show_status}"
802
800
 
801
+ def show_operation = encryption.nil? ? '' : encryption.show_operation
803
802
 
804
- def show_operation = encryption.nil? ? '' : encryption.show_operation
805
803
 
804
+ def show_direction = '-->'
806
805
 
807
- def show_direction = '-->'
808
806
 
807
+ def intact? = live? && source.intact? && destination.intact?
809
808
 
810
- def intact? = live? && source.intact? && destination.intact?
811
809
 
810
+ def refers?(volume) = source.refers?(volume) || destination.refers?(volume)
812
811
 
813
- def refers?(volume) = source.refers?(volume) || destination.refers?(volume)
814
812
 
813
+ def touch
814
+ @generation = [source.generation, destination.generation].max + 1
815
+ super
816
+ end
815
817
 
816
- def touch
817
- @generation = [source.generation, destination.generation].max + 1
818
- super
819
- end
820
818
 
819
+ def format = nil
821
820
 
822
- def format = nil
823
821
 
822
+ def common_options
823
+ [
824
+ '--config', Bitferry.windows? ? 'NUL' : '/dev/null',
825
+ case Bitferry.verbosity
826
+ when :verbose then '--verbose'
827
+ when :quiet then '--quiet'
828
+ else nil
829
+ end,
830
+ Bitferry.verbosity == :verbose ? '--progress' : nil,
831
+ Bitferry.simulate? ? '--dry-run' : nil,
832
+ ].compact
833
+ end
824
834
 
825
- def common_options
826
- [
827
- '--config', Bitferry.windows? ? 'NUL' : '/dev/null',
828
- case Bitferry.verbosity
829
- when :verbose then '--verbose'
830
- when :quiet then '--quiet'
831
- else nil
832
- end,
833
- Bitferry.verbosity == :verbose ? '--progress' : nil,
834
- Bitferry.simulate? ? '--dry-run' : nil,
835
- ].compact
836
- end
837
835
 
836
+ def process_arguments
837
+ ['--filter', "- #{Volume::STORAGE}", '--filter', "- #{Volume::STORAGE_}"] + common_options + process_options + (
838
+ encryption.nil? ? [source.root.to_s, destination.root.to_s] : encryption.arguments(self)
839
+ )
840
+ end
838
841
 
839
- def process_arguments
840
- ['--filter', "- #{Volume::STORAGE}", '--filter', "- #{Volume::STORAGE_}"] + common_options + process_options + (
841
- encryption.nil? ? [source.root.to_s, destination.root.to_s] : encryption.arguments(self)
842
- )
843
- end
844
842
 
843
+ def execute(*args)
844
+ cmd = [Rclone.executable] + args
845
+ cms = cmd.collect(&:shellescape).join(' ')
846
+ puts cms if Bitferry.verbosity == :verbose
847
+ log.info(cms)
848
+ status = Open3.pipeline(cmd).first
849
+ raise "rclone exit code #{status.exitstatus}" unless status.success?
850
+ status.success?
851
+ end
845
852
 
846
- def execute(*args)
847
- cmd = [Rclone.executable] + args
848
- cms = cmd.collect(&:shellescape).join(' ')
849
- puts cms if Bitferry.verbosity == :verbose
850
- log.info(cms)
851
- status = Open3.pipeline(cmd).first
852
- raise "rclone exit code #{status.exitstatus}" unless status.success?
853
- status.success?
854
- end
855
853
 
854
+ def process
855
+ log.info("processing task #{tag}")
856
+ encryption.process(self) unless encryption.nil?
857
+ execute(*process_arguments)
858
+ end
856
859
 
857
- def process
858
- log.info("processing task #{tag}")
859
- encryption.process(self) unless encryption.nil?
860
- execute(*process_arguments)
861
- end
862
860
 
861
+ def externalize
862
+ super.merge(
863
+ source: source.externalize,
864
+ destination: destination.externalize,
865
+ encryption: encryption.nil? ? nil : encryption.externalize,
866
+ rclone: process_options.empty? ? nil : process_options
867
+ ).compact
868
+ end
863
869
 
864
- def externalize
865
- super.merge(
866
- source: source.externalize,
867
- destination: destination.externalize,
868
- encryption: encryption.nil? ? nil : encryption.externalize,
869
- rclone: process_options.empty? ? nil : process_options
870
- ).compact
871
- end
870
+
871
+ def restore(hash)
872
+ initialize(
873
+ restore_endpoint(hash.fetch(:source)),
874
+ restore_endpoint(hash.fetch(:destination)),
875
+ tag: hash.fetch(:task),
876
+ modified: hash.fetch(:modified, DateTime.now),
877
+ process: hash[:rclone],
878
+ encryption: hash[:encryption].nil? ? nil : Rclone::Encryption.restore(hash[:encryption])
879
+ )
880
+ super(hash)
881
+ end
872
882
 
873
883
 
874
- def restore(hash)
875
- initialize(
876
- restore_endpoint(hash.fetch(:source)),
877
- restore_endpoint(hash.fetch(:destination)),
878
- tag: hash.fetch(:task),
879
- modified: hash.fetch(:modified, DateTime.now),
880
- process: hash[:rclone],
881
- encryption: hash[:encryption].nil? ? nil : Rclone::Encryption.restore(hash[:encryption])
882
- )
883
- super(hash)
884
884
  end
885
885
 
886
886
 
887
- end
887
+ class Copy < Task
888
888
 
889
889
 
890
- class Rclone::Copy < Rclone::Task
890
+ def process_arguments = ['copy'] + super
891
891
 
892
892
 
893
- def process_arguments = ['copy'] + super
893
+ def externalize = super.merge(operation: :copy)
894
894
 
895
895
 
896
- def externalize = super.merge(operation: :copy)
896
+ def show_operation = super + 'copy'
897
897
 
898
898
 
899
- def show_operation = super + 'copy'
899
+ end
900
900
 
901
901
 
902
- end
902
+ class Update < Task
903
903
 
904
904
 
905
- class Rclone::Update < Rclone::Task
905
+ def process_arguments = ['copy', '--update'] + super
906
906
 
907
907
 
908
- def process_arguments = ['copy', '--update'] + super
908
+ def externalize = super.merge(operation: :update)
909
909
 
910
910
 
911
- def externalize = super.merge(operation: :update)
911
+ def show_operation = super + 'update'
912
912
 
913
913
 
914
- def show_operation = super + 'update'
914
+ end
915
915
 
916
916
 
917
- end
917
+ class Synchronize < Task
918
918
 
919
919
 
920
- class Rclone::Synchronize < Rclone::Task
920
+ def process_arguments = ['sync'] + super
921
921
 
922
922
 
923
- def process_arguments = ['sync'] + super
923
+ def externalize = super.merge(operation: :synchronize)
924
924
 
925
925
 
926
- def externalize = super.merge(operation: :synchronize)
926
+ def show_operation = super + 'synchronize'
927
927
 
928
928
 
929
- def show_operation = super + 'synchronize'
929
+ end
930
930
 
931
931
 
932
- end
932
+ class Equalize < Task
933
933
 
934
934
 
935
- class Rclone::Equalize < Rclone::Task
935
+ def process_arguments = ['bisync', '--resync'] + super
936
936
 
937
937
 
938
- def process_arguments = ['bisync', '--resync'] + super
938
+ def externalize = super.merge(operation: :equalize)
939
939
 
940
940
 
941
- def externalize = super.merge(operation: :equalize)
941
+ def show_operation = super + 'equalize'
942
942
 
943
943
 
944
- def show_operation = super + 'equalize'
944
+ def show_direction = '<->'
945
945
 
946
946
 
947
- def show_direction = '<->'
947
+ end
948
948
 
949
949
 
950
950
  end
@@ -955,7 +955,7 @@ module Bitferry
955
955
 
956
956
  include Logging
957
957
  extend Logging
958
-
958
+
959
959
 
960
960
  def self.executable = @executable ||= (restic = ENV['RESTIC']).nil? ? 'restic' : restic
961
961
 
@@ -973,272 +973,275 @@ module Bitferry
973
973
  end
974
974
 
975
975
 
976
- end
976
+ class Task < Bitferry::Task
977
977
 
978
978
 
979
- class Restic::Task < Task
979
+ attr_reader :directory, :repository
980
980
 
981
981
 
982
- attr_reader :directory, :repository
982
+ def initialize(directory, repository, **opts)
983
+ super(**opts)
984
+ @directory = directory.is_a?(Endpoint) ? directory : Bitferry.endpoint(directory)
985
+ @repository = repository.is_a?(Endpoint) ? repository : Bitferry.endpoint(repository)
986
+ end
983
987
 
984
988
 
985
- def initialize(directory, repository, **opts)
986
- super(**opts)
987
- @directory = directory.is_a?(Endpoint) ? directory : Bitferry.endpoint(directory)
988
- @repository = repository.is_a?(Endpoint) ? repository : Bitferry.endpoint(repository)
989
- end
990
-
989
+ def create(directory, repository, password, **opts)
990
+ super(directory, repository, **opts)
991
+ raise TypeError, 'unsupported unencrypted endpoint type' unless self.directory.is_a?(Endpoint::Bitferry)
992
+ Volume[self.directory.volume_tag].vault[tag] = Rclone.obscure(@password = password) # Token is stored on the decrypted end only
993
+ end
991
994
 
992
- def create(directory, repository, password, **opts)
993
- super(directory, repository, **opts)
994
- raise TypeError, 'unsupported unencrypted endpoint type' unless self.directory.is_a?(Endpoint::Bitferry)
995
- Volume[self.directory.volume_tag].vault[tag] = Rclone.obscure(@password = password) # Token is stored on the decrypted end only
996
- end
997
995
 
996
+ def password = @password ||= Rclone.reveal(Volume[directory.volume_tag].vault.fetch(tag))
998
997
 
999
- def password = @password ||= Rclone.reveal(Volume[directory.volume_tag].vault.fetch(tag))
1000
998
 
999
+ def intact? = live? && directory.intact? && repository.intact?
1001
1000
 
1002
- def intact? = live? && directory.intact? && repository.intact?
1003
1001
 
1002
+ def refers?(volume) = directory.refers?(volume) || repository.refers?(volume)
1004
1003
 
1005
- def refers?(volume) = directory.refers?(volume) || repository.refers?(volume)
1006
1004
 
1005
+ def touch
1006
+ @generation = [directory.generation, repository.generation].max + 1
1007
+ super
1008
+ end
1007
1009
 
1008
- def touch
1009
- @generation = [directory.generation, repository.generation].max + 1
1010
- super
1011
- end
1012
1010
 
1013
- def format = nil
1011
+ def format = nil
1014
1012
 
1015
1013
 
1016
- def common_options
1017
- [
1018
- case Bitferry.verbosity
1019
- when :verbose then '--verbose'
1020
- when :quiet then '--quiet'
1021
- else nil
1022
- end,
1023
- '-r', repository.root.to_s
1024
- ].compact
1025
- end
1014
+ def common_options
1015
+ [
1016
+ case Bitferry.verbosity
1017
+ when :verbose then '--verbose'
1018
+ when :quiet then '--quiet'
1019
+ else nil
1020
+ end,
1021
+ '-r', repository.root.to_s
1022
+ ].compact
1023
+ end
1026
1024
 
1027
1025
 
1028
- def execute(*args, simulate: false, chdir: nil)
1029
- cmd = [Restic.executable] + args
1030
- ENV['RESTIC_PASSWORD'] = password
1031
- cms = cmd.collect(&:shellescape).join(' ')
1032
- puts cms if Bitferry.verbosity == :verbose
1033
- log.info(cms)
1034
- if simulate
1035
- log.info('(simulated)')
1036
- true
1037
- else
1038
- wd = Dir.getwd unless chdir.nil?
1039
- begin
1040
- Dir.chdir(chdir) unless chdir.nil?
1041
- status = Open3.pipeline(cmd).first
1042
- raise "restic exit code #{status.exitstatus}" unless status.success?
1043
- ensure
1044
- Dir.chdir(wd) unless chdir.nil?
1026
+ def execute(*args, simulate: false, chdir: nil)
1027
+ cmd = [Restic.executable] + args
1028
+ ENV['RESTIC_PASSWORD'] = password
1029
+ cms = cmd.collect(&:shellescape).join(' ')
1030
+ puts cms if Bitferry.verbosity == :verbose
1031
+ log.info(cms)
1032
+ if simulate
1033
+ log.info('(simulated)')
1034
+ true
1035
+ else
1036
+ wd = Dir.getwd unless chdir.nil?
1037
+ begin
1038
+ Dir.chdir(chdir) unless chdir.nil?
1039
+ status = Open3.pipeline(cmd).first
1040
+ raise "restic exit code #{status.exitstatus}" unless status.success?
1041
+ ensure
1042
+ Dir.chdir(wd) unless chdir.nil?
1043
+ end
1045
1044
  end
1046
1045
  end
1047
- end
1048
1046
 
1049
1047
 
1050
- def externalize
1051
- super.merge(
1052
- directory: directory.externalize,
1053
- repository: repository.externalize,
1054
- ).compact
1055
- end
1048
+ def externalize
1049
+ super.merge(
1050
+ directory: directory.externalize,
1051
+ repository: repository.externalize,
1052
+ ).compact
1053
+ end
1056
1054
 
1057
1055
 
1058
- def restore(hash)
1059
- initialize(
1060
- restore_endpoint(hash.fetch(:directory)),
1061
- restore_endpoint(hash.fetch(:repository)),
1062
- tag: hash.fetch(:task),
1063
- modified: hash.fetch(:modified, DateTime.now)
1064
- )
1065
- super(hash)
1066
- end
1056
+ def restore(hash)
1057
+ initialize(
1058
+ restore_endpoint(hash.fetch(:directory)),
1059
+ restore_endpoint(hash.fetch(:repository)),
1060
+ tag: hash.fetch(:task),
1061
+ modified: hash.fetch(:modified, DateTime.now)
1062
+ )
1063
+ super(hash)
1064
+ end
1067
1065
 
1068
1066
 
1069
- end
1067
+ end
1070
1068
 
1071
1069
 
1072
- class Restic::Backup < Restic::Task
1070
+ class Backup < Task
1073
1071
 
1074
1072
 
1075
- PROCESS = {
1076
- default: ['--no-cache']
1077
- }
1078
- PROCESS[nil] = PROCESS[:default]
1073
+ PROCESS = {
1074
+ default: ['--no-cache']
1075
+ }
1076
+ PROCESS[nil] = PROCESS[:default]
1079
1077
 
1080
1078
 
1081
- FORGET = {
1082
- default: ['--prune', '--keep-within-hourly', '24h', '--keep-within-daily', '7d', '--keep-within-weekly', '30d', '--keep-within-monthly', '1y', '--keep-within-yearly', '100y']
1083
- }
1084
- FORGET[nil] = nil # Skip processing retention policy by default
1079
+ FORGET = {
1080
+ default: ['--prune', '--keep-within-hourly', '24h', '--keep-within-daily', '7d', '--keep-within-weekly', '30d', '--keep-within-monthly', '1y', '--keep-within-yearly', '100y']
1081
+ }
1082
+ FORGET[nil] = nil # Skip processing retention policy by default
1085
1083
 
1086
1084
 
1087
- CHECK = {
1088
- default: [],
1089
- full: ['--read-data']
1090
- }
1091
- CHECK[nil] = nil # Skip integrity checking by default
1085
+ CHECK = {
1086
+ default: [],
1087
+ full: ['--read-data']
1088
+ }
1089
+ CHECK[nil] = nil # Skip integrity checking by default
1092
1090
 
1093
1091
 
1094
- attr_reader :forget_options
1095
- attr_reader :check_options
1092
+ attr_reader :forget_options
1093
+ attr_reader :check_options
1096
1094
 
1097
1095
 
1098
- def create(*args, format: nil, process: nil, forget: nil, check: nil, **opts)
1099
- super(*args, **opts)
1100
- @format = format
1101
- @process_options = Bitferry.optional(process, PROCESS)
1102
- @forget_options = Bitferry.optional(forget, FORGET)
1103
- @check_options = Bitferry.optional(check, CHECK)
1104
- end
1096
+ def create(*args, format: nil, process: nil, forget: nil, check: nil, **opts)
1097
+ super(*args, **opts)
1098
+ @format = format
1099
+ @process_options = Bitferry.optional(process, PROCESS)
1100
+ @forget_options = Bitferry.optional(forget, FORGET)
1101
+ @check_options = Bitferry.optional(check, CHECK)
1102
+ end
1105
1103
 
1106
1104
 
1107
- def show_status = "#{show_operation} #{directory.show_status} #{show_direction} #{repository.show_status}"
1105
+ def show_status = "#{show_operation} #{directory.show_status} #{show_direction} #{repository.show_status}"
1108
1106
 
1109
1107
 
1110
- def show_operation = 'encrypt+backup'
1108
+ def show_operation = 'encrypt+backup'
1111
1109
 
1112
1110
 
1113
- def show_direction = '-->'
1111
+ def show_direction = '-->'
1114
1112
 
1115
1113
 
1116
- def process
1117
- begin
1118
- log.info("processing task #{tag}")
1119
- execute('backup', '.', '--tag', "bitferry,#{tag}", '--exclude', Volume::STORAGE, '--exclude', Volume::STORAGE_, *process_options, *common_options_simulate, chdir: directory.root)
1120
- unless check_options.nil?
1121
- log.info("checking repository in #{repository.root}")
1122
- execute('check', *check_options, *common_options)
1123
- end
1124
- unless forget_options.nil?
1125
- log.info("performing repository maintenance tasks in #{repository.root}")
1126
- execute('forget', '--tag', "bitferry,#{tag}", *forget_options.collect(&:to_s), *common_options_simulate)
1114
+ def process
1115
+ begin
1116
+ log.info("processing task #{tag}")
1117
+ execute('backup', '.', '--tag', "bitferry,#{tag}", '--exclude', Volume::STORAGE, '--exclude', Volume::STORAGE_, *process_options, *common_options_simulate, chdir: directory.root)
1118
+ unless check_options.nil?
1119
+ log.info("checking repository in #{repository.root}")
1120
+ execute('check', *check_options, *common_options)
1121
+ end
1122
+ unless forget_options.nil?
1123
+ log.info("performing repository maintenance tasks in #{repository.root}")
1124
+ execute('forget', '--tag', "bitferry,#{tag}", *forget_options.collect(&:to_s), *common_options_simulate)
1125
+ end
1126
+ true
1127
+ rescue
1128
+ false
1127
1129
  end
1128
- true
1129
- rescue
1130
- false
1131
1130
  end
1132
- end
1133
1131
 
1134
1132
 
1135
- def common_options_simulate = common_options + [Bitferry.simulate? ? '--dry-run' : nil].compact
1133
+ def common_options_simulate = common_options + [Bitferry.simulate? ? '--dry-run' : nil].compact
1136
1134
 
1137
1135
 
1138
- def externalize
1139
- restic = {
1140
- process: process_options,
1141
- forget: forget_options,
1142
- check: check_options
1143
- }.compact
1144
- super.merge({
1145
- operation: :backup,
1146
- restic: restic.empty? ? nil : restic
1147
- }.compact)
1148
- end
1136
+ def externalize
1137
+ restic = {
1138
+ process: process_options,
1139
+ forget: forget_options,
1140
+ check: check_options
1141
+ }.compact
1142
+ super.merge({
1143
+ operation: :backup,
1144
+ restic: restic.empty? ? nil : restic
1145
+ }.compact)
1146
+ end
1149
1147
 
1150
1148
 
1151
- def restore(hash)
1152
- super
1153
- opts = hash.fetch(:restic, {})
1154
- @process_options = opts[:process]
1155
- @forget_options = opts[:forget]
1156
- @check_options = opts[:check]
1157
- end
1149
+ def restore(hash)
1150
+ super
1151
+ opts = hash.fetch(:restic, {})
1152
+ @process_options = opts[:process]
1153
+ @forget_options = opts[:forget]
1154
+ @check_options = opts[:check]
1155
+ end
1158
1156
 
1159
1157
 
1160
- def format
1161
- if Bitferry.simulate?
1162
- log.info('skipped repository initialization (simulation)')
1163
- else
1164
- log.info("initializing repository for task #{tag}")
1165
- if @format == true
1166
- log.debug("wiping repository in #{repository.root}")
1167
- ['config', 'data', 'index', 'keys', 'locks', 'snapshots'].each { |x| FileUtils.rm_rf(File.join(repository.root.to_s, x)) }
1168
- end
1169
- if @format == false
1170
- # TODO validate existing repo
1171
- log.info("attached to existing repository for task #{tag} in #{repository.root}")
1158
+ def format
1159
+ if Bitferry.simulate?
1160
+ log.info('skipped repository initialization (simulation)')
1172
1161
  else
1173
- begin
1174
- execute(*common_options, 'init')
1175
- log.info("initialized repository for task #{tag} in #{repository.root}")
1176
- rescue
1177
- log.fatal("failed to initialize repository for task #{tag} in #{repository.root}")
1178
- raise
1162
+ log.info("initializing repository for task #{tag}")
1163
+ if @format == true
1164
+ log.debug("wiping repository in #{repository.root}")
1165
+ ['config', 'data', 'index', 'keys', 'locks', 'snapshots'].each { |x| FileUtils.rm_rf(File.join(repository.root.to_s, x)) }
1166
+ end
1167
+ if @format == false
1168
+ # TODO validate existing repo
1169
+ log.info("attached to existing repository for task #{tag} in #{repository.root}")
1170
+ else
1171
+ begin
1172
+ execute(*common_options, 'init')
1173
+ log.info("initialized repository for task #{tag} in #{repository.root}")
1174
+ rescue
1175
+ log.fatal("failed to initialize repository for task #{tag} in #{repository.root}")
1176
+ raise
1177
+ end
1179
1178
  end
1180
1179
  end
1180
+ @state = :intact
1181
1181
  end
1182
- @state = :intact
1183
- end
1184
1182
 
1185
- end
1186
1183
 
1184
+ end
1187
1185
 
1188
- class Restic::Restore < Restic::Task
1189
1186
 
1187
+ class Restore < Task
1190
1188
 
1191
- PROCESS = {
1192
- default: ['--no-cache', '--sparse']
1193
- }
1194
- PROCESS[nil] = PROCESS[:default]
1195
1189
 
1190
+ PROCESS = {
1191
+ default: ['--no-cache', '--sparse']
1192
+ }
1193
+ PROCESS[nil] = PROCESS[:default]
1196
1194
 
1197
- def create(*args, process: nil, **opts)
1198
- super(*args, **opts)
1199
- @process_options = Bitferry.optional(process, PROCESS)
1200
- end
1201
1195
 
1196
+ def create(*args, process: nil, **opts)
1197
+ super(*args, **opts)
1198
+ @process_options = Bitferry.optional(process, PROCESS)
1199
+ end
1202
1200
 
1203
- def show_status = "#{show_operation} #{repository.show_status} #{show_direction} #{directory.show_status}"
1204
1201
 
1202
+ def show_status = "#{show_operation} #{repository.show_status} #{show_direction} #{directory.show_status}"
1205
1203
 
1206
- def show_operation = 'decrypt+restore'
1207
1204
 
1205
+ def show_operation = 'decrypt+restore'
1208
1206
 
1209
- def show_direction = '-->'
1210
1207
 
1208
+ def show_direction = '-->'
1211
1209
 
1212
- def externalize
1213
- restic = {
1214
- process: process_options
1215
- }.compact
1216
- super.merge({
1217
- operation: :restore,
1218
- restic: restic.empty? ? nil : restic
1219
- }.compact)
1220
- end
1221
1210
 
1211
+ def externalize
1212
+ restic = {
1213
+ process: process_options
1214
+ }.compact
1215
+ super.merge({
1216
+ operation: :restore,
1217
+ restic: restic.empty? ? nil : restic
1218
+ }.compact)
1219
+ end
1222
1220
 
1223
- def restore(hash)
1224
- super
1225
- opts = hash.fetch(:rclone, {})
1226
- @process_options = opts[:process]
1227
- end
1221
+
1222
+ def restore(hash)
1223
+ super
1224
+ opts = hash.fetch(:rclone, {})
1225
+ @process_options = opts[:process]
1226
+ end
1228
1227
 
1229
1228
 
1230
- def process
1231
- log.info("processing task #{tag}")
1232
- begin
1233
- # FIXME restore specifically tagged latest snapshot
1234
- execute('restore', 'latest', '--target', '.', *process_options, *common_options, simulate: Bitferry.simulate?, chdir: directory.root)
1235
- true
1236
- rescue
1237
- false
1229
+ def process
1230
+ log.info("processing task #{tag}")
1231
+ begin
1232
+ # FIXME restore specifically tagged latest snapshot
1233
+ execute('restore', 'latest', '--target', '.', *process_options, *common_options, simulate: Bitferry.simulate?, chdir: directory.root)
1234
+ true
1235
+ rescue
1236
+ false
1237
+ end
1238
1238
  end
1239
+
1240
+
1239
1241
  end
1240
1242
 
1241
1243
 
1244
+
1242
1245
  end
1243
1246
 
1244
1247
 
@@ -1262,109 +1265,109 @@ module Bitferry
1262
1265
  end
1263
1266
 
1264
1267
 
1265
- end
1268
+ class Local < Endpoint
1266
1269
 
1267
1270
 
1268
- class Endpoint::Local < Endpoint
1271
+ attr_reader :root
1269
1272
 
1270
1273
 
1271
- attr_reader :root
1274
+ def initialize(root) = @root = Pathname.new(root).realdirpath
1272
1275
 
1273
1276
 
1274
- def initialize(root) = @root = Pathname.new(root).realdirpath
1277
+ def restore(hash) = initialize(hash.fetch(:root))
1275
1278
 
1276
1279
 
1277
- def restore(hash) = initialize(hash.fetch(:root))
1280
+ def externalize
1281
+ {
1282
+ endpoint: :local,
1283
+ root: root
1284
+ }
1285
+ end
1278
1286
 
1279
1287
 
1280
- def externalize
1281
- {
1282
- endpoint: :local,
1283
- root: root
1284
- }
1285
- end
1288
+ def show_status = root.to_s
1286
1289
 
1287
1290
 
1288
- def show_status = root.to_s
1291
+ def intact? = true
1289
1292
 
1290
1293
 
1291
- def intact? = true
1294
+ def refers?(volume) = false
1292
1295
 
1293
1296
 
1294
- def refers?(volume) = false
1297
+ def generation = 0
1295
1298
 
1296
1299
 
1297
- def generation = 0
1300
+ end
1298
1301
 
1299
1302
 
1300
- end
1303
+ class Rclone < Endpoint
1304
+ # TODO
1305
+ end
1301
1306
 
1302
1307
 
1303
- class Endpoint::Rclone < Endpoint
1304
- # TODO
1305
- end
1308
+ class Bitferry < Endpoint
1306
1309
 
1307
1310
 
1308
- class Endpoint::Bitferry < Endpoint
1311
+ attr_reader :volume_tag
1309
1312
 
1310
1313
 
1311
- attr_reader :volume_tag
1314
+ attr_reader :path
1312
1315
 
1313
1316
 
1314
- attr_reader :path
1317
+ def root = Volume[volume_tag].root.join(path)
1315
1318
 
1316
1319
 
1317
- def root = Volume[volume_tag].root.join(path)
1320
+ def initialize(volume, path)
1321
+ @volume_tag = volume.tag
1322
+ @path = Pathname.new(path)
1323
+ raise ArgumentError, "expected relative path but got #{self.path}" unless (/^[\.\/]/ =~ self.path.to_s).nil?
1324
+ end
1318
1325
 
1319
1326
 
1320
- def initialize(volume, path)
1321
- @volume_tag = volume.tag
1322
- @path = Pathname.new(path)
1323
- raise ArgumentError, "expected relative path but got #{self.path}" unless (/^[\.\/]/ =~ self.path.to_s).nil?
1324
- end
1327
+ def restore(hash)
1328
+ @volume_tag = hash.fetch(:volume)
1329
+ @path = Pathname.new(hash.fetch(:path))
1330
+ end
1325
1331
 
1326
1332
 
1327
- def restore(hash)
1328
- @volume_tag = hash.fetch(:volume)
1329
- @path = Pathname.new(hash.fetch(:path))
1330
- end
1333
+ def externalize
1334
+ {
1335
+ endpoint: :bitferry,
1336
+ volume: volume_tag,
1337
+ path: path
1338
+ }
1339
+ end
1331
1340
 
1332
1341
 
1333
- def externalize
1334
- {
1335
- endpoint: :bitferry,
1336
- volume: volume_tag,
1337
- path: path
1338
- }
1339
- end
1342
+ def show_status = intact? ? ":#{volume_tag}:#{path}" : ":{#{volume_tag}}:#{path}"
1340
1343
 
1341
1344
 
1342
- def show_status = intact? ? ":#{volume_tag}:#{path}" : ":{#{volume_tag}}:#{path}"
1345
+ def intact? = !Volume[volume_tag].nil?
1343
1346
 
1344
1347
 
1345
- def intact? = !Volume[volume_tag].nil?
1348
+ def refers?(volume) = volume.tag == volume_tag
1346
1349
 
1347
1350
 
1348
- def refers?(volume) = volume.tag == volume_tag
1351
+ def generation
1352
+ v = Volume[volume_tag]
1353
+ v ? v.generation : 0
1354
+ end
1349
1355
 
1350
1356
 
1351
- def generation
1352
- v = Volume[volume_tag]
1353
- v ? v.generation : 0
1354
1357
  end
1355
1358
 
1356
1359
 
1357
- end
1360
+ ROUTE = {
1361
+ local: Local,
1362
+ rclone: Rclone,
1363
+ bitferry: Bitferry
1364
+ }
1358
1365
 
1359
1366
 
1360
- Endpoint::ROUTE = {
1361
- local: Endpoint::Local,
1362
- rclone: Endpoint::Rclone,
1363
- bitferry: Endpoint::Bitferry
1364
- }
1367
+ end
1365
1368
 
1366
1369
 
1367
1370
  reset
1368
1371
 
1369
1372
 
1370
- end
1373
+ end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: bitferry
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.0.1
4
+ version: 0.0.2
5
5
  platform: ruby
6
6
  authors:
7
7
  - Oleg A. Khlybov
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2024-03-05 00:00:00.000000000 Z
11
+ date: 2024-03-06 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: rake
@@ -73,6 +73,7 @@ executables:
73
73
  extensions: []
74
74
  extra_rdoc_files: []
75
75
  files:
76
+ - CHANGES.md
76
77
  - README.md
77
78
  - bin/bitferry
78
79
  - lib/bitferry.rb
@@ -97,7 +98,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
97
98
  - !ruby/object:Gem::Version
98
99
  version: '0'
99
100
  requirements: []
100
- rubygems_version: 3.2.33
101
+ rubygems_version: 3.4.19
101
102
  signing_key:
102
103
  specification_version: 4
103
104
  summary: File synchronization/backup automation tool