bitferry 0.0.7 → 0.0.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: f67881c016dae96456c1d0b1baee9b3949eb9bfd46aa5f96cf7232d219c1d7d8
4
- data.tar.gz: a5613291cc0b050ce243c8cf4142d8020c363100b0655cf6fcb405c0b380daba
3
+ metadata.gz: 3f38ab53f3488290ebf6c2e35015d38782230118e0227395d89008a1da7d14dc
4
+ data.tar.gz: 7112db76e809e66ad0585703f5f176428f8db521e721fed54e3fee8e190d6a21
5
5
  SHA512:
6
- metadata.gz: 670ea9837098777a1dfa56df30283f2d6fac41df303003a33cd1e80711d5c4967f09ab72ba2247dbdb00c57830f0310825a064397f5db463e0ee927de72ce6dc
7
- data.tar.gz: 433fb3934ae559cf42eb0a20d051923c0dc5e93eed32f5249dc95a1dde1930d3982fb925efd64f03d418fa9fefa54a74959ec87118c3c26d05ba0927667ee0cf
6
+ metadata.gz: 0ef76707bf8450e0ec948034fe31148f2f24b3205da0bb94e85625be9cc3c5225d91a4fd9c002870ceafa6cb15778c9a1e9a5269d07840d62a56af1a7b8179ae
7
+ data.tar.gz: d20a048c3d003fa7afc8c23f7b054a6989ab09086ecde882a56cb0f30af5bb07d3301bcb650ff902defa590d78b531c376f07123887a41c102a7a2451c6570c3
data/CHANGES.md CHANGED
@@ -1,35 +1,40 @@
1
- ## 0.0.7
2
-
3
- - Fix backend execution issues in GUI
4
- - Add verbosity control in GUI
5
- - GUI info additions
6
-
7
- ## 0.0.6
8
-
9
- - Rudimentary GUI employing FXRuby
10
-
11
- ## 0.0.5
12
-
13
- - Disable cache usage in Restic check profiles
14
- - Show stale tasks in verbose mode only
15
- - Pick the innermost Bitferry volume in case of volumes nesting
16
- - Fix an options profile application bug
17
-
18
- ## 0.0.4
19
-
20
- - Include/exclude path filters
21
- - Include user's home in the Bitferry volume lookup list
22
-
23
- ## 0.0.3
24
-
25
- - Windows bundle
26
- - Doc fixes
27
-
28
- ## 0.0.2
29
-
30
- - Fix infinite nesting of directories during GEM installation
31
- - Doc updates
32
-
33
- ## 0.0.1
34
-
1
+ ## 0.0.8
2
+
3
+ - Proper handling of paths starting with dot(s)
4
+ - Allow volume and task timestamps missing in storage files
5
+
6
+ ## 0.0.7
7
+
8
+ - Fix backend execution issues in GUI
9
+ - Add verbosity control in GUI
10
+ - GUI info additions
11
+
12
+ ## 0.0.6
13
+
14
+ - Rudimentary GUI employing FXRuby
15
+
16
+ ## 0.0.5
17
+
18
+ - Disable cache usage in Restic check profiles
19
+ - Show stale tasks in verbose mode only
20
+ - Pick the innermost Bitferry volume in case of volumes nesting
21
+ - Fix an options profile application bug
22
+
23
+ ## 0.0.4
24
+
25
+ - Include/exclude path filters
26
+ - Include user's home in the Bitferry volume lookup list
27
+
28
+ ## 0.0.3
29
+
30
+ - Windows bundle
31
+ - Doc fixes
32
+
33
+ ## 0.0.2
34
+
35
+ - Fix infinite nesting of directories during GEM installation
36
+ - Doc updates
37
+
38
+ ## 0.0.1
39
+
35
40
  - Initial release
data/README.md CHANGED
@@ -1,231 +1,231 @@
1
- # Bitferry - file synchronization/backup automation
2
-
3
- <div align="right"><i>Ein Backup ist kein Backup</i></div><br><br>
4
-
5
- The [Bitferry](https://github.com/okhlybov/bitferry) is aimed at establishing the automated file synchronization/replication/backup routes between multiple endpoints where the latter can be the local directories, online cloud remotes or portable offline storages.
6
-
7
- The intended usage ranges from maintaining simple directory copy to another location (disk, mount point) to complex many-to-many (online/offline) data replication/backup solution employing portable media as additional data storage and a means of data propagation between the offsites.
8
-
9
- The core idea that drives Bitferry is the conversion of full (absolute) endpoints' paths into the volume-relative ones, where the volume is a data file which is put along the endpoint's data and denotes the root of the directory hierarchy. This makes data position-independent which means that Bitferry is then able to restore the tasks' source-destination endpoint connections in spite of the volume location changes, which is a likely scenario in case of portable storage (different UNIX mount points, Windows drives etc.).
10
-
11
- Bitferry is effectively a frontend to the [Rclone](https://rclone.org) and [Restic](https://restic.net) utilities.
12
-
13
- ## Features
14
-
15
- * Multiplatform (Windows / UNIX / macOSX) operation
16
-
17
- * Automated task-based data processing
18
-
19
- * One way / two way data synchronization
20
-
21
- * Recursive directory copy / update / synchronize
22
-
23
- * Incremental directory backup with snapshotting
24
-
25
- * File/repository password-based end-to-end encryption
26
-
27
- * Online cloud storage relay
28
-
29
- * Offline portable storage (USB flash, HDDs, SSDs etc.) relay
30
-
31
- ## Use cases
32
-
33
- * Maintain an update-only files copy in a separate location on the same site
34
-
35
- * Maintain offline secure two way file synchronization between two offsites
36
-
37
- * Maintain an incremental files backup on a portable medium with multiple offsite copies of the repository
38
-
39
- ## Implementation
40
-
41
- The Bitferry itself is written in [Ruby](https://www.ruby-lang.org) programming language. Being a Ruby code, the Bitferry requires the platform-specific Ruby runtime, version 3.0 or higher.
42
-
43
- The source code is hosted on [GitHub](https://github.com/okhlybov/bitferry) and the binary releases in form of a GEM package are distributed through the [RubyGems](https://rubygems.org/gems/bitferry) repository channel.
44
-
45
- In addition, the platform-specific [Rclone](https://github.com/rclone/rclone/releases) and [Restic](https://github.com/restic/restic/releases) executables are required to be accessible through the `PATH` directory list or through the respective `RCLONE` and `RESTIC` environment variables.
46
-
47
- ## Kickstart
48
-
49
- ### Install Bitferry
50
-
51
- ```shell
52
- gem install bitferry
53
- ```
54
-
55
- ### Prepare source Bitferry volume for a mounted local filesystem
56
-
57
- ```shell
58
- bitferry create volume /data
59
- ```
60
-
61
- ### Prepare destination Bitferry volume for a mounted portable storage
62
-
63
- ```shell
64
- bitferry create volume /mnt/usb-drive
65
- ```
66
-
67
- ### Ensure the volumes are intact
68
-
69
- ```shell
70
- bitferry show
71
- ```
72
-
73
- ```
74
- # Intact volumes
75
-
76
- d2f10024 /data
77
- e42f2d8c /mnt/usb-drive
78
- ```
79
-
80
- ### Create a (Rclone) sync task with data encryption
81
-
82
- ```shell
83
- bitferry create task sync -e /data /mnt/usb-drive/backup
84
- ```
85
-
86
- ### Review the changes
87
-
88
- ```shell
89
- bitferry
90
- ```
91
-
92
- ```
93
- # Intact volumes
94
-
95
- d2f10024 /data
96
- e42f2d8c /mnt/usb-drive
97
-
98
-
99
- # Intact tasks
100
-
101
- 89e1c119 encrypt+synchronize :d2f10024: --> :e42f2d8c:backup
102
- ```
103
-
104
- ### Perform a dry run of the specific task
105
-
106
- ```shell
107
- bitferry process -vn 89e
108
- ```
109
-
110
- <details>
111
- <summary>...</summary>
112
-
113
- ```
114
- rclone sync --filter -\ .bitferry --filter -\ .bitferry\~ --verbose --progress --dry-run --metadata --crypt-filename-encoding base32 --crypt-filename-encryption standard --crypt-remote /mnt/usb-drive/backup /data :crypt:
115
- 2024/03/05 11:46:45 NOTICE: README.md: Skipped copy as --dry-run is set (size 3.073Ki)
116
- 2024/03/05 11:46:45 NOTICE: LICENSE: Skipped copy as --dry-run is set (size 1.467Ki)
117
- 2024/03/05 11:46:45 NOTICE: bitferry.gemspec: Skipped copy as --dry-run is set (size 996)
118
- Transferred: 5.513 KiB / 5.513 KiB, 100%, 0 B/s, ETA -
119
- Transferred: 3 / 3, 100%
120
- Elapsed time: 0.0s
121
- 2024/03/05 11:46:45 NOTICE:
122
- Transferred: 5.513 KiB / 5.513 KiB, 100%, 0 B/s, ETA -
123
- Transferred: 3 / 3, 100%
124
- Elapsed time: 0.0s
125
- ```
126
-
127
- </details>
128
-
129
- ### Process all intact tasks in sequence
130
-
131
- ```shell
132
- bitferry -v x
133
- ```
134
-
135
- <details>
136
- <summary>...</summary>
137
-
138
- ```
139
- rclone sync --filter -\ .bitferry --filter -\ .bitferry\~ --verbose --progress --metadata --crypt-filename-encoding base32 --crypt-filename-encryption standard --crypt-remote /mnt/usb-drive/backup /data :crypt:
140
- 2024/03/05 11:44:31 INFO : LICENSE: Copied (new)
141
- 2024/03/05 11:44:31 INFO : README.md: Copied (new)
142
- 2024/03/05 11:44:31 INFO : bitferry.gemspec: Copied (new)
143
- Transferred: 5.653 KiB / 5.653 KiB, 100%, 0 B/s, ETA -
144
- Transferred: 3 / 3, 100%
145
- Elapsed time: 0.0s
146
- 2024/03/05 11:44:31 INFO :
147
- Transferred: 5.653 KiB / 5.653 KiB, 100%, 0 B/s, ETA -
148
- Transferred: 3 / 3, 100%
149
- Elapsed time: 0.0s
150
- ```
151
-
152
- </details>
153
-
154
- ### Observe the result
155
-
156
- ```shell
157
- ls -l /mnt/usb-drive/backup
158
- ```
159
-
160
- <details>
161
- <summary>...</summary>
162
-
163
- ```
164
- -rw-r--r-- 1 user user 1044 feb 27 17:09 0u1vi7ka5p88u62kof9k6mf2z00354g6fa0c9a0g6di2f0ocds80
165
- -rw-r--r-- 1 user user 1550 jan 29 11:57 21dgu5vs2c4rjfkieeemjvaf78
166
- -rw-r--r-- 1 user user 3195 mar 5 11:43 m9rhq3q2m5h2q5l1ke00u0gdjc
167
- ```
168
-
169
- </details>
170
-
171
- ### Examine the detailed usage instructions
172
-
173
- ```shell
174
- bitferry c t s -h
175
- ```
176
-
177
- <details>
178
- <summary>...</summary>
179
-
180
- ```
181
- Usage:
182
- bitferry c t s [OPTIONS] SOURCE DESTINATION
183
-
184
- Create source --> destination one way file synchronization task.
185
-
186
- The task operates recursively on two specified endpoints.
187
- This task copies newer source files while skipping unchanged files in destination.
188
- Also, it deletes destination files which are non-existent in source.
189
-
190
- The endpoint may be one of:
191
- * directory -- absolute or relative local directory (/data, ../source, c:\data)
192
- * local:directory, :directory -- absolute local directory (:/data, local:c:\data)
193
- * :tag:directory -- path relative to the intact volume matched by (partial) tag (:fa2c:source/data)
194
-
195
- The former case resolves specified directory againt an intact volume to make it volume-relative.
196
- It is an error if there is no intact volume that encompasses specified directory.
197
- The local: directory is left as is (not resolved against volumes).
198
- The :tag: directory is bound to the specified volume.
199
-
200
-
201
-
202
- The encryption mode is controlled by --encrypt or --decrypt options.
203
- The mandatory password will be read from the standard input channel (pipe or keyboard).
204
-
205
- This task employs the Rclone worker.
206
-
207
- Parameters:
208
- SOURCE Source endpoint specifier
209
- DESTINATION Destination endpoint specifier
210
-
211
- Options:
212
- -e Encrypt files in destination using default profile (alias for -E default)
213
- -d Decrypt source files using default profile (alias for -D default)
214
- -u Use extended encryption profile options (applies to -e, -d)
215
- --process, -X OPTIONS Extra task processing profile/options
216
- --encrypt, -E OPTIONS Encrypt files in destination using specified profile/options
217
- --decrypt, -D OPTIONS Decrypt source files using specified profile/options
218
- --version Print version
219
- --verbose, -v Extensive logging
220
- --quiet, -q Disable logging
221
- --dry-run, -n Simulation mode (make no on-disk changes)
222
- -h, --help print help
223
- ```
224
-
225
- </details>
226
-
227
- ## The rest is about to come
228
-
229
- *Cheers!*
230
-
231
- Oleg A. Khlybov <fougas@mail.ru>
1
+ # Bitferry - file synchronization/backup automation
2
+
3
+ <div align="right"><i>Ein Backup ist kein Backup</i></div><br><br>
4
+
5
+ The [Bitferry](https://github.com/okhlybov/bitferry) is aimed at establishing the automated file synchronization/replication/backup routes between multiple endpoints where the latter can be the local directories, online cloud remotes or portable offline storages.
6
+
7
+ The intended usage ranges from maintaining simple directory copy to another location (disk, mount point) to complex many-to-many (online/offline) data replication/backup solution employing portable media as additional data storage and a means of data propagation between the offsites.
8
+
9
+ The core idea that drives Bitferry is the conversion of full (absolute) endpoints' paths into the volume-relative ones, where the volume is a data file which is put along the endpoint's data and denotes the root of the directory hierarchy. This makes data position-independent which means that Bitferry is then able to restore the tasks' source-destination endpoint connections in spite of the volume location changes, which is a likely scenario in case of portable storage (different UNIX mount points, Windows drives etc.).
10
+
11
+ Bitferry is effectively a frontend to the [Rclone](https://rclone.org) and [Restic](https://restic.net) utilities.
12
+
13
+ ## Features
14
+
15
+ * Multiplatform (Windows / UNIX / macOSX) operation
16
+
17
+ * Automated task-based data processing
18
+
19
+ * One way / two way data synchronization
20
+
21
+ * Recursive directory copy / update / synchronize
22
+
23
+ * Incremental directory backup with snapshotting
24
+
25
+ * File/repository password-based end-to-end encryption
26
+
27
+ * Online cloud storage relay
28
+
29
+ * Offline portable storage (USB flash, HDDs, SSDs etc.) relay
30
+
31
+ ## Use cases
32
+
33
+ * Maintain an update-only files copy in a separate location on the same site
34
+
35
+ * Maintain offline secure two way file synchronization between two offsites
36
+
37
+ * Maintain an incremental files backup on a portable medium with multiple offsite copies of the repository
38
+
39
+ ## Implementation
40
+
41
+ The Bitferry itself is written in [Ruby](https://www.ruby-lang.org) programming language. Being a Ruby code, the Bitferry requires the platform-specific Ruby runtime, version 3.0 or higher.
42
+
43
+ The source code is hosted on [GitHub](https://github.com/okhlybov/bitferry) and the binary releases in form of a GEM package are distributed through the [RubyGems](https://rubygems.org/gems/bitferry) repository channel.
44
+
45
+ In addition, the platform-specific [Rclone](https://github.com/rclone/rclone/releases) and [Restic](https://github.com/restic/restic/releases) executables are required to be accessible through the `PATH` directory list or through the respective `RCLONE` and `RESTIC` environment variables.
46
+
47
+ ## Kickstart
48
+
49
+ ### Install Bitferry
50
+
51
+ ```shell
52
+ gem install bitferry
53
+ ```
54
+
55
+ ### Prepare source Bitferry volume for a mounted local filesystem
56
+
57
+ ```shell
58
+ bitferry create volume /data
59
+ ```
60
+
61
+ ### Prepare destination Bitferry volume for a mounted portable storage
62
+
63
+ ```shell
64
+ bitferry create volume /mnt/usb-drive
65
+ ```
66
+
67
+ ### Ensure the volumes are intact
68
+
69
+ ```shell
70
+ bitferry show
71
+ ```
72
+
73
+ ```
74
+ # Intact volumes
75
+
76
+ d2f10024 /data
77
+ e42f2d8c /mnt/usb-drive
78
+ ```
79
+
80
+ ### Create a (Rclone) sync task with data encryption
81
+
82
+ ```shell
83
+ bitferry create task sync -e /data /mnt/usb-drive/backup
84
+ ```
85
+
86
+ ### Review the changes
87
+
88
+ ```shell
89
+ bitferry
90
+ ```
91
+
92
+ ```
93
+ # Intact volumes
94
+
95
+ d2f10024 /data
96
+ e42f2d8c /mnt/usb-drive
97
+
98
+
99
+ # Intact tasks
100
+
101
+ 89e1c119 encrypt+synchronize :d2f10024: --> :e42f2d8c:backup
102
+ ```
103
+
104
+ ### Perform a dry run of the specific task
105
+
106
+ ```shell
107
+ bitferry process -vn 89e
108
+ ```
109
+
110
+ <details>
111
+ <summary>...</summary>
112
+
113
+ ```
114
+ rclone sync --filter -\ .bitferry --filter -\ .bitferry\~ --verbose --progress --dry-run --metadata --crypt-filename-encoding base32 --crypt-filename-encryption standard --crypt-remote /mnt/usb-drive/backup /data :crypt:
115
+ 2024/03/05 11:46:45 NOTICE: README.md: Skipped copy as --dry-run is set (size 3.073Ki)
116
+ 2024/03/05 11:46:45 NOTICE: LICENSE: Skipped copy as --dry-run is set (size 1.467Ki)
117
+ 2024/03/05 11:46:45 NOTICE: bitferry.gemspec: Skipped copy as --dry-run is set (size 996)
118
+ Transferred: 5.513 KiB / 5.513 KiB, 100%, 0 B/s, ETA -
119
+ Transferred: 3 / 3, 100%
120
+ Elapsed time: 0.0s
121
+ 2024/03/05 11:46:45 NOTICE:
122
+ Transferred: 5.513 KiB / 5.513 KiB, 100%, 0 B/s, ETA -
123
+ Transferred: 3 / 3, 100%
124
+ Elapsed time: 0.0s
125
+ ```
126
+
127
+ </details>
128
+
129
+ ### Process all intact tasks in sequence
130
+
131
+ ```shell
132
+ bitferry -v x
133
+ ```
134
+
135
+ <details>
136
+ <summary>...</summary>
137
+
138
+ ```
139
+ rclone sync --filter -\ .bitferry --filter -\ .bitferry\~ --verbose --progress --metadata --crypt-filename-encoding base32 --crypt-filename-encryption standard --crypt-remote /mnt/usb-drive/backup /data :crypt:
140
+ 2024/03/05 11:44:31 INFO : LICENSE: Copied (new)
141
+ 2024/03/05 11:44:31 INFO : README.md: Copied (new)
142
+ 2024/03/05 11:44:31 INFO : bitferry.gemspec: Copied (new)
143
+ Transferred: 5.653 KiB / 5.653 KiB, 100%, 0 B/s, ETA -
144
+ Transferred: 3 / 3, 100%
145
+ Elapsed time: 0.0s
146
+ 2024/03/05 11:44:31 INFO :
147
+ Transferred: 5.653 KiB / 5.653 KiB, 100%, 0 B/s, ETA -
148
+ Transferred: 3 / 3, 100%
149
+ Elapsed time: 0.0s
150
+ ```
151
+
152
+ </details>
153
+
154
+ ### Observe the result
155
+
156
+ ```shell
157
+ ls -l /mnt/usb-drive/backup
158
+ ```
159
+
160
+ <details>
161
+ <summary>...</summary>
162
+
163
+ ```
164
+ -rw-r--r-- 1 user user 1044 feb 27 17:09 0u1vi7ka5p88u62kof9k6mf2z00354g6fa0c9a0g6di2f0ocds80
165
+ -rw-r--r-- 1 user user 1550 jan 29 11:57 21dgu5vs2c4rjfkieeemjvaf78
166
+ -rw-r--r-- 1 user user 3195 mar 5 11:43 m9rhq3q2m5h2q5l1ke00u0gdjc
167
+ ```
168
+
169
+ </details>
170
+
171
+ ### Examine the detailed usage instructions
172
+
173
+ ```shell
174
+ bitferry c t s -h
175
+ ```
176
+
177
+ <details>
178
+ <summary>...</summary>
179
+
180
+ ```
181
+ Usage:
182
+ bitferry c t s [OPTIONS] SOURCE DESTINATION
183
+
184
+ Create source --> destination one way file synchronization task.
185
+
186
+ The task operates recursively on two specified endpoints.
187
+ This task copies newer source files while skipping unchanged files in destination.
188
+ Also, it deletes destination files which are non-existent in source.
189
+
190
+ The endpoint may be one of:
191
+ * directory -- absolute or relative local directory (/data, ../source, c:\data)
192
+ * local:directory, :directory -- absolute local directory (:/data, local:c:\data)
193
+ * :tag:directory -- path relative to the intact volume matched by (partial) tag (:fa2c:source/data)
194
+
195
+ The former case resolves specified directory againt an intact volume to make it volume-relative.
196
+ It is an error if there is no intact volume that encompasses specified directory.
197
+ The local: directory is left as is (not resolved against volumes).
198
+ The :tag: directory is bound to the specified volume.
199
+
200
+
201
+
202
+ The encryption mode is controlled by --encrypt or --decrypt options.
203
+ The mandatory password will be read from the standard input channel (pipe or keyboard).
204
+
205
+ This task employs the Rclone worker.
206
+
207
+ Parameters:
208
+ SOURCE Source endpoint specifier
209
+ DESTINATION Destination endpoint specifier
210
+
211
+ Options:
212
+ -e Encrypt files in destination using default profile (alias for -E default)
213
+ -d Decrypt source files using default profile (alias for -D default)
214
+ -u Use extended encryption profile options (applies to -e, -d)
215
+ --process, -X OPTIONS Extra task processing profile/options
216
+ --encrypt, -E OPTIONS Encrypt files in destination using specified profile/options
217
+ --decrypt, -D OPTIONS Decrypt source files using specified profile/options
218
+ --version Print version
219
+ --verbose, -v Extensive logging
220
+ --quiet, -q Disable logging
221
+ --dry-run, -n Simulation mode (make no on-disk changes)
222
+ -h, --help print help
223
+ ```
224
+
225
+ </details>
226
+
227
+ ## The rest is about to come
228
+
229
+ *Cheers!*
230
+
231
+ Oleg A. Khlybov <fougas@mail.ru>
data/bin/bitferry CHANGED
@@ -1,3 +1,3 @@
1
- #!/usr/bin/env ruby
2
- require 'bitferry/cli'
1
+ #!/usr/bin/env ruby
2
+ require 'bitferry/cli'
3
3
  #
data/bin/bitferryfx CHANGED
@@ -1,3 +1,3 @@
1
- #!/usr/bin/env ruby
2
- require 'bitferry/fx'
1
+ #!/usr/bin/env ruby
2
+ require 'bitferry/fx'
3
3
  #