imap-backup 6.3.0 → 7.0.0.rc1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 1725278faef6eec96440019b9e02e50bd4d46c036bf86e3dbfe0d7ad56350350
4
- data.tar.gz: d923d786db2db4e1b23c28efbdd3104ea709388be3aedf4d82ee7714862001cf
3
+ metadata.gz: d858a14c5bbb9300fd99fe6338cc2ff83fd596071dc2db48ca792f45eee430a3
4
+ data.tar.gz: c28f18cd459cd9c2c9a62fb5a55234615218bcff2c3c46e0422af9c6893e6043
5
5
  SHA512:
6
- metadata.gz: 295c250e9991506cccc422b00e10623ac53acdcb376a8d5f6044406890668c6dc512216af455a55bfb701e3eaaeb4bc4908942416fe7578d6814f4b017a8e465
7
- data.tar.gz: fa50ad39547df62b535f592604fb72477d134e7206c8be9e01dedc6765edb6bc2d3ac1faad75eefb402f9c8ba9a5050e93ca6cfeb015653c68db0a0e0427329f
6
+ metadata.gz: c669d9a170ec1d500e898324bbdc2072cc42597e9c2d30b6b284b7095151afecd0451c9fa7cb0adb93bbf46f1f2b1ae188554507141161acb6ba125349c1cee9
7
+ data.tar.gz: c53ecb54ed6d33ab574caa18e62ae57e214e76975af65c72b11b8b2612d45487422be234c025966c792260d86d20e5431bf27845772e40d82cc8fea0132f9963
data/README.md CHANGED
@@ -5,218 +5,108 @@
5
5
 
6
6
  # imap-backup
7
7
 
8
- *Backup GMail (or other IMAP) accounts to disk*
8
+ Backup, restore and migrate email accounts.
9
+
10
+ The backups can then be restored, used to migrate to another service,
11
+ inspected or exported.
9
12
 
10
13
  * [Source Code]
11
- * [API documentation]
14
+ * [Documentation]
12
15
  * [Rubygem]
13
16
  * [CI Status]
14
17
 
15
18
  [Source Code]: https://github.com/joeyates/imap-backup "Source code at GitHub"
16
- [API documentation]: https://rubydoc.info/gems/imap-backup/frames "RDoc API Documentation at Rubydoc.info"
19
+ [Documentation]: https://rubydoc.info/gems/imap-backup/frames "RDoc API Documentation at Rubydoc.info"
17
20
  [Rubygem]: https://rubygems.org/gems/imap-backup "Ruby gem at rubygems.org"
18
21
  [CI Status]: https://github.com/joeyates/imap-backup/actions/workflows/main.yml
19
22
 
20
- # Installation
23
+ # Backup Emails
21
24
 
22
- ```shell
23
- $ gem install 'imap-backup'
24
- ```
25
+ imap-backup downloads emails and stores them on disk.
25
26
 
26
- # Commands
27
+ The backup is incremental and interruptable, so backups won't get messed if your connection goes down during an operation.
27
28
 
28
- For a full list, run
29
+ # Installation
29
30
 
30
- ```
31
- $ imap-backup help
32
- ```
31
+ ## Homebrew (macOS)
33
32
 
34
- For more information about a command, run
33
+ If you have [Homebrew](https://brew.sh/), do this:
35
34
 
36
- ```shell
37
- $ imap-backup help COMMAND
35
+ ```sh
36
+ brew install imap-backup
38
37
  ```
39
38
 
40
- # Setup
41
-
42
- In order to do backups, you need to add accounts via a menu-driven command
43
- line program:
39
+ ## As a Ruby Gem
44
40
 
45
- Run:
46
-
47
- ```shell
48
- $ imap-backup setup
41
+ ```sh
42
+ gem install imap-backup
49
43
  ```
50
44
 
51
- ## GMail
52
-
53
- To use imap-backup with GMail, you will need to enable 'App passwords' on your account.
54
-
55
- ## Folders
56
-
57
- By default, all folders are backed-up. You can override this by choosing
58
- specific folders.
45
+ If that doesn't work, see the [detailed installation instructions](docs/installation/rubygem.md).
59
46
 
60
- ## Configuration file
47
+ ## From Source Code
61
48
 
62
- `setup` creates the file `~/.imap-backup/config.json`
49
+ If you want to use imap-backup directly from the source code, see [here](docs/installation/source.md).
63
50
 
64
- E.g.:
51
+ # Setup
65
52
 
66
- ```json
67
- {
68
- "accounts": [
69
- {
70
- "username": "my.user@gmail.com",
71
- "password": "secret",
72
- "local_path": "/path/to/backup/root",
73
- "folders":
74
- [
75
- {"name": "[Gmail]/All Mail"},
76
- {"name": "my_folder"}
77
- ]
78
- }
79
- ]
80
- }
81
- ```
53
+ As a first step, you need to add accounts via a menu-driven command
54
+ line program:
82
55
 
83
- It connects to GMail by default, but you can also specify a server:
84
-
85
- ```json
86
- {
87
- "accounts": [
88
- {
89
- "username": "my.user@gmail.com",
90
- "password": "secret",
91
- "server": "my.imap.example.com",
92
- "local_path": "/path/to/backup/root",
93
- "folders":
94
- [
95
- {"name": "[Gmail]/All Mail"},
96
- {"name": "my_folder"}
97
- ]
98
- }
99
- ]
100
- }
101
- ```
56
+ Run:
102
57
 
103
- ## Connection options
104
-
105
- You can override the parameters passed to `Net::IMAP` with `connection_options`.
106
-
107
- Specifically, if you are using a self-signed certificate and get SSL errors, e.g.
108
- `certificate verify failed`, you can choose to not verify the TLS connection:
109
-
110
- ```json
111
- {
112
- "accounts": [
113
- {
114
- "username": "my.user@gmail.com",
115
- "password": "secret",
116
- "server": "my.imap.example.com",
117
- "local_path": "/path/to/backup/root",
118
- "folders": [
119
- {"name": "[Gmail]/All Mail"},
120
- {"name": "my_folder"}
121
- ],
122
- "connection_options": {
123
- "ssl": {"verify_mode": 0},
124
- "port": 993
125
- }
126
- }
127
- ]
128
- }
58
+ ```sh
59
+ imap-backup setup
129
60
  ```
130
61
 
131
- # Security
132
-
133
- Note that email usernames and passwords are held in plain text
134
- in the configuration file.
135
-
136
- The directory ~/.imap-backup, the configuration file and all backup
137
- directories have their access permissions set to only allow access
138
- by your user. This is not done on Windows - see below.
139
-
140
- ## Windows
141
-
142
- Due to the complexity of managing permissions on Windows,
143
- directory and file access permissions are not set explicity.
62
+ ## GMail
144
63
 
145
- A pull request that implements permissions management on Windows
146
- would be welcome!
64
+ To use imap-backup with GMail, you will need to enable 'App passwords' on your account.
147
65
 
148
66
  # Run Backup
149
67
 
150
68
  Manually, from the command line:
151
69
 
152
- ```shell
153
- $ imap-backup
70
+ ```sh
71
+ imap-backup
154
72
  ```
155
73
 
156
74
  Alternatively, add it to your crontab.
157
75
 
158
- # Result
76
+ Emails are stored on disk in [Mbox files](./docs/files/mboxrd.md) for each folder, with metadata
77
+ stored in [Imap files](./docs/files/imap.md).
159
78
 
160
- Each folder is saved to an mbox file.
161
- Alongside each mbox is a file with extension '.imap', which lists the source IMAP
162
- UIDs to allow a full restore.
163
-
164
- # Local commands
79
+ # Commands
165
80
 
166
- There a various commands for viewing local backup status.
81
+ * [folders](./commands/folders.md)
82
+ * [restore](./commands/restore.md)
83
+ * [status](./commands/status.md)
167
84
 
168
- To view the list, use
85
+ For a full list of available commands, run
169
86
 
170
- ```shell
171
- $ imap_backup help local
87
+ ```sh
88
+ imap-backup help
172
89
  ```
173
90
 
174
- # Troubleshooting
175
-
176
- If you have problems:
91
+ For more information about a command, run
177
92
 
178
- 1. ensure that you have the latest release,
179
- 2. turn on debugging output:
180
-
181
- ```json
182
- {
183
- "accounts":
184
- [
185
- ...
186
- ],
187
- "debug": true
188
- }
93
+ ```sh
94
+ imap-backup help COMMAND
189
95
  ```
190
96
 
191
- # Restore
192
-
193
- All missing messages are pushed to the IMAP server.
194
- Existing messages are left unchanged.
97
+ ## Configuration
195
98
 
196
- This functionality requires that the IMAP server supports the UIDPLUS
197
- extension to IMAP4.
99
+ `imap-backup setup` creates the file `~/.imap-backup/config.json`.
198
100
 
199
- # Other Usage
200
-
201
- List IMAP folders:
202
-
203
- ```shell
204
- $ imap-backup folders
205
- ```
101
+ [More information about configuration is available in the specific documentation](./docs/configuration.md).
206
102
 
207
- Get statistics of emails to download per folder:
208
-
209
- ```shell
210
- $ imap-backup status
211
- ```
103
+ # Troubleshooting
212
104
 
213
- # Design Goals
105
+ If you have problems:
214
106
 
215
- * Secure - use a local file protected by permissions
216
- * Restartable - calculate start point based on already downloaded messages
217
- * Standalone - do not rely on an email client or MTA
107
+ 1. ensure that you have the latest release,
108
+ 2. turn on debugging output via the `imap-backup setup` main menu.
218
109
 
219
- # Documentation
110
+ # Development
220
111
 
221
- * [Development](./docs/development.md)
222
- * [Restore](./docs/restore.md)
112
+ See the [Development documentation](./docs/development.md)
@@ -0,0 +1,23 @@
1
+ Configuration is stored in a JSON file.
2
+
3
+ The format is documented [here](./docs/files/config.json).
4
+
5
+ # Folders
6
+
7
+ By default, all folders are backed-up. You can override this by choosing
8
+ specific folders.
9
+
10
+ # Connection options
11
+
12
+ You can override the parameters passed to `Net::IMAP` with `connection_options`.
13
+
14
+ Specifically, if you are using a self-signed certificate and get SSL errors, e.g.
15
+ `certificate verify failed`, you can choose to not verify the TLS connection.
16
+
17
+ Connection options can be entered via `imap-backup setup` as JSON.
18
+
19
+ Choose the account, then 'modify connection options'.
20
+
21
+ For example:
22
+
23
+ ![Entering connection options as JSON](./images/entering-connection-options-as-json.png "Entering connection options as JSON")
data/docs/development.md CHANGED
@@ -1,3 +1,9 @@
1
+ # Design Goals
2
+
3
+ * Secure - use a local file protected by permissions
4
+ * Restartable - calculate start point based on already downloaded messages
5
+ * Standalone - do not rely on an email client or MTA
6
+
1
7
  # Testing
2
8
 
3
9
  ## Feature Specs
data/imap-backup.gemspec CHANGED
@@ -19,10 +19,12 @@ Gem::Specification.new do |gem|
19
19
 
20
20
  gem.executables = gem.files.grep(%r{^bin/}).map { |f| File.basename(f) }
21
21
  gem.require_paths = ["lib"]
22
- gem.required_ruby_version = ">= 2.5"
22
+ gem.required_ruby_version = ">= 2.6"
23
23
 
24
24
  gem.add_runtime_dependency "highline"
25
25
  gem.add_runtime_dependency "mail"
26
+ gem.add_runtime_dependency "net-imap"
27
+ gem.add_runtime_dependency "net-smtp"
26
28
  gem.add_runtime_dependency "os"
27
29
  gem.add_runtime_dependency "rake"
28
30
  gem.add_runtime_dependency "thor", "~> 1.1"
@@ -73,7 +73,7 @@ module Imap::Backup
73
73
  examine
74
74
  fetch_data_items =
75
75
  retry_on_error(errors: UID_FETCH_RETRY_CLASSES) do
76
- client.uid_fetch(uids, [BODY_ATTRIBUTE])
76
+ client.uid_fetch(uids, [BODY_ATTRIBUTE, "FLAGS"])
77
77
  end
78
78
  return nil if fetch_data_items.nil?
79
79
 
@@ -82,18 +82,19 @@ module Imap::Backup
82
82
 
83
83
  {
84
84
  uid: attributes["UID"],
85
- body: attributes[BODY_ATTRIBUTE]
85
+ body: attributes[BODY_ATTRIBUTE],
86
+ flags: attributes["FLAGS"]
86
87
  }
87
88
  end
88
89
  rescue FolderNotFound
89
90
  nil
90
91
  end
91
92
 
92
- def append(message)
93
+ def append(message, flags: nil)
93
94
  body = message.imap_body
94
95
  date = message.date&.to_time
95
96
  retry_on_error(errors: APPEND_RETRY_CLASSES, limit: 3) do
96
- response = client.append(utf7_encoded_name, body, nil, date)
97
+ response = client.append(utf7_encoded_name, body, flags, date)
97
98
  extract_uid(response)
98
99
  end
99
100
  end
@@ -73,7 +73,7 @@ module Imap::Backup
73
73
  Skipped #{uid}
74
74
  MESSAGE
75
75
 
76
- serializer.append uid, message
76
+ serializer.append uid, message, []
77
77
  end
78
78
  end
79
79
 
@@ -30,7 +30,7 @@ module Imap::Backup
30
30
 
31
31
  def save
32
32
  ensure_loaded!
33
- FileUtils.mkdir(path) if !File.directory?(path)
33
+ FileUtils.mkdir_p(path) if !File.directory?(path)
34
34
  make_private(path) if !windows?
35
35
  remove_modified_flags
36
36
  remove_deleted_accounts
@@ -79,6 +79,7 @@ module Imap::Backup
79
79
  def handle_uid_and_body(uid_and_body, index)
80
80
  uid = uid_and_body[:uid]
81
81
  body = uid_and_body[:body]
82
+ flags = uid_and_body[:flags]
82
83
  case
83
84
  when !body
84
85
  info("Fetch returned empty body - skipping")
@@ -86,7 +87,7 @@ module Imap::Backup
86
87
  info("Fetch returned empty UID - skipping")
87
88
  else
88
89
  debug("uid: #{uid} (#{index}/#{uids.count}) - #{body.size} bytes")
89
- serializer.append uid, body
90
+ serializer.append uid, body, flags
90
91
  end
91
92
  end
92
93
 
@@ -16,7 +16,7 @@ module Imap::Backup
16
16
  ensure_destination_empty!
17
17
 
18
18
  Logger.logger.debug "[#{folder.name}] #{count} to migrate"
19
- serializer.each_message(serializer.uids).with_index do |(uid, message), i|
19
+ serializer.each_message(serializer.uids).with_index do |(uid, message, flags), i|
20
20
  next if message.nil?
21
21
 
22
22
  log_prefix = "[#{folder.name}] uid: #{uid} (#{i + 1}/#{count}) -"
@@ -25,7 +25,7 @@ module Imap::Backup
25
25
  )
26
26
 
27
27
  begin
28
- folder.append(message)
28
+ folder.append(message, flags: flags)
29
29
  rescue StandardError => e
30
30
  Logger.logger.warn "#{log_prefix} append error: #{e}"
31
31
  end
@@ -10,7 +10,7 @@ module Imap::Backup
10
10
  @mbox = mbox
11
11
  end
12
12
 
13
- def run(uid:, message:)
13
+ def run(uid:, message:, flags:)
14
14
  raise "Can't add messages without uid_validity" if !imap.uid_validity
15
15
 
16
16
  uid = uid.to_i
@@ -21,19 +21,20 @@ module Imap::Backup
21
21
  return
22
22
  end
23
23
 
24
- do_append uid, message
24
+ do_append uid, message, flags
25
25
  end
26
26
 
27
27
  private
28
28
 
29
- def do_append(uid, message)
29
+ def do_append(uid, message, flags)
30
30
  mboxrd_message = Email::Mboxrd::Message.new(message)
31
31
  initial = mbox.length || 0
32
32
  mbox_appended = false
33
33
  begin
34
- mbox.append mboxrd_message.to_serialized
34
+ serialized = mboxrd_message.to_serialized
35
+ mbox.append serialized
35
36
  mbox_appended = true
36
- imap.append uid
37
+ imap.append uid, serialized.length, flags
37
38
  rescue StandardError => e
38
39
  mbox.rewind(initial) if mbox_appended
39
40
 
@@ -2,7 +2,7 @@ require "json"
2
2
 
3
3
  module Imap::Backup
4
4
  class Serializer::Imap
5
- CURRENT_VERSION = 2
5
+ CURRENT_VERSION = 3
6
6
 
7
7
  attr_reader :folder_path
8
8
  attr_reader :loaded
@@ -11,7 +11,7 @@ module Imap::Backup
11
11
  @folder_path = folder_path
12
12
  @loaded = false
13
13
  @uid_validity = nil
14
- @uids = nil
14
+ @messages = nil
15
15
  @version = nil
16
16
  end
17
17
 
@@ -23,21 +23,38 @@ module Imap::Backup
23
23
  true
24
24
  end
25
25
 
26
- def append(uid)
27
- uids << uid
26
+ def append(uid, length, flags = [])
27
+ offset =
28
+ if messages.empty?
29
+ 0
30
+ else
31
+ last_message = messages[-1]
32
+ last_message[:offset] + last_message[:length]
33
+ end
34
+ messages << {uid: uid, offset: offset, length: length, flags: flags}
28
35
  save
29
36
  end
30
37
 
38
+ def get(uid)
39
+ messages.find { |m| m[:uid] == uid }
40
+ end
41
+
31
42
  def delete
32
43
  return if !exist?
33
44
 
34
45
  File.unlink(pathname)
46
+ @loaded = false
47
+ @messages = nil
48
+ @uid_validity = nil
49
+ @version = nil
35
50
  end
36
51
 
52
+ # Deprecated
37
53
  def include?(uid)
38
54
  uids.include?(uid)
39
55
  end
40
56
 
57
+ # Deprecated
41
58
  def index(uid)
42
59
  uids.find_index(uid)
43
60
  end
@@ -60,20 +77,26 @@ module Imap::Backup
60
77
  def uid_validity=(value)
61
78
  ensure_loaded
62
79
  @uid_validity = value
63
- @uids ||= []
64
80
  save
65
81
  end
66
82
 
67
- def uids
83
+ # Make private
84
+ def messages
68
85
  ensure_loaded
69
- @uids || []
86
+ @messages
87
+ end
88
+
89
+ # Deprecated
90
+ def uids
91
+ messages.map { |m| m[:uid] }
70
92
  end
71
93
 
72
94
  def update_uid(old, new)
73
- index = uids.find_index(old.to_i)
95
+ index = messages.find_index { |m| m[:uid] == old }
74
96
  return if index.nil?
75
97
 
76
- uids[index] = new.to_i
98
+ updated = messages[index].merge({uid: new})
99
+ messages[index] = updated
77
100
  save
78
101
  end
79
102
 
@@ -97,11 +120,11 @@ module Imap::Backup
97
120
 
98
121
  data = load
99
122
  if data
100
- @uids = data[:uids].map(&:to_i)
123
+ @messages = data[:messages]
101
124
  @uid_validity = data[:uid_validity]
102
125
  @version = data[:version]
103
126
  else
104
- @uids = []
127
+ @messages = []
105
128
  @uid_validity = nil
106
129
  @version = CURRENT_VERSION
107
130
  end
@@ -121,8 +144,10 @@ module Imap::Backup
121
144
 
122
145
  return nil if !data.key?(:version)
123
146
  return nil if !data.key?(:uid_validity)
124
- return nil if !data.key?(:uids)
125
- return nil if !data[:uids].is_a?(Array)
147
+ return nil if !data.key?(:messages)
148
+ return nil if !data[:messages].is_a?(Array)
149
+
150
+ data[:messages].each { |m| m[:flags] = m[:flags].map(&:to_sym) }
126
151
 
127
152
  data
128
153
  end
@@ -135,7 +160,7 @@ module Imap::Backup
135
160
  data = {
136
161
  version: @version,
137
162
  uid_validity: @uid_validity,
138
- uids: @uids
163
+ messages: @messages
139
164
  }
140
165
  content = data.to_json
141
166
  File.open(pathname, "w") { |f| f.write content }
@@ -16,6 +16,13 @@ module Imap::Backup
16
16
  end
17
17
  end
18
18
 
19
+ def read(offset, length)
20
+ File.open(pathname, "rb") do |f|
21
+ f.seek offset
22
+ f.read length
23
+ end
24
+ end
25
+
19
26
  def delete
20
27
  return if !exist?
21
28
 
@@ -8,8 +8,8 @@ module Imap::Backup
8
8
  @mbox_pathname = mbox_pathname
9
9
  end
10
10
 
11
- def each
12
- return enum_for(:each) if !block_given?
11
+ def each(&block)
12
+ return enum_for(:each) if !block
13
13
 
14
14
  File.open(mbox_pathname, "rb") do |f|
15
15
  lines = []
@@ -26,8 +26,14 @@ module Imap::Backup
26
26
  end
27
27
  end
28
28
 
29
- yield lines.join if lines.count.positive?
29
+ block.call(lines.join) if lines.count.positive?
30
30
  end
31
31
  end
32
+
33
+ def map(&block)
34
+ return enum_for(:map) if !block
35
+
36
+ each.map { |line| block.call(line) }
37
+ end
32
38
  end
33
39
  end
@@ -12,17 +12,16 @@ module Imap::Backup
12
12
  end
13
13
 
14
14
  def run(uids:)
15
- indexes = uids.each.with_object({}) do |uid_maybe_string, acc|
15
+ uids.each do |uid_maybe_string|
16
16
  uid = uid_maybe_string.to_i
17
- index = imap.index(uid)
18
- acc[index] = uid if index
19
- end
20
- enumerator = Serializer::MboxEnumerator.new(mbox.pathname)
21
- enumerator.each.with_index do |raw, i|
22
- uid = indexes[i]
23
- next if !uid
17
+ message = imap.get(uid)
18
+
19
+ next if !message
20
+
21
+ raw = mbox.read(message[:offset], message[:length])
22
+ body = Email::Mboxrd::Message.from_serialized(raw)
24
23
 
25
- yield uid, Email::Mboxrd::Message.from_serialized(raw)
24
+ yield message[:uid], body, message[:flags]
26
25
  end
27
26
  end
28
27
  end
@@ -0,0 +1,90 @@
1
+ require "json"
2
+
3
+ require "imap/backup/serializer/mbox_enumerator"
4
+
5
+ module Imap::Backup
6
+ class Serializer::Version2Migrator
7
+ attr_reader :folder_path
8
+
9
+ def initialize(folder_path)
10
+ @folder_path = folder_path
11
+ end
12
+
13
+ def required?
14
+ return false if !mbox_exists?
15
+ return false if !imap_exists?
16
+ return false if !data
17
+ return false if data[:version] != 2
18
+ return false if !data[:uid_validity]
19
+ return false if !uids.is_a?(Array)
20
+
21
+ true
22
+ end
23
+
24
+ def run
25
+ return false if !required?
26
+
27
+ messages = message_uids_and_lengths
28
+
29
+ return false if !messages
30
+
31
+ imap.delete
32
+ imap.uid_validity = data[:uid_validity]
33
+ messages.map { |m| imap.append(m[:uid], m[:length]) }
34
+
35
+ true
36
+ end
37
+
38
+ private
39
+
40
+ def imap_pathname
41
+ "#{folder_path}.imap"
42
+ end
43
+
44
+ def imap_exists?
45
+ File.exist?(imap_pathname)
46
+ end
47
+
48
+ def mbox_pathname
49
+ "#{folder_path}.mbox"
50
+ end
51
+
52
+ def mbox_exists?
53
+ File.exist?(mbox_pathname)
54
+ end
55
+
56
+ def data
57
+ @data ||=
58
+ begin
59
+ content = File.read(imap_pathname)
60
+ JSON.parse(content, symbolize_names: true)
61
+ rescue JSON::ParserError
62
+ nil
63
+ end
64
+ end
65
+
66
+ def uids
67
+ data[:uids]
68
+ end
69
+
70
+ def message_uids_and_lengths
71
+ enumerator = Serializer::MboxEnumerator.new(mbox_pathname)
72
+ messages = enumerator.map.with_index do |raw, i|
73
+ length = raw.length
74
+ message = {
75
+ uid: uids[i],
76
+ length: length
77
+ }
78
+ message
79
+ end
80
+
81
+ return nil if messages.count != uids.count
82
+
83
+ messages
84
+ end
85
+
86
+ def imap
87
+ @imap ||= Serializer::Imap.new(folder_path)
88
+ end
89
+ end
90
+ end
@@ -6,6 +6,7 @@ require "imap/backup/serializer/imap"
6
6
  require "imap/backup/serializer/mbox"
7
7
  require "imap/backup/serializer/mbox_enumerator"
8
8
  require "imap/backup/serializer/message_enumerator"
9
+ require "imap/backup/serializer/version2_migrator"
9
10
  require "imap/backup/serializer/unused_name_finder"
10
11
 
11
12
  module Imap::Backup
@@ -31,6 +32,8 @@ module Imap::Backup
31
32
  # Returns true if there are existing, valid files
32
33
  # false otherwise (in which case any existing files are deleted)
33
34
  def validate!
35
+ optionally_migrate2to3
36
+
34
37
  return true if imap.valid? && mbox.valid?
35
38
 
36
39
  imap.delete
@@ -62,11 +65,11 @@ module Imap::Backup
62
65
  internal_force_uid_validity(value)
63
66
  end
64
67
 
65
- def append(uid, message)
68
+ def append(uid, message, flags)
66
69
  validate!
67
70
 
68
71
  appender = Serializer::Appender.new(folder: folder, imap: imap, mbox: mbox)
69
- appender.run(uid: uid, message: message)
72
+ appender.run(uid: uid, message: message, flags: flags)
70
73
  end
71
74
 
72
75
  def load(uid_maybe_string)
@@ -79,12 +82,6 @@ module Imap::Backup
79
82
  internal_load_nth(message_index)
80
83
  end
81
84
 
82
- def load_nth(index)
83
- validate!
84
-
85
- internal_load_nth(index)
86
- end
87
-
88
85
  def each_message(required_uids, &block)
89
86
  validate!
90
87
 
@@ -134,6 +131,19 @@ module Imap::Backup
134
131
  end
135
132
  end
136
133
 
134
+ def optionally_migrate2to3
135
+ migrator = Version2Migrator.new(folder_path)
136
+ return if !migrator.required?
137
+
138
+ Logger.logger.info <<~MESSAGE
139
+ Local metadata for folder '#{folder_path}' is currently stored in the version 2 format.
140
+
141
+ This will now be transformed into the version 3 format.
142
+ MESSAGE
143
+
144
+ migrator.run
145
+ end
146
+
137
147
  def folder_path
138
148
  self.class.folder_path_for(path: path, folder: folder)
139
149
  end
@@ -13,7 +13,6 @@ module Imap::Backup
13
13
  def email(default = "")
14
14
  highline.ask("email address: ") do |q|
15
15
  q.default = default
16
- q.readline = true
17
16
  q.validate = EMAIL_MATCHER
18
17
  q.responses[:not_valid] = "Enter a valid email address "
19
18
  end
@@ -19,14 +19,14 @@ module Imap::Backup
19
19
  return if count.zero?
20
20
 
21
21
  Logger.logger.debug "[#{folder.name}] #{count} to restore"
22
- serializer.each_message(missing_uids).with_index do |(uid, message), i|
23
- upload_message uid, message, i + 1
22
+ serializer.each_message(missing_uids).with_index do |(uid, message, flags), i|
23
+ upload_message uid, message, flags, i + 1
24
24
  end
25
25
  end
26
26
 
27
27
  private
28
28
 
29
- def upload_message(uid, message, index)
29
+ def upload_message(uid, message, flags, index)
30
30
  return if message.nil?
31
31
 
32
32
  log_prefix = "[#{folder.name}] uid: #{uid} (#{index}/#{count}) -"
@@ -35,7 +35,7 @@ module Imap::Backup
35
35
  )
36
36
 
37
37
  begin
38
- new_uid = folder.append(message)
38
+ new_uid = folder.append(message, flags: flags)
39
39
  serializer.update_uid(uid, new_uid)
40
40
  rescue StandardError => e
41
41
  Logger.logger.warn "#{log_prefix} append error: #{e}"
@@ -1,9 +1,9 @@
1
1
  module Imap; end
2
2
 
3
3
  module Imap::Backup
4
- MAJOR = 6
5
- MINOR = 3
4
+ MAJOR = 7
5
+ MINOR = 0
6
6
  REVISION = 0
7
- PRE = nil
7
+ PRE = "rc1".freeze
8
8
  VERSION = [MAJOR, MINOR, REVISION, PRE].compact.map(&:to_s).join(".")
9
9
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: imap-backup
3
3
  version: !ruby/object:Gem::Version
4
- version: 6.3.0
4
+ version: 7.0.0.rc1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Joe Yates
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2022-08-27 00:00:00.000000000 Z
11
+ date: 2022-08-30 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: highline
@@ -38,6 +38,34 @@ dependencies:
38
38
  - - ">="
39
39
  - !ruby/object:Gem::Version
40
40
  version: '0'
41
+ - !ruby/object:Gem::Dependency
42
+ name: net-imap
43
+ requirement: !ruby/object:Gem::Requirement
44
+ requirements:
45
+ - - ">="
46
+ - !ruby/object:Gem::Version
47
+ version: '0'
48
+ type: :runtime
49
+ prerelease: false
50
+ version_requirements: !ruby/object:Gem::Requirement
51
+ requirements:
52
+ - - ">="
53
+ - !ruby/object:Gem::Version
54
+ version: '0'
55
+ - !ruby/object:Gem::Dependency
56
+ name: net-smtp
57
+ requirement: !ruby/object:Gem::Requirement
58
+ requirements:
59
+ - - ">="
60
+ - !ruby/object:Gem::Version
61
+ version: '0'
62
+ type: :runtime
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '0'
41
69
  - !ruby/object:Gem::Dependency
42
70
  name: os
43
71
  requirement: !ruby/object:Gem::Requirement
@@ -189,6 +217,7 @@ files:
189
217
  - LICENSE
190
218
  - README.md
191
219
  - bin/imap-backup
220
+ - docs/configuration.md
192
221
  - docs/development.md
193
222
  - docs/restore.md
194
223
  - docs/setup.md
@@ -237,6 +266,7 @@ files:
237
266
  - lib/imap/backup/serializer/mbox_enumerator.rb
238
267
  - lib/imap/backup/serializer/message_enumerator.rb
239
268
  - lib/imap/backup/serializer/unused_name_finder.rb
269
+ - lib/imap/backup/serializer/version2_migrator.rb
240
270
  - lib/imap/backup/setup.rb
241
271
  - lib/imap/backup/setup/account.rb
242
272
  - lib/imap/backup/setup/account/header.rb
@@ -264,12 +294,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
264
294
  requirements:
265
295
  - - ">="
266
296
  - !ruby/object:Gem::Version
267
- version: '2.5'
297
+ version: '2.6'
268
298
  required_rubygems_version: !ruby/object:Gem::Requirement
269
299
  requirements:
270
- - - ">="
300
+ - - ">"
271
301
  - !ruby/object:Gem::Version
272
- version: '0'
302
+ version: 1.3.1
273
303
  requirements: []
274
304
  rubygems_version: 3.1.6
275
305
  signing_key: