imap-backup 6.1.0 → 7.0.0.rc1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2a05fedc57275cadd2a1bd22efbe88ef5ec6ee72843400499cac8b8df8e80c35
-  data.tar.gz: f0e8e859b5fb1bb8480730b3ea62abc7470f5d0ddb469113bbdac0e4f2a88d1a
+  metadata.gz: d858a14c5bbb9300fd99fe6338cc2ff83fd596071dc2db48ca792f45eee430a3
+  data.tar.gz: c28f18cd459cd9c2c9a62fb5a55234615218bcff2c3c46e0422af9c6893e6043
 SHA512:
-  metadata.gz: bfb4de1a3048a9f29ee30fb56c7493524a7d688a6eab2ec3307005ba3c8d341a99c961acd1d8f588384dc91571b4528482931e52724df42dff19a26553568fb0
-  data.tar.gz: a411ad26b4ce3eb631c97d94c9242c1eb48fd57d23cffcec84f5c829d9354b506115b6c8278ee38430e27f0d6ca1400d55f74adce5eb99f94db2963f235c6d7a
+  metadata.gz: c669d9a170ec1d500e898324bbdc2072cc42597e9c2d30b6b284b7095151afecd0451c9fa7cb0adb93bbf46f1f2b1ae188554507141161acb6ba125349c1cee9
+  data.tar.gz: c53ecb54ed6d33ab574caa18e62ae57e214e76975af65c72b11b8b2612d45487422be234c025966c792260d86d20e5431bf27845772e40d82cc8fea0132f9963

data/README.md

@@ -5,218 +5,108 @@
 
 # imap-backup
 
-*Backup GMail (or other IMAP) accounts to disk*
+Backup, restore and migrate email accounts.
+
+The backups can then be restored, used to migrate to another service,
+inspected or exported.
 
   * [Source Code]
-  * [API documentation]
+  * [Documentation]
   * [Rubygem]
   * [CI Status]
 
 [Source Code]: https://github.com/joeyates/imap-backup "Source code at GitHub"
-[API documentation]: https://rubydoc.info/gems/imap-backup/frames "RDoc API Documentation at Rubydoc.info"
+[Documentation]: https://rubydoc.info/gems/imap-backup/frames "RDoc API Documentation at Rubydoc.info"
 [Rubygem]: https://rubygems.org/gems/imap-backup "Ruby gem at rubygems.org"
 [CI Status]: https://github.com/joeyates/imap-backup/actions/workflows/main.yml
 
-# Installation
+# Backup Emails
 
-```shell
-$ gem install 'imap-backup'
-```
+imap-backup downloads emails and stores them on disk.
 
-# Commands
+The backup is incremental and interruptable, so backups won't get messed if your connection goes down during an operation.
 
-For a full list, run
+# Installation
 
-```
-$ imap-backup help
-```
+## Homebrew (macOS)
 
-For more information about a command, run
+If you have [Homebrew](https://brew.sh/), do this:
 
-```shell
-$ imap-backup help COMMAND
+```sh
+brew install imap-backup
 ```
 
-# Setup
-
-In order to do backups, you need to add accounts via a menu-driven command
-line program:
+## As a Ruby Gem
 
-Run:
-
-```shell
-$ imap-backup setup
+```sh
+gem install imap-backup
 ```
 
-## GMail
-
-To use imap-backup with GMail, you will need to enable 'App passwords' on your account.
-
-## Folders
-
-By default, all folders are backed-up. You can override this by choosing
-specific folders.
+If that doesn't work, see the [detailed installation instructions](docs/installation/rubygem.md).
 
-## Configuration file
+## From Source Code
 
-`setup` creates the file `~/.imap-backup/config.json`
+If you want to use imap-backup directly from the source code, see [here](docs/installation/source.md).
 
-E.g.:
+# Setup
 
-```json
-{
-  "accounts": [
-    {
-      "username": "my.user@gmail.com",
-      "password": "secret",
-      "local_path": "/path/to/backup/root",
-      "folders":
-        [
-          {"name": "[Gmail]/All Mail"},
-          {"name": "my_folder"}
-        ]
-    }
-  ]
-}
-```
+As a first step, you need to add accounts via a menu-driven command
+line program:
 
-It connects to GMail by default, but you can also specify a server:
-
-```json
-{
-  "accounts": [
-    {
-      "username": "my.user@gmail.com",
-      "password": "secret",
-      "server": "my.imap.example.com",
-      "local_path": "/path/to/backup/root",
-      "folders":
-        [
-          {"name": "[Gmail]/All Mail"},
-          {"name": "my_folder"}
-        ]
-    }
-  ]
-}
-```
+Run:
 
-## Connection options
-
-You can override the parameters passed to `Net::IMAP` with `connection_options`.
-
-Specifically, if you are using a self-signed certificate and get SSL errors, e.g.
-`certificate verify failed`, you can choose to not verify the TLS connection:
-
-```json
-{
-  "accounts": [
-    {
-      "username": "my.user@gmail.com",
-      "password": "secret",
-      "server": "my.imap.example.com",
-      "local_path": "/path/to/backup/root",
-      "folders": [
-        {"name": "[Gmail]/All Mail"},
-        {"name": "my_folder"}
-      ],
-      "connection_options": {
-        "ssl": {"verify_mode": 0},
-        "port": 993
-      }
-    }
-  ]
-}
+```sh
+imap-backup setup
 ```
 
-# Security
-
-Note that email usernames and passwords are held in plain text
-in the configuration file.
-
-The directory ~/.imap-backup, the configuration file and all backup
-directories have their access permissions set to only allow access
-by your user. This is not done on Windows - see below.
-
-## Windows
-
-Due to the complexity of managing permissions on Windows,
-directory and file access permissions are not set explicity.
+## GMail
 
-A pull request that implements permissions management on Windows
-would be welcome!
+To use imap-backup with GMail, you will need to enable 'App passwords' on your account.
 
 # Run Backup
 
 Manually, from the command line:
 
-```shell
-$ imap-backup
+```sh
+imap-backup
 ```
 
 Alternatively, add it to your crontab.
 
-# Result
+Emails are stored on disk in [Mbox files](./docs/files/mboxrd.md) for each folder, with metadata
+stored in [Imap files](./docs/files/imap.md).
 
-Each folder is saved to an mbox file.
-Alongside each mbox is a file with extension '.imap', which lists the source IMAP
-UIDs to allow a full restore.
-
-# Local commands
+# Commands
 
-There a various commands for viewing local backup status.
+* [folders](./commands/folders.md)
+* [restore](./commands/restore.md)
+* [status](./commands/status.md)
 
-To view the list, use
+For a full list of available commands, run
 
-```shell
-$ imap_backup help local
+```sh
+imap-backup help
 ```
 
-# Troubleshooting
-
-If you have problems:
+For more information about a command, run
 
-1. ensure that you have the latest release,
-2. turn on debugging output:
-
-```json
-{
-  "accounts":
-  [
-    ...
-  ],
-  "debug": true
-}
+```sh
+imap-backup help COMMAND
 ```
 
-# Restore
-
-All missing messages are pushed to the IMAP server.
-Existing messages are left unchanged.
+## Configuration
 
-This functionality requires that the IMAP server supports the UIDPLUS
-extension to IMAP4.
+`imap-backup setup` creates the file `~/.imap-backup/config.json`.
 
-# Other Usage
-
-List IMAP folders:
-
-```shell
-$ imap-backup folders
-```
+[More information about configuration is available in the specific documentation](./docs/configuration.md).
 
-Get statistics of emails to download per folder:
-
-```shell
-$ imap-backup status
-```
+# Troubleshooting
 
-# Design Goals
+If you have problems:
 
-* Secure - use a local file protected by permissions
-* Restartable - calculate start point based on already downloaded messages
-* Standalone - do not rely on an email client or MTA
+1. ensure that you have the latest release,
+2. turn on debugging output via the `imap-backup setup` main menu.
 
-# Documentation
+# Development
 
-* [Development](./docs/development.md)
-* [Restore](./docs/restore.md)
+See the [Development documentation](./docs/development.md)

data/docs/configuration.md

@@ -0,0 +1,23 @@
+Configuration is stored in a JSON file.
+
+The format is documented [here](./docs/files/config.json).
+
+# Folders
+
+By default, all folders are backed-up. You can override this by choosing
+specific folders.
+
+# Connection options
+
+You can override the parameters passed to `Net::IMAP` with `connection_options`.
+
+Specifically, if you are using a self-signed certificate and get SSL errors, e.g.
+`certificate verify failed`, you can choose to not verify the TLS connection.
+
+Connection options can be entered via `imap-backup setup` as JSON.
+
+Choose the account, then 'modify connection options'.
+
+For example:
+
+![Entering connection options as JSON](./images/entering-connection-options-as-json.png "Entering connection options as JSON")

data/docs/development.md

@@ -1,3 +1,9 @@
+# Design Goals
+
+* Secure - use a local file protected by permissions
+* Restartable - calculate start point based on already downloaded messages
+* Standalone - do not rely on an email client or MTA
+
 # Testing
 
 ## Feature Specs

data/imap-backup.gemspec

@@ -19,10 +19,12 @@ Gem::Specification.new do |gem|
 
   gem.executables   = gem.files.grep(%r{^bin/}).map { |f| File.basename(f) }
   gem.require_paths = ["lib"]
-  gem.required_ruby_version = ">= 2.5"
+  gem.required_ruby_version = ">= 2.6"
 
   gem.add_runtime_dependency "highline"
   gem.add_runtime_dependency "mail"
+  gem.add_runtime_dependency "net-imap"
+  gem.add_runtime_dependency "net-smtp"
   gem.add_runtime_dependency "os"
   gem.add_runtime_dependency "rake"
   gem.add_runtime_dependency "thor", "~> 1.1"

data/lib/imap/backup/account/connection.rb

@@ -39,7 +39,14 @@ module Imap::Backup
       client
       ensure_account_folder
       each_folder do |folder, serializer|
-        next if !folder.exist?
+        begin
+          next if !folder.exist?
+        rescue Encoding::UndefinedConversionError
+          message = "Skipping backup for '#{folder.name}' " \
+                    "as it is not UTF-7 encoded correctly"
+          Logger.logger.info message
+          next
+        end
 
         Logger.logger.debug "[#{folder.name}] running backup"
         serializer.apply_uid_validity(folder.uid_validity)

data/lib/imap/backup/account/folder.rb

@@ -73,7 +73,7 @@ module Imap::Backup
       examine
       fetch_data_items =
         retry_on_error(errors: UID_FETCH_RETRY_CLASSES) do
-          client.uid_fetch(uids, [BODY_ATTRIBUTE])
+          client.uid_fetch(uids, [BODY_ATTRIBUTE, "FLAGS"])
         end
       return nil if fetch_data_items.nil?
 
@@ -82,18 +82,19 @@ module Imap::Backup
 
         {
           uid: attributes["UID"],
-          body: attributes[BODY_ATTRIBUTE]
+          body: attributes[BODY_ATTRIBUTE],
+          flags: attributes["FLAGS"]
         }
       end
     rescue FolderNotFound
       nil
     end
 
-    def append(message)
+    def append(message, flags: nil)
       body = message.imap_body
       date = message.date&.to_time
       retry_on_error(errors: APPEND_RETRY_CLASSES, limit: 3) do
-        response = client.append(utf7_encoded_name, body, nil, date)
+        response = client.append(utf7_encoded_name, body, flags, date)
         extract_uid(response)
       end
     end

data/lib/imap/backup/cli/utils.rb

@@ -73,7 +73,7 @@ module Imap::Backup
             Skipped #{uid}
           MESSAGE
 
-          serializer.append uid, message
+          serializer.append uid, message, []
         end
       end
 

data/lib/imap/backup/client/default.rb

@@ -7,14 +7,16 @@ module Imap::Backup
   class Client::Default
     extend Forwardable
     def_delegators :imap, *%i(
-      append authenticate create disconnect examine expunge
-      login responses select uid_fetch uid_search uid_store
+      append authenticate create expunge login
+      responses uid_fetch uid_search uid_store
     )
 
     attr_reader :args
+    attr_accessor :state
 
     def initialize(*args)
       @args = args
+      @state = nil
     end
 
     def list
@@ -26,6 +28,28 @@ module Imap::Backup
       mailbox_lists.map { |ml| extract_name(ml) }
     end
 
+    # Track mailbox selection during delegation to Net::IMAP instance
+
+    def disconnect
+      imap.disconnect
+      self.state = nil
+    end
+
+    def examine(mailbox)
+      return if state == [:examine, mailbox]
+
+      result = imap.examine(mailbox)
+      self.state = [:examine, mailbox]
+      result
+    end
+
+    def select(mailbox)
+      return if state == [:select, mailbox]
+
+      imap.select(mailbox)
+      self.state = [:select, mailbox]
+    end
+
     private
 
     def imap

data/lib/imap/backup/configuration.rb

@@ -30,7 +30,7 @@ module Imap::Backup
 
     def save
       ensure_loaded!
-      FileUtils.mkdir(path) if !File.directory?(path)
+      FileUtils.mkdir_p(path) if !File.directory?(path)
       make_private(path) if !windows?
       remove_modified_flags
       remove_deleted_accounts

data/lib/imap/backup/downloader.rb

@@ -79,6 +79,7 @@ module Imap::Backup
     def handle_uid_and_body(uid_and_body, index)
       uid = uid_and_body[:uid]
       body = uid_and_body[:body]
+      flags = uid_and_body[:flags]
       case
       when !body
         info("Fetch returned empty body - skipping")
@@ -86,7 +87,7 @@ module Imap::Backup
         info("Fetch returned empty UID - skipping")
       else
         debug("uid: #{uid} (#{index}/#{uids.count}) - #{body.size} bytes")
-        serializer.append uid, body
+        serializer.append uid, body, flags
       end
     end
 

data/lib/imap/backup/migrator.rb

@@ -16,7 +16,7 @@ module Imap::Backup
       ensure_destination_empty!
 
       Logger.logger.debug "[#{folder.name}] #{count} to migrate"
-      serializer.each_message(serializer.uids).with_index do |(uid, message), i|
+      serializer.each_message(serializer.uids).with_index do |(uid, message, flags), i|
         next if message.nil?
 
         log_prefix = "[#{folder.name}] uid: #{uid} (#{i + 1}/#{count}) -"
@@ -24,7 +24,11 @@ module Imap::Backup
           "#{log_prefix} #{message.supplied_body.size} bytes"
         )
 
-        folder.append(message)
+        begin
+          folder.append(message, flags: flags)
+        rescue StandardError => e
+          Logger.logger.warn "#{log_prefix} append error: #{e}"
+        end
       end
     end
 

data/lib/imap/backup/serializer/appender.rb

@@ -10,7 +10,7 @@ module Imap::Backup
       @mbox = mbox
     end
 
-    def run(uid:, message:)
+    def run(uid:, message:, flags:)
       raise "Can't add messages without uid_validity" if !imap.uid_validity
 
       uid = uid.to_i
@@ -21,19 +21,20 @@ module Imap::Backup
         return
       end
 
-      do_append uid, message
+      do_append uid, message, flags
     end
 
     private
 
-    def do_append(uid, message)
+    def do_append(uid, message, flags)
       mboxrd_message = Email::Mboxrd::Message.new(message)
       initial = mbox.length || 0
       mbox_appended = false
       begin
-        mbox.append mboxrd_message.to_serialized
+        serialized = mboxrd_message.to_serialized
+        mbox.append serialized
         mbox_appended = true
-        imap.append uid
+        imap.append uid, serialized.length, flags
       rescue StandardError => e
         mbox.rewind(initial) if mbox_appended
 

data/lib/imap/backup/serializer/imap.rb

@@ -2,7 +2,7 @@ require "json"
 
 module Imap::Backup
   class Serializer::Imap
-    CURRENT_VERSION = 2
+    CURRENT_VERSION = 3
 
     attr_reader :folder_path
     attr_reader :loaded
@@ -11,7 +11,7 @@ module Imap::Backup
       @folder_path = folder_path
       @loaded = false
       @uid_validity = nil
-      @uids = nil
+      @messages = nil
       @version = nil
     end
 
@@ -23,21 +23,38 @@ module Imap::Backup
       true
     end
 
-    def append(uid)
-      uids << uid
+    def append(uid, length, flags = [])
+      offset =
+        if messages.empty?
+          0
+        else
+          last_message = messages[-1]
+          last_message[:offset] + last_message[:length]
+        end
+      messages << {uid: uid, offset: offset, length: length, flags: flags}
       save
     end
 
+    def get(uid)
+      messages.find { |m| m[:uid] == uid }
+    end
+
     def delete
       return if !exist?
 
       File.unlink(pathname)
+      @loaded = false
+      @messages = nil
+      @uid_validity = nil
+      @version = nil
     end
 
+    # Deprecated
     def include?(uid)
       uids.include?(uid)
     end
 
+    # Deprecated
     def index(uid)
       uids.find_index(uid)
     end
@@ -60,20 +77,26 @@ module Imap::Backup
     def uid_validity=(value)
       ensure_loaded
       @uid_validity = value
-      @uids ||= []
       save
     end
 
-    def uids
+    # Make private
+    def messages
       ensure_loaded
-      @uids || []
+      @messages
+    end
+
+    # Deprecated
+    def uids
+      messages.map { |m| m[:uid] }
     end
 
     def update_uid(old, new)
-      index = uids.find_index(old.to_i)
+      index = messages.find_index { |m| m[:uid] == old }
       return if index.nil?
 
-      uids[index] = new.to_i
+      updated = messages[index].merge({uid: new})
+      messages[index] = updated
       save
     end
 
@@ -97,11 +120,11 @@ module Imap::Backup
 
       data = load
       if data
-        @uids = data[:uids].map(&:to_i)
+        @messages = data[:messages]
         @uid_validity = data[:uid_validity]
         @version = data[:version]
       else
-        @uids = []
+        @messages = []
         @uid_validity = nil
         @version = CURRENT_VERSION
       end
@@ -121,8 +144,10 @@ module Imap::Backup
 
       return nil if !data.key?(:version)
       return nil if !data.key?(:uid_validity)
-      return nil if !data.key?(:uids)
-      return nil if !data[:uids].is_a?(Array)
+      return nil if !data.key?(:messages)
+      return nil if !data[:messages].is_a?(Array)
+
+      data[:messages].each { |m| m[:flags] = m[:flags].map(&:to_sym) }
 
       data
     end
@@ -135,7 +160,7 @@ module Imap::Backup
       data = {
         version: @version,
         uid_validity: @uid_validity,
-        uids: @uids
+        messages: @messages
       }
       content = data.to_json
       File.open(pathname, "w") { |f| f.write content }

data/lib/imap/backup/serializer/mbox.rb

@@ -16,6 +16,13 @@ module Imap::Backup
       end
     end
 
+    def read(offset, length)
+      File.open(pathname, "rb") do |f|
+        f.seek offset
+        f.read length
+      end
+    end
+
     def delete
       return if !exist?
 

data/lib/imap/backup/serializer/mbox_enumerator.rb

@@ -8,8 +8,8 @@ module Imap::Backup
       @mbox_pathname = mbox_pathname
     end
 
-    def each
-      return enum_for(:each) if !block_given?
+    def each(&block)
+      return enum_for(:each) if !block
 
       File.open(mbox_pathname, "rb") do |f|
         lines = []
@@ -26,8 +26,14 @@ module Imap::Backup
           end
         end
 
-        yield lines.join if lines.count.positive?
+        block.call(lines.join) if lines.count.positive?
       end
     end
+
+    def map(&block)
+      return enum_for(:map) if !block
+
+      each.map { |line| block.call(line) }
+    end
   end
 end

data/lib/imap/backup/serializer/message_enumerator.rb

@@ -12,17 +12,16 @@ module Imap::Backup
     end
 
     def run(uids:)
-      indexes = uids.each.with_object({}) do |uid_maybe_string, acc|
+      uids.each do |uid_maybe_string|
         uid = uid_maybe_string.to_i
-        index = imap.index(uid)
-        acc[index] = uid if index
-      end
-      enumerator = Serializer::MboxEnumerator.new(mbox.pathname)
-      enumerator.each.with_index do |raw, i|
-        uid = indexes[i]
-        next if !uid
+        message = imap.get(uid)
+
+        next if !message
+
+        raw = mbox.read(message[:offset], message[:length])
+        body = Email::Mboxrd::Message.from_serialized(raw)
 
-        yield uid, Email::Mboxrd::Message.from_serialized(raw)
+        yield message[:uid], body, message[:flags]
       end
     end
   end

data/lib/imap/backup/serializer/version2_migrator.rb

@@ -0,0 +1,90 @@
+require "json"
+
+require "imap/backup/serializer/mbox_enumerator"
+
+module Imap::Backup
+  class Serializer::Version2Migrator
+    attr_reader :folder_path
+
+    def initialize(folder_path)
+      @folder_path = folder_path
+    end
+
+    def required?
+      return false if !mbox_exists?
+      return false if !imap_exists?
+      return false if !data
+      return false if data[:version] != 2
+      return false if !data[:uid_validity]
+      return false if !uids.is_a?(Array)
+
+      true
+    end
+
+    def run
+      return false if !required?
+
+      messages = message_uids_and_lengths
+
+      return false if !messages
+
+      imap.delete
+      imap.uid_validity = data[:uid_validity]
+      messages.map { |m| imap.append(m[:uid], m[:length]) }
+
+      true
+    end
+
+    private
+
+    def imap_pathname
+      "#{folder_path}.imap"
+    end
+
+    def imap_exists?
+      File.exist?(imap_pathname)
+    end
+
+    def mbox_pathname
+      "#{folder_path}.mbox"
+    end
+
+    def mbox_exists?
+      File.exist?(mbox_pathname)
+    end
+
+    def data
+      @data ||=
+        begin
+          content = File.read(imap_pathname)
+          JSON.parse(content, symbolize_names: true)
+        rescue JSON::ParserError
+          nil
+        end
+    end
+
+    def uids
+      data[:uids]
+    end
+
+    def message_uids_and_lengths
+      enumerator = Serializer::MboxEnumerator.new(mbox_pathname)
+      messages = enumerator.map.with_index do |raw, i|
+        length = raw.length
+        message = {
+          uid: uids[i],
+          length: length
+        }
+        message
+      end
+
+      return nil if messages.count != uids.count
+
+      messages
+    end
+
+    def imap
+      @imap ||= Serializer::Imap.new(folder_path)
+    end
+  end
+end

data/lib/imap/backup/serializer.rb

@@ -6,6 +6,7 @@ require "imap/backup/serializer/imap"
 require "imap/backup/serializer/mbox"
 require "imap/backup/serializer/mbox_enumerator"
 require "imap/backup/serializer/message_enumerator"
+require "imap/backup/serializer/version2_migrator"
 require "imap/backup/serializer/unused_name_finder"
 
 module Imap::Backup
@@ -31,6 +32,8 @@ module Imap::Backup
     # Returns true if there are existing, valid files
     # false otherwise (in which case any existing files are deleted)
     def validate!
+      optionally_migrate2to3
+
       return true if imap.valid? && mbox.valid?
 
       imap.delete
@@ -62,11 +65,11 @@ module Imap::Backup
       internal_force_uid_validity(value)
     end
 
-    def append(uid, message)
+    def append(uid, message, flags)
       validate!
 
       appender = Serializer::Appender.new(folder: folder, imap: imap, mbox: mbox)
-      appender.run(uid: uid, message: message)
+      appender.run(uid: uid, message: message, flags: flags)
     end
 
     def load(uid_maybe_string)
@@ -79,12 +82,6 @@ module Imap::Backup
       internal_load_nth(message_index)
     end
 
-    def load_nth(index)
-      validate!
-
-      internal_load_nth(index)
-    end
-
     def each_message(required_uids, &block)
       validate!
 
@@ -134,6 +131,19 @@ module Imap::Backup
         end
     end
 
+    def optionally_migrate2to3
+      migrator = Version2Migrator.new(folder_path)
+      return if !migrator.required?
+
+      Logger.logger.info <<~MESSAGE
+        Local metadata for folder '#{folder_path}' is currently stored in the version 2 format.
+
+        This will now be transformed into the version 3 format.
+      MESSAGE
+
+      migrator.run
+    end
+
     def folder_path
       self.class.folder_path_for(path: path, folder: folder)
     end

data/lib/imap/backup/setup/account.rb

@@ -17,6 +17,10 @@ module Imap::Backup
     end
 
     def run
+      if !account.local_path
+        account.local_path = File.join(config.path, account.username.tr("@", "_"))
+      end
+
       catch :done do
         loop do
           Kernel.system("clear")

data/lib/imap/backup/setup/asker.rb

@@ -13,7 +13,6 @@ module Imap::Backup
     def email(default = "")
       highline.ask("email address: ") do |q|
         q.default               = default
-        q.readline              = true
         q.validate              = EMAIL_MATCHER
         q.responses[:not_valid] = "Enter a valid email address "
       end

data/lib/imap/backup/setup/backup_path.rb

@@ -12,17 +12,13 @@ module Imap::Backup
 
     def run
       account.local_path = highline.ask("backup directory: ") do |q|
-        q.default  = account.local_path || default
+        q.default  = account.local_path
         q.readline = true
         q.validate = ->(path) { path_modification_validator(path) }
         q.responses[:not_valid] = "Choose a different directory "
       end
     end
 
-    def default
-      File.join(config.path, account.username.tr("@", "_"))
-    end
-
     private
 
     def highline

data/lib/imap/backup/uploader.rb

@@ -19,14 +19,14 @@ module Imap::Backup
       return if count.zero?
 
       Logger.logger.debug "[#{folder.name}] #{count} to restore"
-      serializer.each_message(missing_uids).with_index do |(uid, message), i|
-        upload_message uid, message, i + 1
+      serializer.each_message(missing_uids).with_index do |(uid, message, flags), i|
+        upload_message uid, message, flags, i + 1
       end
     end
 
     private
 
-    def upload_message(uid, message, index)
+    def upload_message(uid, message, flags, index)
       return if message.nil?
 
       log_prefix = "[#{folder.name}] uid: #{uid} (#{index}/#{count}) -"
@@ -35,7 +35,7 @@ module Imap::Backup
       )
 
       begin
-        new_uid = folder.append(message)
+        new_uid = folder.append(message, flags: flags)
         serializer.update_uid(uid, new_uid)
       rescue StandardError => e
         Logger.logger.warn "#{log_prefix} append error: #{e}"

data/lib/imap/backup/version.rb

@@ -1,9 +1,9 @@
 module Imap; end
 
 module Imap::Backup
-  MAJOR    = 6
-  MINOR    = 1
+  MAJOR    = 7
+  MINOR    = 0
   REVISION = 0
-  PRE      = nil
+  PRE      = "rc1".freeze
   VERSION  = [MAJOR, MINOR, REVISION, PRE].compact.map(&:to_s).join(".")
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: imap-backup
 version: !ruby/object:Gem::Version
-  version: 6.1.0
+  version: 7.0.0.rc1
 platform: ruby
 authors:
 - Joe Yates
-autorequire:
+autorequire: 
 bindir: bin
 cert_chain: []
-date: 2022-07-11 00:00:00.000000000 Z
+date: 2022-08-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: highline
@@ -38,6 +38,34 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '0'
+- !ruby/object:Gem::Dependency
+  name: net-imap
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: net-smtp
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 - !ruby/object:Gem::Dependency
   name: os
   requirement: !ruby/object:Gem::Requirement
@@ -189,6 +217,7 @@ files:
 - LICENSE
 - README.md
 - bin/imap-backup
+- docs/configuration.md
 - docs/development.md
 - docs/restore.md
 - docs/setup.md
@@ -237,6 +266,7 @@ files:
 - lib/imap/backup/serializer/mbox_enumerator.rb
 - lib/imap/backup/serializer/message_enumerator.rb
 - lib/imap/backup/serializer/unused_name_finder.rb
+- lib/imap/backup/serializer/version2_migrator.rb
 - lib/imap/backup/setup.rb
 - lib/imap/backup/setup/account.rb
 - lib/imap/backup/setup/account/header.rb
@@ -256,7 +286,7 @@ licenses:
 - MIT
 metadata:
   rubygems_mfa_required: 'true'
-post_install_message:
+post_install_message: 
 rdoc_options: []
 require_paths:
 - lib
@@ -264,15 +294,15 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '2.5'
+      version: '2.6'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - ">"
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
 rubygems_version: 3.1.6
-signing_key:
+signing_key: 
 specification_version: 4
 summary: Backup GMail (or other IMAP) accounts to disk
 test_files: []