imap-backup 14.4.4 → 14.4.5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fc28eee3a2ef71157ebe6e50490488ce92b101bb3c4e02a46964b398ddc85d83
-  data.tar.gz: e6ee9d519463a25bf391fb71bf426d59aeaf27072afdf6306bbe631703a82dfb
+  metadata.gz: e492fd50201bbf2424231905b4cdb6fe3d44d99fa446c624b851c9ef50eb686f
+  data.tar.gz: e2ed159ec5e92e7d97a685093456491afe94d34e70702d4e7a0449cf150009dd
 SHA512:
-  metadata.gz: 0004c4453debba185805fdce233f450570b6bf5a11ae6976641abf91b3adbbc33f62c8da195bcc3eccf99119203cf270bd1506c6620564a81f0c64197814c738
-  data.tar.gz: 378acbd1e668f87b76ebd2db0649344e2904f8992ee4c60aff2f5cf539a145b411fc39bdca962fc4c60e81ed444919b84bf3c305a0199b09fbbd1eb1c87d5d72
+  metadata.gz: d0f4acc27fa79b789c83298af7788c01e9b8c837aa5086d1675e98b7ef80abd7fe1b3decb4b36c3f586126f7d9dc54499358bc05f7e65e0de3cbc29207233188
+  data.tar.gz: e036ad78bff71f98012907444f5eb36c56e01f3a3a8d2f37fbb0b06f5c1850eaddd26a4be86a8581a8f25e322731eac06a25a18853ac4433a2793cbd603b8727

data/README.md

@@ -1,201 +1,131 @@
+# imap-backup API Documentation
+
 ![Version](https://img.shields.io/gem/v/imap-backup?label=Version&logo=rubygems)
 [![Build Status](https://github.com/joeyates/imap-backup/actions/workflows/main.yml/badge.svg)][CI Status]
 ![Coverage](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/joeyates/b54fe758bfb405c04bef72dad293d707/raw/coverage.json)
 ![License](https://img.shields.io/github/license/joeyates/imap-backup?color=brightgreen&label=License)
 [![Stars](https://img.shields.io/github/stars/joeyates/imap-backup?style=social)][GitHub Stars]
+![Activity](https://img.shields.io/github/last-commit/joeyates/imap-backup/main)
 
-# imap-backup
-
-Backup, restore and migrate email accounts.
-
-# Quick Start
-
-```sh
-brew install imap-backup # for macOS
-gem install imap-backup --no-document # for Linux
-imap-backup setup
-imap-backup
-```
-
-# Modes
-
-There are two types of backups:
-
-* Keep all (the default) - progressively saves a local copy of all emails,
-* Mirror - adds and deletes emails from the local copy to keep it up to date with the account.
+[CI Status]: https://github.com/joeyates/imap-backup/actions/workflows/main.yml
+[GitHub Stars]: https://github.com/joeyates/imap-backup/stargazers "GitHub Stars"
 
-# What You Can Do with a Backup
+This is the developer documentation for imap-backup's **code**.
 
-* Migrate - use the local copy to populate emails on another account. This is a once-only action that deletes any existing emails on the destination account.
-* Mirror - make a destination account match the local copy. This action can be repeated.
-* Restore - push the local copy back to the original account.
+Usage documentation is on [GitHub](https://github.com/joeyates/imap-backup).
 
-See below for a [full list of commands](#commands).
+You can get an overview of the program's structure from the
+{file:ARCHITECTURE.md ARCHITECTURE} file.
 
-# Installation
+The {file:CHANGELOG.md CHANGELOG} has a history of the changes to the program.
 
-## Docker or Podman
+# Design Goals
 
-If you have Docker or Podman installed, the easist way to use imap-backup
-is via the container image.
+* Secure - use a local configuration file protected by permissions
+* Restartable - calculate start point based on already downloaded messages
+* Standalone - do not rely on an email client or MTA
 
-You'll need to choose a path on your computer where your backups will be saved,
-we'll use `./my-data` here.
+# Repository
 
-If you have just one account, you can do as follows
+After cloning the repo, run the following command to get
+better `git blame` output:
 
 ```sh
-docker run -v ./my-data:/data -ti ghcr.io/joeyates/imap-backup:latest \
-  imap-backup single backup \
-  --email me@example.com --password mysecret --server imap.example.com \
-  --path /data/me_example.com
+git config --local blame.ignoreRevsFile .git-blame-ignore-revs
 ```
 
-Podman will work exactly the same.
+# Testing
 
-If you have multiple accounts, you can create a configuration file.
+## Feature Specs
 
-You'll need to choose a path on your computer where your configuration will be saved,
-we'll use `./my-config` here.
+Specs under `specs/features` are integration specs.
+Some of these specs run against two local IMAP servers
+controlled by Podman (or Docker) Compose.
 
-First, run the menu-driven setup program to configure your accounts
+Start them before running the test suite
 
 ```sh
-docker run -ti -v ./my-config:/config -v ./my-data:/data -ti ghcr.io/joeyates/imap-backup:latest \
-  imap-backup setup -c /config/imap-backup.json
+$ podman-compose -f dev/compose.yml up -d
 ```
 
-Then, run the backup
+or, with Docker
 
 ```sh
-docker run -v ./my-config:/config -v ./my-data:/data -ti ghcr.io/joeyates/imap-backup:latest \
-  imap-backup backup -c /config/imap-backup.json
+$ docker-compose -f dev/compose.yml up -d
 ```
 
-
-## Homebrew (macOS)
-
-![Homebrew installs](https://img.shields.io/homebrew/installs/dm/imap-backup?label=Homebrew%20installs)
-
-If you have [Homebrew](https://brew.sh/), do this:
+Then, run all specs
 
 ```sh
-brew install imap-backup
+$ rspec
 ```
 
-## As a Ruby Gem
+To exclude container-based tests
 
 ```sh
-gem install imap-backup --no-document
+$ rspec --tag ~docker
 ```
 
-If that doesn't work, see the [detailed installation instructions](docs/installation/rubygem.md).
-
-## From Source Code
-
-If you want to use imap-backup directly from the source code, see [here](docs/installation/source.md).
-
-# Setup
-
-Normally you will want to backup a number of email accounts.
-Doing so requires the creation of a config file.
-
-You do this via a menu-driven command line program:
-
-Run:
+To run **just** the feature specs
 
 ```sh
-imap-backup setup
+rspec spec/features/**/*_spec.rb
 ```
 
-As an alternative, if you only want to backup a single account,
-you can pass all the necessary parameters directly to the `single backup` command
-(see the [`single backup`](docs/commands/single-backup.md) docs).
-
-## GMail
-
-To use imap-backup with GMail, Office 365 and other services that require
-OAuth2 authentication, you can use [email-oauth2-proxy](https://github.com/simonrob/email-oauth2-proxy).
-See [this blog post about using imap-backup with email-oauth2-proxy](https://joeyates.info/posts/back-up-gmail-accounts-with-imap-backup-using-email-oauth2-proxy/).
+## Full Test Run
 
-# Backup
-
-Manually, from the command line:
+The full test run includes RSpec specs **and** Rubocop checks
 
 ```sh
-imap-backup
+rake
 ```
 
-Alternatively, add it to your crontab.
+# Test Debugging
 
-Backups can also be inspected, for example via [`local show`](docs/commands/local-show.md)
-and exported via [`utils export-to-thunderbird`](docs/commands/utils-export-to-thunderbird.md).
+The feature specs are run 'out of process' via the Aruba gem.
+In order to see debugging output from the process,
+use `last_command_started.output`.
 
-# Commands
+# Older Rubies
 
-* [`backup`](docs/commands/backup.md)
-* [`local accounts`](docs/commands/local-accounts.md)
-* [`local check`](docs/commands/local-check.md)
-* [`local folders`](docs/commands/local-folders.md)
-* [`local list`](docs/commands/local-list.md)
-* [`local show`](docs/commands/local-show.md)
-* [`migrate`](docs/commands/migrate.md)
-* [`mirror`](docs/commands/mirror.md)
-* [`remote folders`](docs/commands/remote-folders.md)
-* [`restore`](docs/commands/restore.md)
-* [`setup`](docs/commands/setup.md)
-* [`single backup`](docs/commands/single-backup.md)
-* [`utils export-to-thunderbird`](docs/commands/utils-export-to-thunderbird.md)
-* [`utils ignore-history`](docs/commands/utils-ignore-history.md)
+A Containerfile is available to allow testing with all available Ruby versions,
+see the README in the `dev` directory.
 
-For a full list of available commands, run
+# Performance Specs
 
 ```sh
-imap-backup help
+PERFORMANCE=1 rspec --order=defined
 ```
 
-For more information about a command, run
-
-```sh
-imap-backup help COMMAND
-```
+Beware: the performance spec (just backup for now) takes a very
+long time to run, approximately 24 hours!
 
-# Performance
+# Access Docker imap server
 
-There are a couple of performance tweaks that you can use
-to improve backup speed.
+```ruby
+require "net/imap"
+require_relative "spec/features/support/30_email_server_helpers"
 
-These are activated via two settings:
+include EmailServerHelpers
 
-* Global setting "Delay download writes",
-* Account setting "Multi-fetch size".
+test_connection = test_server_connection_parameters
 
-See [the performance document](docs/performance.md) for more information.
+test_imap = Net::IMAP.new(test_connection[:server], test_connection[:connection_options])
+test_imap.login(test_connection[:username], test_connection[:password])
 
-# Troubleshooting
+message = "From: #{test_connection[:username]}\nSubject: Some Subject\n\nHello!\n"
+response = test_imap.append("INBOX", message, nil, nil)
 
-If you have problems:
+test_imap.examine("INBOX")
+uids = test_imap.uid_search(["ALL"]).sort
 
-1. ensure that you have the latest release,
-2. run `imap-backup` with the `-v` or `--verbose` parameter.
-
-# Development
-
-![Activity](https://img.shields.io/github/last-commit/joeyates/imap-backup/main)
-
-See the [Development documentation](./docs/development.md) for notes
-on development and testing.
-
-See [the CHANGELOG](./CHANGELOG.md) for a list of changes that have been
-made in each release.
+fetch_data_items = test_imap.uid_fetch(uids, ["BODY[]"])
+```
 
-* [Source Code]
-* [Code Documentation]
-* [Rubygem]
-* [CI Status]
+# Contributing
 
-[Source Code]: https://github.com/joeyates/imap-backup "Source code at GitHub"
-[GitHub Stars]: https://github.com/joeyates/imap-backup/stargazers "GitHub Stars"
-[Code Documentation]: https://rubydoc.info/gems/imap-backup/frames "Code Documentation at Rubydoc.info"
-[Rubygem]: https://rubygems.org/gems/imap-backup "Ruby gem at rubygems.org"
-[CI Status]: https://github.com/joeyates/imap-backup/actions/workflows/main.yml
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Added some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/docs/documentation.md

@@ -0,0 +1,19 @@
+# Documentation
+
+This project uses [yard](https://yardoc.org/) to generate its documentation.
+
+# Development
+
+Run an autoreloading document viewer
+
+```
+yard server --reload
+```
+
+# Coverage
+
+See what's not documented
+
+```
+yard stats --list-undoc
+```

data/lib/imap/backup/account/backup.rb

@@ -15,6 +15,8 @@ module Imap::Backup
       @refresh = refresh
     end
 
+    # Runs the backup
+    # @return [void]
     def run
       Logger.logger.info "Running backup of account: #{account.username}"
       # start the connection so we get logging messages in the right order

data/lib/imap/backup/account/backup_folders.rb

@@ -5,7 +5,7 @@ module Imap; end
 module Imap::Backup
   class Account; end
 
-  # Iterates over the account folders that are to be backed up
+  # Enumerates over the account folders that are to be backed up
   class Account::BackupFolders
     include Enumerable
 
@@ -14,6 +14,9 @@ module Imap::Backup
       @account = account
     end
 
+    # Runs the enumeration
+    # @yieldparam folder [Account::Folder] the online folder
+    # @return [void]
     def each(&block)
       return enum_for(:each) if !block
 
@@ -39,6 +42,9 @@ module Imap::Backup
       names.each { |name| block.call(Account::Folder.new(client, name)) }
     end
 
+    # Runs a map operation over the folders
+    # @yieldparam folder [Account::Folder] the online folder
+    # @return The results of the map operation
     def map(&block)
       each.map do |folder|
         block.call(folder)

data/lib/imap/backup/account/client_factory.rb

@@ -18,6 +18,7 @@ module Imap::Backup
       @server = nil
     end
 
+    # @return [Client::AutomaticLoginWrapper] a client for the account
     def run
       options = provider_options
       Logger.logger.debug(

data/lib/imap/backup/account/folder.rb

@@ -12,12 +12,15 @@ module Imap::Backup
 
   # Handles access to a folder on an IMAP server
   class Account::Folder
+    # An error that is thrown if a requestd folder does not exist
     class FolderNotFound < StandardError; end
 
     extend Forwardable
     include RetryOnError
 
+    # @return [Client::Default]
     attr_reader :client
+    # @return [String] the name of the folder
     attr_reader :name
 
     def initialize(client, name)
@@ -26,6 +29,7 @@ module Imap::Backup
       @uid_validity = nil
     end
 
+    # @raise any error that occurs more than 10 times
     def exist?
       previous_level = Imap::Backup::Logger.logger.level
       previous_debug = Net::IMAP.debug
@@ -42,6 +46,8 @@ module Imap::Backup
       Net::IMAP.debug = previous_debug
     end
 
+    # Creates the folder on the server
+    # @return [void]
     def create
       return if exist?
 
@@ -50,6 +56,8 @@ module Imap::Backup
       end
     end
 
+    # @raise any error that occurs more than 10 times
+    # @return [Integer] the folder's UID validity
     def uid_validity
       @uid_validity ||=
         begin
@@ -58,6 +66,8 @@ module Imap::Backup
         end
     end
 
+    # @raise any error that occurs more than 10 times
+    # @return [Array<Integer>] the folders message UIDs
     def uids
       examine
       client.uid_search(["ALL"]).sort
@@ -75,6 +85,8 @@ module Imap::Backup
       []
     end
 
+    # @raise any error that occurs more than 10 times
+    # @return [Array<Hash>, nil] the requested messages
     def fetch_multi(uids, attr = [BODY_ATTRIBUTE, "FLAGS"])
       examine
       fetch_data_items =
@@ -96,6 +108,8 @@ module Imap::Backup
       nil
     end
 
+    # Uploads a message
+    # @return [void]
     def append(message)
       body = message.imap_body
       date = message.date&.to_time
@@ -106,17 +120,23 @@ module Imap::Backup
       end
     end
 
+    # Deletes multiple messages
+    # @return [void]
     def delete_multi(uids)
       add_flags(uids, [:Deleted])
       client.expunge
     end
 
+    # Sets one or more flags on a group of messages
+    # @return [void]
     def set_flags(uids, flags)
       client.select(utf7_encoded_name)
       flags.reject! { |f| f == :Recent }
       client.uid_store(uids, "FLAGS", flags)
     end
 
+    # Adds one or more flags to a group of messages
+    # @return [void]
     def add_flags(uids, flags)
       # Use read-write access, via `select`
       client.select(utf7_encoded_name)
@@ -124,11 +144,15 @@ module Imap::Backup
       client.uid_store(uids, "+FLAGS", flags)
     end
 
+    # Removes one or more flags from a group of messages
+    # @return [void]
     def remove_flags(uids, flags)
       client.select(utf7_encoded_name)
       client.uid_store(uids, "-FLAGS", flags)
     end
 
+    # Deletes all messages from the folder
+    # @return [void]
     def clear
       existing = uids
       return if existing.empty?
@@ -137,6 +161,8 @@ module Imap::Backup
       client.expunge
     end
 
+    # @raise any error that occurs more than 10 times
+    # @return [Array<Integer>] the UIDs of messages with the 'UNSEEN' flag
     def unseen(uids)
       messages = uids.map(&:to_s).join(",")
       examine

data/lib/imap/backup/account/folder_backup.rb

@@ -10,7 +10,7 @@ module Imap; end
 module Imap::Backup
   class Account; end
 
-  # Runs a backup for a single account folder
+  # Implements backup for a single folder
   class Account::FolderBackup
     def initialize(account:, folder:, refresh: false)
       @account = account
@@ -18,6 +18,9 @@ module Imap::Backup
       @refresh = refresh
     end
 
+    # Runs the backup
+    # @raise [RuntimeError] if the configured download strategy is incorrect
+    # @return [void]
     def run
       folder_ok = folder_ok?
       return if !folder_ok

data/lib/imap/backup/account/folder_ensurer.rb

@@ -12,6 +12,9 @@ module Imap::Backup
       @account = account
     end
 
+    # Creates the account's base directory and sets its permissions
+    # @raise [RuntimeError] is the account's backup path is not set
+    # @return [void]
     def run
       raise "The backup path for #{account.username} is not set" if !account.local_path
 

data/lib/imap/backup/account/local_only_folder_deleter.rb

@@ -14,6 +14,8 @@ module Imap::Backup
       @account = account
     end
 
+    # Runs the deletion operation
+    # @return [void]
     def run
       backup_folders = Account::BackupFolders.new(
         client: account.client, account: account

data/lib/imap/backup/account/restore.rb

@@ -12,6 +12,8 @@ module Imap::Backup
       @account = account
     end
 
+    # Runs the restore operation
+    # @return [void]
     def run
       serialized_folders = Account::SerializedFolders.new(account: account)
       serialized_folders.each do |serializer, folder|

data/lib/imap/backup/account/serialized_folders.rb

@@ -9,7 +9,7 @@ module Imap; end
 module Imap::Backup
   class Account; end
 
-  # Enumerates over the folders that are backed up to an account
+  # Enumerates over an account's backed-up folders
   class Account::SerializedFolders
     include Enumerable
 
@@ -17,6 +17,10 @@ module Imap::Backup
       @account = account
     end
 
+    # Runs the enumeration
+    # @yieldparam serializer [Serializer] the folder's serializer
+    # @yieldparam folder [Account::Folder] the online folder
+    # @return [void]
     def each(&block)
       return enum_for(:each) if !block
 

data/lib/imap/backup/account.rb

@@ -14,10 +14,9 @@ module Imap::Backup
     # The username of the account (usually the same as the email address)
     # @return [String]
     attr_reader :username
-    # The password of the Account
+    # @return [String] password of the Account
     attr_reader :password
-    # The path where backups will be saved
-    # @return [String]
+    # @return [String] the path where backups will be saved
     attr_reader :local_path
     # @overload folders
     #   The list of folders that have been configured for the Account
@@ -38,6 +37,7 @@ module Imap::Backup
     attr_reader :folder_blacklist
     # Should all emails be backed up progressively, or should emails
     # which are deleted from the server be deleted locally?
+    # @return [Boolean]
     attr_reader :mirror_mode
     # The address of the IMAP server
     # @return [String]
@@ -61,8 +61,6 @@ module Imap::Backup
     # mark messages as '\Seen' when accessed).
     # @return [Boolean]
     attr_reader :reset_seen_flags_after_fetch
-    # Tracks changes to the Account's attributes
-    attr_reader :changes
 
     def initialize(options)
       @username = options[:username]
@@ -123,6 +121,7 @@ module Imap::Backup
     end
 
     # Resets the store of changes, indicating that the current state is the saved state
+    # @return [void]
     def clear_changes
       @changes = {}
     end
@@ -134,16 +133,12 @@ module Imap::Backup
       @marked_for_deletion = true
     end
 
-    # Indicates whether the account has been flagged for deletion during setup
-    #
-    # @return [Boolean]
+    # @return [Boolean] whether the account has been flagged for deletion during setup
     def marked_for_deletion?
       @marked_for_deletion
     end
 
-    # Returns all Account data for serialization
-    #
-    # @return [Hash]
+    # @return [Hash] all Account data for serialization
     def to_h
       h = {username: @username, password: @password}
       h[:local_path] = @local_path if @local_path
@@ -180,24 +175,30 @@ module Imap::Backup
       update(:local_path, value)
     end
 
+    # @raise [RuntimeError] if the supplied value is not an Array
+    # @return [void]
     def folders=(value)
       raise "folders must be an Array" if !value.is_a?(Array)
 
       update(:folders, value)
     end
 
+    # @return [void]
     def folder_blacklist=(value)
       update(:folder_blacklist, value)
     end
 
+    # @return [void]
     def mirror_mode=(value)
       update(:mirror_mode, value)
     end
 
+    # @return [void]
     def server=(value)
       update(:server, value)
     end
 
+    # @return [void]
     def connection_options=(value)
       parsed =
         if value == ""
@@ -232,12 +233,15 @@ module Imap::Backup
       update(:multi_fetch_size, parsed)
     end
 
+    # @return [void]
     def reset_seen_flags_after_fetch=(value)
       update(:reset_seen_flags_after_fetch, value)
     end
 
     private
 
+    attr_reader :changes
+
     def update(field, value)
       key = :"@#{field}"
       if changes[field]

data/lib/imap/backup/cli/backup.rb

@@ -10,6 +10,7 @@ module Imap; end
 module Imap::Backup
   class CLI < Thor; end
 
+  # Runs backups of configured accounts
   class CLI::Backup < Thor
     include Thor::Actions
     include CLI::Helpers
@@ -19,6 +20,8 @@ module Imap::Backup
       @options = options
     end
 
+    # @!method run
+    #   @return [void]
     no_commands do
       def run
         config = load_config(**options)

data/lib/imap/backup/cli/folder_enumerator.rb

@@ -9,6 +9,7 @@ module Imap; end
 module Imap::Backup
   class CLI < Thor; end
 
+  # Implements a folder enumerator for backed-up accounts
   class CLI::FolderEnumerator
     def initialize(
       destination:,
@@ -26,6 +27,11 @@ module Imap::Backup
       @source_prefix = source_prefix
     end
 
+    # Enumerates backed-up folders
+    # When called without a block, returns an Enumerator
+    # @yieldparam serializer [Serializer] the folder's serializer
+    # @yieldparam folder [Account::Folder] the online folder
+    # @return [Enumerator, void]
     def each
       return enum_for(:each) if !block_given?