imap-backup 14.4.4 → 14.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fc28eee3a2ef71157ebe6e50490488ce92b101bb3c4e02a46964b398ddc85d83
-  data.tar.gz: e6ee9d519463a25bf391fb71bf426d59aeaf27072afdf6306bbe631703a82dfb
+  metadata.gz: ee294a8f4d7c0af90c725939b3f4cd7826ac521194460af3446e7cf5378629f4
+  data.tar.gz: c67cac7ddfb2448a9cb22b642b53be02b157959d51f3677bc37ec891da2471c8
 SHA512:
-  metadata.gz: 0004c4453debba185805fdce233f450570b6bf5a11ae6976641abf91b3adbbc33f62c8da195bcc3eccf99119203cf270bd1506c6620564a81f0c64197814c738
-  data.tar.gz: 378acbd1e668f87b76ebd2db0649344e2904f8992ee4c60aff2f5cf539a145b411fc39bdca962fc4c60e81ed444919b84bf3c305a0199b09fbbd1eb1c87d5d72
+  metadata.gz: e1b2f9affe114ea775b4f7f7d4d2db4349e03b3ccb842885b3423c28c2b84ff6e8872b224a520654e0cef8eb353e9f0ea268e9d0dcda2666b07709f5071a0db7
+  data.tar.gz: 2feab585634685c45606d6e4668e52190af3fd193cf3c8468ea5b306a8834257c2944ca4c668ee5e1f9423ebfb3cf605de9d6c8052a983d57f3fe0df07d105c8

data/README.md

@@ -1,201 +1,131 @@
+# imap-backup API Documentation
+
 ![Version](https://img.shields.io/gem/v/imap-backup?label=Version&logo=rubygems)
 [![Build Status](https://github.com/joeyates/imap-backup/actions/workflows/main.yml/badge.svg)][CI Status]
 ![Coverage](https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/joeyates/b54fe758bfb405c04bef72dad293d707/raw/coverage.json)
 ![License](https://img.shields.io/github/license/joeyates/imap-backup?color=brightgreen&label=License)
 [![Stars](https://img.shields.io/github/stars/joeyates/imap-backup?style=social)][GitHub Stars]
+![Activity](https://img.shields.io/github/last-commit/joeyates/imap-backup/main)
 
-# imap-backup
-
-Backup, restore and migrate email accounts.
-
-# Quick Start
-
-```sh
-brew install imap-backup # for macOS
-gem install imap-backup --no-document # for Linux
-imap-backup setup
-imap-backup
-```
-
-# Modes
-
-There are two types of backups:
-
-* Keep all (the default) - progressively saves a local copy of all emails,
-* Mirror - adds and deletes emails from the local copy to keep it up to date with the account.
+[CI Status]: https://github.com/joeyates/imap-backup/actions/workflows/main.yml
+[GitHub Stars]: https://github.com/joeyates/imap-backup/stargazers "GitHub Stars"
 
-# What You Can Do with a Backup
+This is the developer documentation for imap-backup's **code**.
 
-* Migrate - use the local copy to populate emails on another account. This is a once-only action that deletes any existing emails on the destination account.
-* Mirror - make a destination account match the local copy. This action can be repeated.
-* Restore - push the local copy back to the original account.
+Usage documentation is on [GitHub](https://github.com/joeyates/imap-backup).
 
-See below for a [full list of commands](#commands).
+You can get an overview of the program's structure from the
+{file:ARCHITECTURE.md ARCHITECTURE} file.
 
-# Installation
+The {file:CHANGELOG.md CHANGELOG} has a history of the changes to the program.
 
-## Docker or Podman
+# Design Goals
 
-If you have Docker or Podman installed, the easist way to use imap-backup
-is via the container image.
+* Secure - use a local configuration file protected by permissions
+* Restartable - calculate start point based on already downloaded messages
+* Standalone - do not rely on an email client or MTA
 
-You'll need to choose a path on your computer where your backups will be saved,
-we'll use `./my-data` here.
+# Repository
 
-If you have just one account, you can do as follows
+After cloning the repo, run the following command to get
+better `git blame` output:
 
 ```sh
-docker run -v ./my-data:/data -ti ghcr.io/joeyates/imap-backup:latest \
-  imap-backup single backup \
-  --email me@example.com --password mysecret --server imap.example.com \
-  --path /data/me_example.com
+git config --local blame.ignoreRevsFile .git-blame-ignore-revs
 ```
 
-Podman will work exactly the same.
+# Testing
 
-If you have multiple accounts, you can create a configuration file.
+## Feature Specs
 
-You'll need to choose a path on your computer where your configuration will be saved,
-we'll use `./my-config` here.
+Specs under `specs/features` are integration specs.
+Some of these specs run against two local IMAP servers
+controlled by Podman (or Docker) Compose.
 
-First, run the menu-driven setup program to configure your accounts
+Start them before running the test suite
 
 ```sh
-docker run -ti -v ./my-config:/config -v ./my-data:/data -ti ghcr.io/joeyates/imap-backup:latest \
-  imap-backup setup -c /config/imap-backup.json
+$ podman-compose -f dev/compose.yml up -d
 ```
 
-Then, run the backup
+or, with Docker
 
 ```sh
-docker run -v ./my-config:/config -v ./my-data:/data -ti ghcr.io/joeyates/imap-backup:latest \
-  imap-backup backup -c /config/imap-backup.json
+$ docker-compose -f dev/compose.yml up -d
 ```
 
-
-## Homebrew (macOS)
-
-![Homebrew installs](https://img.shields.io/homebrew/installs/dm/imap-backup?label=Homebrew%20installs)
-
-If you have [Homebrew](https://brew.sh/), do this:
+Then, run all specs
 
 ```sh
-brew install imap-backup
+$ rspec
 ```
 
-## As a Ruby Gem
+To exclude container-based tests
 
 ```sh
-gem install imap-backup --no-document
+$ rspec --tag ~container
 ```
 
-If that doesn't work, see the [detailed installation instructions](docs/installation/rubygem.md).
-
-## From Source Code
-
-If you want to use imap-backup directly from the source code, see [here](docs/installation/source.md).
-
-# Setup
-
-Normally you will want to backup a number of email accounts.
-Doing so requires the creation of a config file.
-
-You do this via a menu-driven command line program:
-
-Run:
+To run **just** the feature specs
 
 ```sh
-imap-backup setup
+rspec spec/features/**/*_spec.rb
 ```
 
-As an alternative, if you only want to backup a single account,
-you can pass all the necessary parameters directly to the `single backup` command
-(see the [`single backup`](docs/commands/single-backup.md) docs).
-
-## GMail
-
-To use imap-backup with GMail, Office 365 and other services that require
-OAuth2 authentication, you can use [email-oauth2-proxy](https://github.com/simonrob/email-oauth2-proxy).
-See [this blog post about using imap-backup with email-oauth2-proxy](https://joeyates.info/posts/back-up-gmail-accounts-with-imap-backup-using-email-oauth2-proxy/).
+## Full Test Run
 
-# Backup
-
-Manually, from the command line:
+The full test run includes RSpec specs **and** Rubocop checks
 
 ```sh
-imap-backup
+rake
 ```
 
-Alternatively, add it to your crontab.
+# Test Debugging
 
-Backups can also be inspected, for example via [`local show`](docs/commands/local-show.md)
-and exported via [`utils export-to-thunderbird`](docs/commands/utils-export-to-thunderbird.md).
+The feature specs are run 'out of process' via the Aruba gem.
+In order to see debugging output from the process,
+use `last_command_started.output`.
 
-# Commands
+# Older Rubies
 
-* [`backup`](docs/commands/backup.md)
-* [`local accounts`](docs/commands/local-accounts.md)
-* [`local check`](docs/commands/local-check.md)
-* [`local folders`](docs/commands/local-folders.md)
-* [`local list`](docs/commands/local-list.md)
-* [`local show`](docs/commands/local-show.md)
-* [`migrate`](docs/commands/migrate.md)
-* [`mirror`](docs/commands/mirror.md)
-* [`remote folders`](docs/commands/remote-folders.md)
-* [`restore`](docs/commands/restore.md)
-* [`setup`](docs/commands/setup.md)
-* [`single backup`](docs/commands/single-backup.md)
-* [`utils export-to-thunderbird`](docs/commands/utils-export-to-thunderbird.md)
-* [`utils ignore-history`](docs/commands/utils-ignore-history.md)
+A Containerfile is available to allow testing with all available Ruby versions,
+see the README in the `dev` directory.
 
-For a full list of available commands, run
+# Performance Specs
 
 ```sh
-imap-backup help
+PERFORMANCE=1 rspec --order=defined
 ```
 
-For more information about a command, run
-
-```sh
-imap-backup help COMMAND
-```
+Beware: the performance spec (just backup for now) takes a very
+long time to run, approximately 24 hours!
 
-# Performance
+# Access Docker imap server
 
-There are a couple of performance tweaks that you can use
-to improve backup speed.
+```ruby
+require "net/imap"
+require_relative "spec/features/support/30_email_server_helpers"
 
-These are activated via two settings:
+include EmailServerHelpers
 
-* Global setting "Delay download writes",
-* Account setting "Multi-fetch size".
+test_connection = test_server_connection_parameters
 
-See [the performance document](docs/performance.md) for more information.
+test_imap = Net::IMAP.new(test_connection[:server], test_connection[:connection_options])
+test_imap.login(test_connection[:username], test_connection[:password])
 
-# Troubleshooting
+message = "From: #{test_connection[:username]}\nSubject: Some Subject\n\nHello!\n"
+response = test_imap.append("INBOX", message, nil, nil)
 
-If you have problems:
+test_imap.examine("INBOX")
+uids = test_imap.uid_search(["ALL"]).sort
 
-1. ensure that you have the latest release,
-2. run `imap-backup` with the `-v` or `--verbose` parameter.
-
-# Development
-
-![Activity](https://img.shields.io/github/last-commit/joeyates/imap-backup/main)
-
-See the [Development documentation](./docs/development.md) for notes
-on development and testing.
-
-See [the CHANGELOG](./CHANGELOG.md) for a list of changes that have been
-made in each release.
+fetch_data_items = test_imap.uid_fetch(uids, ["BODY[]"])
+```
 
-* [Source Code]
-* [Code Documentation]
-* [Rubygem]
-* [CI Status]
+# Contributing
 
-[Source Code]: https://github.com/joeyates/imap-backup "Source code at GitHub"
-[GitHub Stars]: https://github.com/joeyates/imap-backup/stargazers "GitHub Stars"
-[Code Documentation]: https://rubydoc.info/gems/imap-backup/frames "Code Documentation at Rubydoc.info"
-[Rubygem]: https://rubygems.org/gems/imap-backup "Ruby gem at rubygems.org"
-[CI Status]: https://github.com/joeyates/imap-backup/actions/workflows/main.yml
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Added some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/docs/documentation.md

@@ -0,0 +1,19 @@
+# Documentation
+
+This project uses [yard](https://yardoc.org/) to generate its documentation.
+
+# Development
+
+Run an autoreloading document viewer
+
+```
+yard server --reload
+```
+
+# Coverage
+
+See what's not documented
+
+```
+yard stats --list-undoc
+```

data/imap-backup.gemspec

@@ -27,7 +27,7 @@ Gem::Specification.new do |gem|
   gem.add_runtime_dependency "os"
   gem.add_runtime_dependency "rake"
   gem.add_runtime_dependency "thor", "~> 1.1"
-  gem.add_runtime_dependency "thunderbird", ">= 0.0.0"
+  gem.add_runtime_dependency "thunderbird", "0.3.0"
 
   gem.metadata = {
     "rubygems_mfa_required" => "true"

data/lib/imap/backup/account/backup.rb

@@ -15,6 +15,8 @@ module Imap::Backup
       @refresh = refresh
     end
 
+    # Runs the backup
+    # @return [void]
     def run
       Logger.logger.info "Running backup of account: #{account.username}"
       # start the connection so we get logging messages in the right order

data/lib/imap/backup/account/backup_folders.rb

@@ -5,7 +5,7 @@ module Imap; end
 module Imap::Backup
   class Account; end
 
-  # Iterates over the account folders that are to be backed up
+  # Enumerates over the account folders that are to be backed up
   class Account::BackupFolders
     include Enumerable
 
@@ -14,6 +14,9 @@ module Imap::Backup
       @account = account
     end
 
+    # Runs the enumeration
+    # @yieldparam folder [Account::Folder] the online folder
+    # @return [void]
     def each(&block)
       return enum_for(:each) if !block
 
@@ -39,6 +42,9 @@ module Imap::Backup
       names.each { |name| block.call(Account::Folder.new(client, name)) }
     end
 
+    # Runs a map operation over the folders
+    # @yieldparam folder [Account::Folder] the online folder
+    # @return The results of the map operation
     def map(&block)
       each.map do |folder|
         block.call(folder)

data/lib/imap/backup/account/client_factory.rb

@@ -18,6 +18,7 @@ module Imap::Backup
       @server = nil
     end
 
+    # @return [Client::AutomaticLoginWrapper] a client for the account
     def run
       options = provider_options
       Logger.logger.debug(

data/lib/imap/backup/account/folder.rb

@@ -12,12 +12,15 @@ module Imap::Backup
 
   # Handles access to a folder on an IMAP server
   class Account::Folder
+    # An error that is thrown if a requestd folder does not exist
     class FolderNotFound < StandardError; end
 
     extend Forwardable
     include RetryOnError
 
+    # @return [Client::Default]
     attr_reader :client
+    # @return [String] the name of the folder
     attr_reader :name
 
     def initialize(client, name)
@@ -26,6 +29,7 @@ module Imap::Backup
       @uid_validity = nil
     end
 
+    # @raise any error that occurs more than 10 times
     def exist?
       previous_level = Imap::Backup::Logger.logger.level
       previous_debug = Net::IMAP.debug
@@ -42,6 +46,8 @@ module Imap::Backup
       Net::IMAP.debug = previous_debug
     end
 
+    # Creates the folder on the server
+    # @return [void]
     def create
       return if exist?
 
@@ -50,6 +56,8 @@ module Imap::Backup
       end
     end
 
+    # @raise any error that occurs more than 10 times
+    # @return [Integer] the folder's UID validity
     def uid_validity
       @uid_validity ||=
         begin
@@ -58,6 +66,8 @@ module Imap::Backup
         end
     end
 
+    # @raise any error that occurs more than 10 times
+    # @return [Array<Integer>] the folders message UIDs
     def uids
       examine
       client.uid_search(["ALL"]).sort
@@ -75,6 +85,8 @@ module Imap::Backup
       []
     end
 
+    # @raise any error that occurs more than 10 times
+    # @return [Array<Hash>, nil] the requested messages
     def fetch_multi(uids, attr = [BODY_ATTRIBUTE, "FLAGS"])
       examine
       fetch_data_items =
@@ -96,6 +108,8 @@ module Imap::Backup
       nil
     end
 
+    # Uploads a message
+    # @return [void]
     def append(message)
       body = message.imap_body
       date = message.date&.to_time
@@ -106,17 +120,23 @@ module Imap::Backup
       end
     end
 
+    # Deletes multiple messages
+    # @return [void]
     def delete_multi(uids)
       add_flags(uids, [:Deleted])
       client.expunge
     end
 
+    # Sets one or more flags on a group of messages
+    # @return [void]
     def set_flags(uids, flags)
       client.select(utf7_encoded_name)
       flags.reject! { |f| f == :Recent }
       client.uid_store(uids, "FLAGS", flags)
     end
 
+    # Adds one or more flags to a group of messages
+    # @return [void]
     def add_flags(uids, flags)
       # Use read-write access, via `select`
       client.select(utf7_encoded_name)
@@ -124,11 +144,15 @@ module Imap::Backup
       client.uid_store(uids, "+FLAGS", flags)
     end
 
+    # Removes one or more flags from a group of messages
+    # @return [void]
     def remove_flags(uids, flags)
       client.select(utf7_encoded_name)
       client.uid_store(uids, "-FLAGS", flags)
     end
 
+    # Deletes all messages from the folder
+    # @return [void]
     def clear
       existing = uids
       return if existing.empty?
@@ -137,6 +161,8 @@ module Imap::Backup
       client.expunge
     end
 
+    # @raise any error that occurs more than 10 times
+    # @return [Array<Integer>] the UIDs of messages with the 'UNSEEN' flag
     def unseen(uids)
       messages = uids.map(&:to_s).join(",")
       examine

data/lib/imap/backup/account/folder_backup.rb

@@ -10,7 +10,7 @@ module Imap; end
 module Imap::Backup
   class Account; end
 
-  # Runs a backup for a single account folder
+  # Implements backup for a single folder
   class Account::FolderBackup
     def initialize(account:, folder:, refresh: false)
       @account = account
@@ -18,6 +18,9 @@ module Imap::Backup
       @refresh = refresh
     end
 
+    # Runs the backup
+    # @raise [RuntimeError] if the configured download strategy is incorrect
+    # @return [void]
     def run
       folder_ok = folder_ok?
       return if !folder_ok

data/lib/imap/backup/account/folder_ensurer.rb

@@ -12,6 +12,9 @@ module Imap::Backup
       @account = account
     end
 
+    # Creates the account's base directory and sets its permissions
+    # @raise [RuntimeError] is the account's backup path is not set
+    # @return [void]
     def run
       raise "The backup path for #{account.username} is not set" if !account.local_path
 

data/lib/imap/backup/account/local_only_folder_deleter.rb

@@ -14,13 +14,15 @@ module Imap::Backup
       @account = account
     end
 
+    # Runs the deletion operation
+    # @return [void]
     def run
       backup_folders = Account::BackupFolders.new(
         client: account.client, account: account
       )
       wanted = backup_folders.map(&:name)
       serialized_folders = Account::SerializedFolders.new(account: account)
-      serialized_folders.each do |serializer, _folder|
+      serialized_folders.each_key do |serializer|
         serializer.delete if !wanted.include?(serializer.folder)
       end
     end

data/lib/imap/backup/account/restore.rb

@@ -12,6 +12,8 @@ module Imap::Backup
       @account = account
     end
 
+    # Runs the restore operation
+    # @return [void]
     def run
       serialized_folders = Account::SerializedFolders.new(account: account)
       serialized_folders.each do |serializer, folder|

data/lib/imap/backup/account/serialized_folders.rb

@@ -9,7 +9,7 @@ module Imap; end
 module Imap::Backup
   class Account; end
 
-  # Enumerates over the folders that are backed up to an account
+  # Enumerates over an account's backed-up folders
   class Account::SerializedFolders
     include Enumerable
 
@@ -17,6 +17,10 @@ module Imap::Backup
       @account = account
     end
 
+    # Runs the enumeration over local serializers and remote folders
+    # @yieldparam serializer [Serializer] the folder's serializer
+    # @yieldparam folder [Account::Folder] the online folder
+    # @return [void]
     def each(&block)
       return enum_for(:each) if !block
 
@@ -28,6 +32,32 @@ module Imap::Backup
       end
     end
 
+    # Runs the enumeration over each local serializer
+    # @yieldparam serializer [Serializer] the folder's serializer
+    # @return [void]
+    def each_key(&block)
+      return enum_for(:each_key) if !block
+
+      glob.each do |path|
+        name = path.relative_path_from(base).to_s[0..-6]
+        serializer = Serializer.new(account.local_path, name)
+        block.call(serializer)
+      end
+    end
+
+    # Runs the enumeration over each remote folder
+    # @yieldparam folder [Account::Folder] the online folder
+    # @return [void]
+    def each_value(&block)
+      return enum_for(:each_value) if !block
+
+      glob.each do |path|
+        name = path.relative_path_from(base).to_s[0..-6]
+        folder = Account::Folder.new(account.client, name)
+        block.call(folder)
+      end
+    end
+
     private
 
     attr_reader :account