mclone 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 55aeb4153e4c0c1387458da22ac441d0a53dfb7778845f94ccc8e6c48ef699db
+  data.tar.gz: 32d565d1ea60267cd3da77ae03b408a8a42d02a7a6bf3ab6d4ff30ac9e9d3c3c
+SHA512:
+  metadata.gz: 00eef197ed719146957579af92909974433514ebfeff86d6ca4f97cd1a9ff0969fbed7848a468166d1ca20049d014576bc2ddb9027154a91383aed9860c54cc7
+  data.tar.gz: 130f20d592e4c73f320e6e7680f4b100c840f06258eca96c28f7c036e230f305de760f908187b88a8b40cf36dc3834af93724798c7bdb0bfa1b4c017414b517f

data/README.md

@@ -0,0 +1,370 @@
+# Mclone
+
+[Mclone](https://github.com/okhlybov/mclone) is a utility for offline file synchronization utilizing the
+[Rclone](https://rclone.org) as a backend for doing actual file transfer.
+
+## Purpose
+
+Suppose you have a (large amount of) data which needs to be either distributed across many storages or simply backed up.
+For example, consider a terabyte private media archive one can not afford to lose.
+
+As the data gets periodically updated, there is a need for regular synchronization.
+When the use of online cloud storage is not an option due storage space or security reasons, the good ol' offline
+backing up comes back into play.
+
+A sane backup strategy mandates the data copies to be physically separated - be it a next room (building, city or planet)
+computer or just an external drive. Or, even better, the two computers' storages - a primary, where all activity takes place,
+a mirror storage which holds the backup, and a portable storage (USB flash disc, exteral HDD or SSD - whatever) which
+serves as both an intermediate storage and a means of propagating the changes between the primary and the mirror.
+
+In a more complex scenario there may be multiple one-way or two-way point-to-point data transfer routes between the storages,
+employing portable storage as a "shuttle" or a "ferry".
+
+All in all the synchronization task boils down to copying or synchronizing the contents of two local directories. However,
+since portable storage is involved, the actual file paths may change between synchronizations as a storage device can be
+mounted under different mount points on *NIX system or change the disk drive on Windows system.
+
+While the Rclone itself is a great tool for local file synchronization, typing the command line for execution in this case
+becomes tedious and error prone where the possible cost of error is a backup corruption due to wrong paths or misspelled flags.
+
+This is where the Mclone comes in.
+It is designed to automatize the Rclone synchronization process by memorizing the command line options and detecting
+the proper source and destination locations wherever they are.
+
+## Installation
+
+Mclone is written in [Ruby](https://www.ruby-lang.org) language and is distributed in the form of the Ruby [GEM](https://rubygems.org).
+
+Once the Ruby runtime is properly set, the Mclone itself is installed with
+
+```shell
+$ gem install mclone
+```
+
+Obviously, the Rclone installation is also required.
+The Mclone will use either the contents of the `RCLONE` environment variable if exists or look though
+the `PATH` environment variable to locate the `rclone` executable.
+
+Once properly installed, the Mclone provides the `mclone` command line utility.
+
+```shell
+$ mclone -h
+```
+## Basic use case
+
+Let's start with the simplest case.
+
+Suppose you have a data directory `/data` and you'd want to set up the backup of the `/data/files` subdirectory
+into a backup directory `/mnt/backup`. The latter may be an ordinary directory or a mounted portable storage, or whatever.
+
+### 1. Format the Mclone volumes
+
+Mclone has a notion of a volume - a file system directory containing the `.mclone` file, which is used as a root directory
+for all Mclone operations.
+
+By default, in order to detect currently available volumes the Mclone scans all mount points on *NIX systems and
+all available disk drives on Windows system. Additionally, a static volume directories list to consider can be specified
+in the `MCLONE_PATH` environment variable which is a PATH-like list of directories separated by the double colon `:`
+on *NIX systems or the semicolon `;` on Windows system.
+
+If the `/data` is a regular directory, it won't be picked up by the Mclone automatically, so it needs to be put into
+the environment for later reuse
+
+```shell
+export MCLONE_PATH=/data
+```
+
+On the other hand, if the `/mnt/backup` is a mount point for a portable storage, it will be autodetected,
+therefore there is no need to put it there.
+
+Both source and destination endpoints have to "formatted" in order to be recognized as the Mclone volumes
+
+```shell
+$ mclone volume create /data
+$ mclone volume create /mnt/backup
+```
+
+After that, `mclone info` can be used to review the recognized volumes
+
+```shell
+$ mclone info
+
+# Mclone version 0.1.0
+
+## Volumes
+
+* [6bfa4a2d] :: (/data)
+* [7443e311] :: (/mnt/backup)
+```
+
+Each volume is identified by the randomly generated tag shown within the square brackets `[...]`.
+_Obviously, the tags will be different in your case._
+
+### 2. Create the Mclone task
+
+A Mclone task corresponds to a single Rclone command. It contains the source and destination volume identifiers,
+the source and destination subdirectories _relative to the respective volumes_,
+as well as additional Rclone command line arguments to be used.
+
+_There can be multiple tasks linking different source and destination volumes as well as their respective subdirectores._
+
+A task with all defaults is created with
+
+```shell
+$ mclone task create /data/files /mnt/backup/files
+```
+
+Note that at with point there is no need to use the above volume tags as they will be auto-determined during task creation.
+
+Again, use the `mclone info` to review the changes
+
+```shell
+# Mclone version 0.1.0
+
+## Volumes
+
+* [6bfa4a2d] :: (/data)
+* [7443e311] :: (/mnt/backup)
+
+## Intact tasks
+
+* [cef63f5e] :: update [6bfa4a2d](files) -> [7443e311](files) :: include **
+```
+
+The output literally means: ready to process (intact) update `cef63f5e`  task from the `files` source subdirectory of the
+`6bfa4a2d`  volume to the `files` destination subdirectory of the `7443e311`  volume
+including `**` all files and subdirectories.
+
+Again, the task's tag is randomly generated and will be different in your case.
+
+There are two kinds of tasks to encounter - intact and stale.
+
+An intact task is a task which is fully ready for processing with the Rclone.
+As with the volumes, its tag is shown in the square brackets `[...]`
+
+Conversely, a stale task is not ready for processing due to currently missing source or destination volume.
+A stale task's tag is shown in the angle brackets `<...>`. Also, a missing stale task's volume tag will also be shown in
+the angle brackets.
+
+Thank to the indirection in the source and destination directories, **this task will be handled properly regardless of the
+portable storage directory it will be mounted in next time provided that it will be detectable by the Mclone**.
+
+The same applies to the Windows system where the portable storage can be appear as different disk drives and yet
+be detectable by the Mclone.
+
+### 3. Modify the Mclone task
+
+Once a task is created, its source and destination volumes and directories get fixed and can not be changed.
+Therefore the only way to modify it is to start from scratch preceded by the task deletion with the `mclone task delete` command.
+
+A task's optional parameters however can be modified afterwards with the `mclone task modify` command.
+
+Suppose you'd want to change the operation mode from default updating to synchronization and exclude `.bak` files.
+
+```shell
+$ mclone task modify -m sync -x '*.bak' cef
+```
+
+This time the task is identified by its tag instead of a directory.
+
+Note the mode and task's tag abbreviations: `synchronize` is reduced to `sync` (or it can be cut down further to `sy`)
+and the tag is reduced from full `cef63f5e` to `cef` for convenience and type saving.
+Any part of the full word can be used as an abbreviation provided it is unique among all other full words of the same kind
+otherwise the Mclone will bail out with error.
+
+The abbreviations are supported for operation mode, volume and task tags.
+
+Behold the changes
+
+```shell
+$ mclone info
+
+# Mclone version 0.1.0
+
+## Volumes
+
+* [6bfa4a2d] :: (/data)
+* [7443e311] :: (/mnt/backup)
+
+## Intact tasks
+
+* [cef63f5e] :: synchronize [6bfa4a2d](files) -> [7443e311](files) :: include ** :: exclude *.bak
+```
+
+### 4. Process the tasks
+
+Once created all intact tasks can be (sequentially) processed with the `mclone task process` command.
+
+```shell
+$ mclone task process
+```
+
+If specific tasks need to be processed, their (possibly abbreviated) tags are specified as command line arguments
+
+
+```shell
+$ mclone task process cef
+```
+
+Technically, for a task to be processed the Mclone renders the full source and destination path names from the respective
+volume locations and relative paths and passes them along with other options to the Rclone to do the actual processing.
+
+Thats it. No more need to determine (and type in) current locations of the backup directory and retype all those Rclone arguments
+for every occasion.
+
+## Advanced use case
+
+Now back to the triple storage scenario outlined above.
+
+Let **S** be a source storage from where the data needs to be backed up, **D** be a destination storage where the data is to
+be mirrored and **P** be a portable storage which serves as both an intermediate storage and a means of the **S->D** data propagation.
+
+In this case the full data propagation graph is **S->P->D**.
+
+### 1. Set up the S->P route
+
+1.1. Plug in the **P** portable storage to the **S**'s computer and mount it.
+
+1.2. As shown in the basic use case, create **S**'s and **P**'s volumes, then create a **S->P** task.
+
+1.3. Unplug **P**.
+
+At this point **S** and **P** are now separated and each carry its own copy of the **S->P** task.
+
+### 2. Set up the P->D route
+
+2.1. Plug in the **P** portable storage to the **D**'s computer and mount it.
+
+Note that at this point the **S->P** is a stale task as **D**'s computer knows nothing about **S** storage.
+
+2.2. Create the **D**'s volume, then create a **P->D** task.
+Note that **P** at this point already contains a volume and therefore must not be formatted.
+
+2.3. Unplug **P**.
+
+Now **S** and **D** are formatted and carry the respective tasks.
+**P** contains its own copies of both **S->P** and **P->D** tasks.
+
+### 3. Process the **S->P->D** route
+
+3.1. Plug in **P** to the **S**'s computer and mount it.
+
+3.2. Process the intact tasks. In this case it is the **S->P** task (**P->D** is stale at this point).
+
+3.3. Unplug **P**.
+
+**P** now carries its own copy of the **S**'s data.
+
+3.4. Plug in **P** to the **D**'s computer and mount it.
+
+3.5. Process the intact tasks. In this case it is the **P->D** task (**S->P** is stale at this point).
+
+3.6. Unplug **P**.
+
+_Voilà!_
+Both **P** and **D** now carry a copy of the **S**'s data.
+
+There may be more complex data propagation scenarios with  multiple source and destination storages utilizing the portable
+storage in the above way.
+
+Consider a two-way synchronization between two storages with a portable ferry which carries and propagates data in both directions.
+
+## Whats next
+
+### On-screen help
+
+Every `mclone` (sub)command has its own help page which can be shown with `--help` option
+
+```shell
+$ mclone task create --help
+
+Usage:
+    mclone task create [OPTIONS] SOURCE DESTINATION
+
+Parameters:
+    SOURCE                   Source path
+    DESTINATION              Destination path
+
+Options:
+    -m, --mode MODE          Operation mode (update | synchronize | copy | move) (default: "update")
+    -i, --include PATTERN    Include paths pattern (default: "**")
+    -x, --exclude PATTERN    Exclude paths pattern
+    -f, --force              Insist on potentially dangerous activities (default: false)
+    -n, --dry-run            Simulation mode with no on-disk modifications (default: false)
+    -v, --verbose            Verbose operation (default: false)
+    -h, --help               print help
+```
+
+### File filtering
+
+The Mclone passes its include and exclude options to the Rclone.
+The pattern format is an extended glob (`*.dat`) format described in detail in the corresponding Rclone
+documentation [section](https://rclone.org/filtering).
+
+### Dry run
+
+The Mclone respects the Rclone's dry run mode activated with `--dry-run` command line option in which case
+no volume (`.mclone`) files are ever touched (created, overwritten) during any operation.
+The Rclone is run during task processing but in turn is supplied with this option.
+
+### Force mode
+
+The Mclone will refuse to automatically perform certain actions which are considered dangerous, such as deleting a volume
+or overwriting existing task.
+In this case a `--force` command line option should be used to pass through.
+
+### Task operation modes
+
+#### Update
+
+* Copy source files which are newer than the destination's or have different size or checksum.
+
+* Do not delete destination files which are nonexistent in the source.
+
+* Do not copy source files which are older than the destination's.
+
+A default refreshing mode which is considered to be least harmful with respect to the unintentional data override.
+
+Rclone [command](https://rclone.org/commands/rclone_copy): `copy --update`.
+
+#### Synchronize
+
+* Copy source files which are newer than the destination's or have different size or checksum.
+
+* Delete destination files which are nonexistent in the source.
+
+* Copy source files which are older than the destination's.
+
+This is the mirroring mode which makes destination completely identical to the source.
+
+Rclone [command](https://rclone.org/commands/rclone_sync): `sync`.
+
+#### Copy
+
+* Copy source files which are newer than the destination's or have different size or checksum.
+
+* Do not delete destination files which are nonexistent in the source.
+
+* Do not copy source files which are older than the destination's.
+
+This mode is much like synchronize with only difference that it does not delete files.
+
+Rclone [command](https://rclone.org/commands/rclone_copy): `copy`.
+
+#### Move
+
+* Copy source files which are newer than the destination's or have different size or checksum.
+
+* Do not delete destination files which are nonexistent in the source.
+
+* Do not copy source files which are older than the destination's.
+
+* Delete source files after successful copy to the destination.
+
+Rclone [command](https://rclone.org/commands/rclone_move): `move`.
+
+## The end
+
+Cheers,
+
+Oleg A. Khlybov <fougas@mail.ru>

data/bin/mclone

@@ -0,0 +1,144 @@
+#!/usr/bin/env ruby
+
+# frozen_string_literal: true
+
+require 'clamp'
+require 'mclone'
+
+include Mclone
+
+begin
+
+Clamp do
+
+  using Refinements
+
+  option ['-f', '--force'], :flag, 'Insist on potentially dangerous activities', default: false
+  option ['-n', '--dry-run'], :flag, 'Simulation mode with no on-disk modifications', default: false
+  option ['-v', '--verbose'], :flag, 'Verbose operation', default: false
+
+  option ['-V', '--version'], :flag, 'Show version' do
+    puts VERSION
+    exit(true)
+  end
+
+  def session
+    session = Mclone::Session.new
+    session.force = force?
+    session.simulate = dry_run?
+    session.verbose = verbose?
+    session.restore_volumes!
+    session
+  end
+
+  def resolve_mode(mode)
+    case (m = Task::MODES.resolve(mode)).size
+    when 0 then raise(Task::Error, %(no modes matching pattern "#{mode}"))
+    when 1 then m.first
+    else raise(Task::Error, %(ambiguous mode pattern "#{mode}"))
+    end
+  end
+
+  subcommand 'info', 'Output information on volumes & tasks' do
+    def execute
+      s = session
+      $stdout.puts "# Mclone version #{Mclone::VERSION}"
+      $stdout.puts
+      $stdout.puts '## Volumes'
+      $stdout.puts
+      s.volumes.each do |volume|
+        $stdout.puts "* [#{volume.id}] :: (#{volume.root})"
+      end
+      stales = []
+      intacts = []
+      intact_tasks = s.intact_tasks
+      s.tasks.each do |task|
+        ts = (t = intact_tasks[task]).nil? ? "<#{task.id}>" : "[#{task.id}]"
+        svs = s.volumes.volume(task.source_id).nil? ? "<#{task.source_id}>" : "[#{task.source_id}]"
+        dvs = s.volumes.volume(task.destination_id).nil? ? "<#{task.destination_id}>" : "[#{task.destination_id}]"
+        xs = ["* #{ts} :: #{task.mode} #{svs}(#{task.source_root}) -> #{dvs}(#{task.destination_root})"]
+        xs << "include #{task.include}" unless task.include.nil? || task.include.empty?
+        xs << "exclude #{task.exclude}" unless task.exclude.nil? || task.exclude.empty?
+        (t.nil? ? stales : intacts) << xs.join(' :: ')
+      end
+      unless intacts.empty?
+        $stdout.puts
+        $stdout.puts '## Intact tasks'
+        $stdout.puts
+        intacts.each { |x| $stdout.puts x }
+      end
+      unless stales.empty?
+        $stdout.puts
+        $stdout.puts '## Stale tasks'
+        $stdout.puts
+        stales.each { |x| $stdout.puts x }
+      end
+    end
+  end
+
+  subcommand 'volume', 'Volume operations' do
+
+    subcommand ['new', 'create'], 'Create new volume' do
+      parameter 'DIRECTORY', 'Directory to become a Mclone volume'
+      def execute
+        session.format_volume!(directory)
+      end
+    end
+
+    subcommand 'delete', 'Delete existing volume' do
+      parameter 'VOLUME', 'Volume ID pattern'
+      def execute
+        session.delete_volume!(volume).commit!
+      end
+    end
+
+  end
+
+  subcommand ['task'], 'Task operations' do
+
+    def self.set_task_opts
+      modes = Task::MODES.collect(&:to_s).join(' | ')
+      option ['-m', '--mode'], 'MODE', "Operation mode (#{modes})", default: Task::MODES.first.to_s
+      option ['-i', '--include'], 'PATTERN', 'Include paths pattern', default: '**'
+      option ['-x', '--exclude'], 'PATTERN', 'Exclude paths pattern'
+    end
+
+    subcommand ['new', 'create'], 'Create new SOURCE -> DESTINATION task' do
+      set_task_opts
+      parameter 'SOURCE', 'Source path'
+      parameter 'DESTINATION', 'Destination path'
+      def execute
+        session.create_task!(source, destination, mode: resolve_mode(mode), include: include, exclude: exclude).commit!
+      end
+    end
+
+    subcommand 'modify', 'Modify existing task' do
+      set_task_opts
+      parameter 'TASK', 'Task ID pattern'
+      def execute
+        session.modify_task!(task, mode: resolve_mode(mode), include: include, exclude: exclude).commit!
+      end
+    end
+
+    subcommand 'delete', 'Delete existing task' do
+      parameter 'TASK', 'Task ID pattern'
+      def execute
+        session.delete_task!(task).commit!
+      end
+    end
+
+    subcommand 'process', 'Process specified tasks' do
+      parameter '[TASK] ...', 'Task ID pattern(s)', attribute_name: :tasks
+      def execute
+        session.process_tasks!(*tasks)
+      end
+    end
+
+  end
+
+end
+
+rescue Mclone::Error
+  $stderr.puts "ERROR: #{$!.message}"
+  exit(false)
+end

data/lib/mclone.rb

@@ -0,0 +1,647 @@
+# frozen_string_literal: true
+
+
+require 'date'
+require 'json'
+require 'fileutils'
+require 'securerandom'
+
+
+#
+module Mclone
+
+
+  VERSION = '0.1.0'
+
+
+  #
+  class Error < StandardError
+
+  end
+
+  #
+  module Refinements
+
+    refine ::Hash do
+      # Same as #dig but raises KeyError exception on any non-existent key
+      def extract(*args)
+        case args.size
+        when 0 then raise(KeyError, 'non-empty key sequence expected')
+        when 1 then fetch(args.first)
+        else fetch(args.shift).extract(*args)
+        end
+      end
+    end
+
+    refine ::Array do
+      # Return a list of items which fully or partially match the specified pattern
+      def resolve(partial)
+        rx = Regexp.new(partial)
+        collect { |item| rx.match?(item.to_s) ? item : nil }.compact
+      end
+    end
+
+  end
+
+
+  using Refinements
+
+
+  # Two-way mapping between an object and its ID
+  class ObjectSet
+
+    include Enumerable
+
+    #
+    def each_id(&code)
+      @ids.each_key(&code)
+    end
+
+    #
+    def each(&code)
+      @objects.each_value(&code)
+    end
+
+    #
+    def empty?
+      @objects.empty?
+    end
+
+    def initialize
+      @ids = {} # { id => object }
+      @objects = {} # { object => object }
+      @modified = false
+    end
+
+    # Return ID of the object considered equal to the specified obj or nil
+    def id(obj)
+      @objects[obj]&.id
+    end
+
+    # Return object with specified ID or nil
+    def object(id)
+      @ids[id]
+    end
+
+    # Return object considered equal to obj or nil
+    def [](obj)
+      @objects[obj]
+    end
+
+    #
+    def modified?
+      @modified
+    end
+
+    def commit!
+      @modified = false
+      self
+    end
+
+    # Unregister an object considered equal to the specified obj and return true if object has been actually removed
+    private def forget(obj)
+      !@ids.delete(@objects.delete(obj)&.id).nil?
+    end
+
+    # Return a list of registered IDs (fully or partially) matching the specified pattern
+    def resolve(pattern)
+      each_id.to_a.resolve(pattern)
+    end
+
+    # Either add brand new object or replace existing one equal to the specified object
+    def <<(obj)
+      forget(obj)
+      @objects[obj] = @ids[obj.id] = obj
+      @modified = true
+      obj
+    end
+
+    # Remove object considered equal to the specified obj
+    def >>(obj)
+      @modified = true if (status = forget(obj))
+      status
+    end
+
+    # Add all tasks from enumerable
+    def merge!(objs)
+      objs.each { |x| self << x }
+      self
+    end
+
+  end
+
+
+  #
+  class Task
+
+    #
+    class Error < Mclone::Error
+    end
+
+    #
+    attr_reader :id
+
+    #
+    attr_reader :source_id, :destination_id
+
+    #
+    attr_reader :source_root, :destination_root
+
+    #
+    attr_reader :mtime
+
+    #
+    attr_reader :mode
+
+    #
+    attr_reader :include, :exclude
+
+    def hash
+      @hash ||= source_id.hash ^ destination_id.hash ^ source_root.hash ^ destination_root.hash
+    end
+
+    def eql?(other)
+      equal?(other) || (
+        source_id == other.source_id &&
+        destination_id == other.destination_id &&
+        source_root == other.source_root &&
+        destination_root == other.destination_root
+      )
+    end
+
+    alias == eql?
+
+    #
+    def initialize(source_id, source_root, destination_id, destination_root)
+      @touch = false # Indicates that the time stamp should be updated whenever state of self is altered
+      @id = SecureRandom.hex(4)
+      @source_id = source_id
+      @destination_id = destination_id
+      @source_root = source_root
+      @destination_root = destination_root
+      self.mode = :update
+      self.include = '**'
+      self.exclude = ''
+    ensure
+      @touch = true
+      touch!
+    end
+
+    #
+    MODES = [:update, :synchronize, :copy, :move].freeze
+
+    #
+    def mode=(mode)
+      unless mode.nil?
+        @mode = MODES.include?(mode = mode.intern) ? mode : raise(Task::Error, %(unknown mode "#{mode}"))
+        touch!
+      end
+    end
+
+    #
+    def include=(mask)
+      unless mask.nil?
+        @include = mask # TODO extensive verification
+        touch!
+      end
+    end
+
+    #
+    def exclude=(mask)
+      unless mask.nil?
+        @exclude = mask # TODO extensive verification
+        touch!
+      end
+    end
+
+    #
+    def self.restore(hash)
+      obj = allocate
+      obj.send(:from_h, hash)
+      obj
+    end
+
+    #
+    private def from_h(hash)
+      initialize(hash.extract(:source, :volume), hash.extract(:source, :root), hash.extract(:destination, :volume), hash.extract(:destination, :root))
+      @touch = false
+      @id = hash.extract(:task)
+      self.mode = hash.extract(:mode)
+      # Deleting the mtime key from json
+      @mtime = DateTime.parse(hash.extract(:mtime)) rescue @mtime
+      self.include = hash.extract(:include) rescue nil
+      self.exclude = hash.extract(:exclude) rescue nil
+    ensure
+      @touch = true
+    end
+
+    #
+    def to_h
+      {
+        mode: mode,
+        include: include,
+        exclude: exclude,
+        task: id,
+        source: {volume: source_id, root: source_root},
+        destination: {volume: destination_id, root: destination_root},
+        mtime: mtime
+      }
+    end
+
+    #
+    def touch!
+      @mtime = DateTime.now if @touch
+    end
+  end
+
+
+  #
+  class TaskSet < ObjectSet
+
+    alias task object
+
+    # Add new task or replace existing one with outdated timestamp
+    def <<(task)
+      t = self[task]
+      super if t.nil? || (!t.nil? && t.mtime < task.mtime)
+      task
+    end
+
+    #
+    def resolve(id)
+      case (ids = super).size
+      when 0
+        raise(Task::Error, %(no task matching "#{id}" pattern found))
+      when 1
+        ids.first
+      else
+        raise(Task::Error, %(ambiguous "#{id}" pattern: two or more tasks match))
+      end
+    end
+
+  end
+
+
+  #
+  class Volume
+
+    #
+    class Error < Mclone::Error
+
+    end
+
+    #
+    VERSION = 0
+
+    #
+    FILE = '.mclone'
+
+    #
+    attr_reader :id
+
+    #
+    attr_reader :file
+
+    #
+    attr_reader :tasks
+
+    #
+    def root
+      @root ||= File.realpath(File.dirname(file))
+    end
+
+    #
+    def initialize(file)
+      @file = file
+      @id = SecureRandom.hex(4)
+      @tasks = VolumeTaskSet.new(self)
+    end
+
+    #
+    def self.restore(file)
+      obj = allocate
+      obj.send(:from_file, file)
+      obj
+    end
+
+    #
+    private def from_file(file)
+      initialize(file)
+      hash = JSON.parse(IO.read(file), symbolize_names: true)
+      raise(Volume::Error, %(unsupported Mclone volume format version "#{version}")) unless hash.extract(:mclone) == VERSION
+      @id = hash.extract(:volume).to_s
+      hash.extract(:tasks).each { |t| @tasks << Task.restore(t) }
+    end
+
+    #
+    def dirty?
+      tasks.modified?
+    end
+
+    #
+    def hash
+      id.hash
+    end
+
+    #
+    def eql?(other)
+      equal?(other) || id == other.id
+    end
+
+    #
+    def commit!(force = false)
+      if force || dirty?
+        open(file, 'w') do |stream|
+          stream << JSON.pretty_generate(to_h)
+          tasks.commit!
+        end
+      end
+    end
+
+    #
+    def to_h
+      {mclone: VERSION, volume: id, tasks: tasks.collect(&:to_h)}
+    end
+
+    # Volume-bound set of tasks belonging to the specific volume
+    class VolumeTaskSet < Mclone::TaskSet
+
+      def initialize(volume)
+        @volume = volume
+        super()
+      end
+
+      # Accept only the tasks referencing the volume as either source or destination
+      def <<(task)
+        task.source_id == @volume.id || task.destination_id == @volume.id ? super : task
+      end
+
+    end
+  end
+
+
+  #
+  class VolumeSet < ObjectSet
+
+    alias volume object
+
+    #
+    def resolve(id)
+      case (ids = super).size
+      when 0
+        raise(Volume::Error, %(no volume matching "#{id}" pattern found))
+      when 1
+        ids.first
+      else
+        raise(Volume::Error, %(ambiguous "#{id}" pattern: two or more volumes match))
+      end
+    end
+
+  end
+
+
+  #
+  class Session
+
+    #
+    class Error < Mclone::Error
+
+    end
+
+    #
+    attr_reader :volumes
+
+    #
+    def simulate?
+      @simulate == true
+    end
+
+    #
+    def verbose?
+      @verbose == true
+    end
+
+    #
+    def force?
+      @force == true
+    end
+
+    #
+    attr_writer :simulate, :verbose, :force
+
+    #
+    def initialize
+      @volumes = VolumeSet.new
+    end
+
+    #
+    def format_volume!(dir)
+      mclone = File.join(dir, Volume::FILE)
+      raise(Session::Error, %(refuse to overwrite existing Mclone volume file "#{mclone}")) if File.exist?(mclone) && !force?
+      volume = Volume.new(mclone)
+      @volumes << volume
+      volume.commit!(true) unless simulate? # Force creating a new (empty) volume
+      self
+    end
+
+    #
+    def restore_volume!(dir)
+      volume = Volume.restore(File.join(dir, Volume::FILE))
+      @volumes << volume
+      self
+    end
+
+    #
+    def restore_volumes!
+      (Mclone.environment_mounts + Mclone.system_mounts).each { |dir| restore_volume!(dir) rescue Errno::ENOENT }
+      self
+    end
+
+    #
+    def delete_volume!(id)
+      volume = volumes.volume(id = volumes.resolve(id))
+      raise(Session::Error, %(refuse to delete non-empty Mclone volume file "#{volume.file}")) unless volume.tasks.empty? || force?
+      volumes >> volume
+      FileUtils.rm_f(volume.file) unless simulate?
+      self
+    end
+
+    #
+    def create_task!(source, destination, mode: :update, include: '**', exclude: '')
+      task = Task.new(*locate(source), *locate(destination))
+      task.mode = mode
+      task.include = include
+      task.exclude = exclude
+      volumes.each do |volume|
+        t = volume.tasks[task]
+        raise(Session::Error, %(refuse to overwrite existing task "#{t.id}")) unless t.nil? || force?
+        volume.tasks << task # It is a volume's responsibility to collect appropriate tasks, see Volume::TaskSet#<<
+      end
+      self
+    end
+
+    #
+    def modify_task!(id, mode:, include:, exclude:)
+      ts = tasks
+      task = ts.task(ts.resolve(id)).clone
+      task.mode = mode
+      task.include = include
+      task.exclude = exclude
+      volumes.each { |volume| volume.tasks << task }
+      self
+    end
+
+    #
+    def delete_task!(id)
+      ts = tasks
+      task = ts.task(ts.resolve(id))
+      volumes.each { |volume| volume.tasks >> task }
+      self
+    end
+
+    #
+    def process_tasks!(*ids)
+      ts = intact_tasks
+      ids = ts.collect(&:id) if ids.empty?
+      rclone = 'rclone' if (rclone = ENV['RCLONE']).nil?
+      ids.collect { |id| ts.task(ts.resolve(id)) }.each do |task|
+        args = [rclone]
+        opts = [
+          simulate? ? '--dry-run' : nil,
+          verbose? ? '--verbose' : nil
+        ].compact
+        case task.mode
+        when :update
+          args << 'copy'
+          opts << '--update'
+        when :synchronize
+          args << 'sync'
+        when :copy
+          args << 'copy'
+        when :move
+          args << 'move'
+        end
+        opts.append('--filter', "- /#{Volume::FILE}")
+        opts.append('--filter', "- #{task.exclude}") unless task.exclude.nil? || task.exclude.empty?
+        opts.append('--filter', "+ #{task.include}") unless task.include.nil? || task.include.empty?
+        args.concat(opts)
+        args.append(File.join(volumes.volume(task.source_id).root, task.source_root), File.join(volumes.volume(task.destination_id).root, task.destination_root))
+        case system(*args)
+        when nil then raise(Session::Error, %(failed to execute "#{args.first}"))
+        when false then exit($?)
+        end
+      end
+    end
+
+    # Collect all tasks from all loaded volumes
+    def tasks
+      tasks = SessionTaskSet.new(self)
+      volumes.each { |volume| tasks.merge!(volume.tasks) }
+      tasks
+    end
+
+    # Collect all tasks from all loaded volumes which are ready to be executed
+    def intact_tasks
+      tasks = IntactTaskSet.new(self)
+      volumes.each { |volume| tasks.merge!(volume.tasks) }
+      tasks
+    end
+
+    #
+    private def locate(path)
+      path = File.realpath(path)
+      x = volumes.each.collect { |v| Regexp.new(%!^#{v.root}/?(.*)!, Mclone.windows? ? Regexp::IGNORECASE : nil) =~ path ? [v.root, v.id, $1] : nil }.compact
+      if x.empty?
+        raise(Session::Error, %(path "#{path}" does not belong to a loaded Mclone volume))
+      else
+        root, volume, path = x.sort { |a,b| a.first.size <=> b.first.size}.last
+        [volume, path]
+      end
+    end
+
+    #
+    def commit!
+      volumes.each { |v| v.commit!(force?) } unless simulate?
+      self
+    end
+
+    #
+    class SessionTaskSet < Mclone::TaskSet
+
+      def initialize(session)
+        @session = session
+        super()
+      end
+
+    end
+
+    # Session-bound set of intact tasks for which both source and destination volumes are loaded
+    class IntactTaskSet < SessionTaskSet
+
+      # Accept only intact tasks for which both source and destination volumes are loaded
+      def <<(task)
+        @session.volumes.volume(task.source_id).nil? || @session.volumes.volume(task.destination_id).nil? ? task : super
+      end
+
+    end
+
+  end
+
+  # Return true if run in the real Windows environment (e.g. not in real *NIX or various emulation layers such as MSYS, Cygwin etc.)
+  def self.windows?
+    @@windows ||= /^(mingw)/.match?(RbConfig::CONFIG['target_os']) # RubyInstaller's MRI, other MinGW-build MRI
+  end
+
+  # Match OS-specific system mount points (/dev /proc etc.) which normally should be omitted when scanning for Mclone voulmes
+  UNIX_SYSTEM_MOUNTS = %r!^/(dev|sys|proc|run)!
+
+  # TODO handle Windows variants
+  # Specify OS-specific path name list separator (such as in the $PATH environment variable)
+  PATH_LIST_SEPARATOR = windows? ? ';' : ':'
+
+  # Return list of live user-provided mounts (mount points on *NIX and disk drives on Windows) which may contain Mclone volumes
+  # Look for the $mclone_PATH environment variable
+  def self.environment_mounts
+    ENV['MCLONE_PATH'].split(PATH_LIST_SEPARATOR).collect { |path| File.directory?(path) ? path : nil }.compact rescue []
+  end
+
+  # Return list of live system-managed mounts (mount points on *NIX and disk drives on Windows) which may contain Mclone volumes
+  case RbConfig::CONFIG['target_os']
+  when 'linux'
+    # Linux OS
+    def self.system_mounts
+      # Query on /proc for currently mounted file systems
+      IO.readlines('/proc/self/mountstats').collect do |line|
+        mount = line.split[4]
+        UNIX_SYSTEM_MOUNTS.match?(mount) || !File.directory?(mount) ? nil : mount
+      end.compact
+    end
+    # TODO handle Windows variants
+  when /^mingw/ # RubyInstaller's MRI
+    require 'ffi'
+    module FileAPI
+      extend FFI::Library
+      ffi_lib 'kernel32'
+      ffi_convention :stdcall
+      attach_function :disks_mask, :GetLogicalDrives, [], :ulong
+    end
+    def self.system_mounts
+      mask = FileAPI.disks_mask
+      mounts = []
+      ('A'..'Z').each do |x|
+        mounts << "#{x}:" if mask & 1 == 1
+        mask >>= 1
+      end
+      mounts
+    end
+  else
+    # Generic *NIX-like OS, including Cygwin & MSYS(2)
+    def self.system_mounts
+      # Use $(mount) system utility to obtain currently mounted file systems
+      %x(mount).split("\n").collect do |line|
+        mount = line.split[2]
+        UNIX_SYSTEM_MOUNTS.match?(mount) || !File.directory?(mount) ? nil : mount
+      end.compact
+    end
+  end
+end

metadata

@@ -0,0 +1,76 @@
+--- !ruby/object:Gem::Specification
+name: mclone
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Oleg A. Khlybov
+autorequire:
+bindir: bin
+cert_chain: []
+date: 2021-06-22 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: clamp
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: ffi
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.15'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.15'
+description:
+email:
+- fougas@mail.ru
+executables:
+- mclone
+extensions: []
+extra_rdoc_files:
+- README.md
+files:
+- README.md
+- bin/mclone
+- lib/mclone.rb
+homepage: https://github.com/okhlybov/mclone
+licenses:
+- BSD-3-Clause
+metadata: {}
+post_install_message:
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 2.5.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.9
+signing_key:
+specification_version: 4
+summary: Rclone frontend for offline synchronization
+test_files: []