photein 0.0.2 → 0.0.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8358c084cdf56bab539d9f429706a606cb346b33e7832e056054b2a4ead67e08
-  data.tar.gz: c9e77f835d2ba2d39a3edb08e5fc6973a1871b82c52a823e3bf15a446c263f80
+  metadata.gz: 852660eba8fefe43e1b18967162a149c13f35f85e811f08089403da3f2dd0652
+  data.tar.gz: 8a6af0e7ce270985f6a427dfc509be093d884baedccd33ac778611285fedf71a
 SHA512:
-  metadata.gz: '08be8726129d9b2c2ffcfe4f9ba206db82619c6b6cd1f8f50339d7390988a3c224c9ecb69da7dedc6edfee5e0c479f6ea4119551af75716039e55eda4635cdc7'
-  data.tar.gz: a91a5a3be62081f8a25333f323a1ed98f2e34421134878679d5af8f3fce674c8e7ef8e1a299d4caa3afd9364af38fbe6ef8136045bfe15ac614afd9cf4d2f9ba
+  metadata.gz: f7fc315331d2de5da4df3fb5aacfd7a58220f3581e3753a2c01802e428432e57ab89ac7d66c44bbecfbd8aae7f2e8966e6a18c1857ed26b68b0e2dc046bbf780
+  data.tar.gz: 27f432d3dae62069f2197051e523da3db8e3743467ce2dd17be4e7788720ef74090beb345ee98072d1df2cffe86fb7be22bde32d8d4460048eb7603f92537dbe

data/README.md

@@ -1,16 +1,30 @@
-Ph📸tein
-========
+Ph📸tein / 🔀ferase
+===================
 
 All your photos under one roof.
 
-What does it do?
+What do they do?
 ----------------
 
-This repo provides two related programs: `photein` and `xferase`.
+`photein` is a CLI utility for batch-importing
+your personal photos & videos into a central library.
+
+`xferase` is an always-on background service that uses `photein`
+to continuously import new photos & videos as they come in.
+
+When combined with other software,
+they can be used as a kind of self-hosted / DIY alternative
+to cloud photo services like Google Photos or iCloud.
+
+> ⚠️ **Note**
+>
+> Unlike true cloud photo services,
+> this approach works by keeping a full copy of your photo library
+> in local storage on each device you sync to.
 
 ### Photein
 
-Photein is a CLI utility for managing your photos **at the filesystem level**.
+Photein manages your photos **at the filesystem level**.
 It won’t let you browse or edit your photos,
 but it will give them a uniform folder structure and filenames,
 no matter where they come from:
@@ -35,6 +49,9 @@ no matter where they come from:
     └── VID_20210212_081939.mp4             └── 2021-04-28_000008.jpg
 ```
 
+Photein generates these folders & filenames
+based on metadata timestamps, filename timestamps, or file creation times.
+
 > ⚠️ **Note**
 >
 > If you use a photo management app that decides
@@ -43,8 +60,7 @@ no matter where they come from:
 
 ### Xferase
 
-Xferase is a background service built on top of photein.
-It watches a directory of your choosing,
+Xferase watches a directory of your choosing (its “inbox”),
 and whenever any files are placed there,
 it automatically imports them into your photo library.
 
@@ -53,9 +69,13 @@ It creates and manages two parallel copies of your library
 and ensures that when you delete a photo from one,
 it is automatically removed from the other.
 
-When combined with other software,
-Xferase can be used as a self-hosted / DIY alternative
-to cloud photo services like Google Photos or iCloud.
+With the help of [Syncthing][] and systemd,
+you can automatically pull new photos from your camera or Android phone
+into Xferase’s inbox.
+Syncthing can also push your complete, web-optimized photo library
+back to your phone (or, say, push your hi-res library out to another machine).
+
+[Syncthing]: https://syncthing.net/
 
 Why?
 ----
@@ -70,7 +90,8 @@ I could not find any existing software product that:
 
    (I want to be able to access my photos from the file manager,
    find them in an “Open...” dialog,
-   or sync them to other devices with tools like Dropbox or Syncthing.)
+   or sync them to other devices with generic tools
+   like rsync, Dropbox, or Syncthing.)
 
 3. comes with **no recurring subscription fee**—or better yet, is FOSS
 
@@ -80,12 +101,6 @@ I could not find any existing software product that:
 
 4. works with Linux
 
-> ⚠️ **Note**
->
-> Strictly speaking, Photein does not handle requirement #1;
-> for that, use Xferase in combination with other software,
-> such as Syncthing or systemd.
-
 Installation
 ------------
 
@@ -146,8 +161,7 @@ Contributions welcome.
 > it defines expectations against the effects of `system('photein <args>')`.
 >
 > Because `Kernel#system` runs the given command in a subprocess, 
-> it prints to a different stdout than
-> native Ruby code in a normal RSpec example.
+> it prints to a different stdout than `rspec` itself.
 > This makes test failures cumbersome to debug,
 > because `puts` statements never appear in the test output,
 > and `binding.pry` will cause the test to appear to hang

data/bin/xferase

@@ -0,0 +1,170 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# TODO: Explain yourself
+
+# Parse opts -------------------------------------------------------------------
+require 'photein'
+require 'optparse'
+
+parser = OptionParser.new do |opts|
+  opts.version = Photein::VERSION
+  opts.banner  = <<~BANNER
+    Usage: xferase [--version] [-h | --help] [<args>]
+  BANNER
+
+  opts.on(
+    '-i INBOX', '--inbox=INBOX',
+    'path to the inbox'
+  )
+
+  opts.on(
+    '-s STAGING', '--staging=STAGING',
+    'path to the staging directory'
+  )
+
+  opts.on(
+    '-o LIB_ORIG', '--lib-orig=LIB_ORIG',
+    'path to the original/hi-res library'
+  )
+
+  opts.on(
+    '-w LIB_WEB', '--lib-web=LIB_WEB',
+    'path to the web-optimized library'
+  )
+
+  opts.on(
+    '-d INTERVAL', '--debounce=INTERVAL',
+    'wait n seconds for additional files before import'
+  )
+end
+
+begin
+  params = {}
+  parser.parse!(into: params)
+  params.freeze
+
+  %i[inbox staging lib-orig lib-web].each do |type|
+    raise "no #{type} directory given" if !params.key?(type)
+  end
+rescue => e
+  warn(e.message)
+  warn(parser.help) if e.is_a?(OptionParser::ParseError)
+  exit 1
+end
+
+# Config -----------------------------------------------------------------------
+require 'logger'
+
+DEBOUNCE_WAIT = params.fetch(:'debounce') { 60 }.to_i # seconds
+SRC_DEST_MAP = {
+  "#{params.fetch(:staging)}/desktop" => params.fetch(:'lib-orig'),
+  "#{params.fetch(:staging)}/web" => params.fetch(:'lib-web'),
+}
+
+IMPORT_CMD = %(
+  photein \
+    --verbose \
+    --source %s \
+    --dest %s \
+    --optimize-for %s
+)
+
+FORMAT_GROUPS = [%w(.jpg .dng .heic), %w(.mov .mp4), %w(.png)]
+
+LOGGER = Logger.new($stdout).tap(&:info!)
+
+# Setup ------------------------------------------------------------------------
+require 'debouncer'
+require 'fileutils'
+require 'pathname'
+require 'rb-inotify'
+require 'tmpdir'
+
+Thread.abort_on_exception = true
+SRC_DEST_MAP.keys.each(&FileUtils.method(:mkdir_p))
+
+mutex = Mutex.new
+debouncer = Debouncer.new(DEBOUNCE_WAIT) do |*files|
+  SRC_DEST_MAP.keys.each { |dir| FileUtils.ln(files, dir) }
+  FileUtils.rm(files + Dir["#{params.fetch(:staging)}/web/*.DNG"]) # FIXME (ugly hack)
+
+  break if debouncer.inspect_params[:threads] > 2 # don't let threads pile up
+
+  mutex.synchronize do
+    SRC_DEST_MAP.each do |src, dest|
+      system(IMPORT_CMD % [src, dest, src.split('/').last])
+    end
+  rescue => e
+    warn e.message
+  end
+end.reducer(:+)
+
+# Resume from interruption/failure ---------------------------------------------
+debouncer.call unless SRC_DEST_MAP.keys.all?(&Dir.method(:empty?))
+
+Dir["#{params.fetch(:inbox)}/**/*"]
+  .select(&File.method(:file?))
+  .each { |path| debouncer.call(path) }
+
+# Start ------------------------------------------------------------------------
+Thread.new do
+  call_debouncer = ->(event) do
+    debouncer.call(event.absolute_name)
+  rescue => e
+    warn e.message
+  end
+
+  import_notifier = INotify::Notifier.new
+  import_notifier.watch(params.fetch(:inbox), :close_write, &call_debouncer)
+
+  # NOTE: inotify is not recursive,
+  # so subdirectories must be watched separately!
+  # (Why do Syncthing folders get special treatment?
+  # Because ST works by creating hidden tempfiles and moving them upon completion)
+  stfolders, simple_subdirs = Dir["#{params.fetch(:inbox)}/**/*"]
+    .select(&File.method(:directory?))
+    .partition { |dir| Dir.children(dir).include?('.stfolder') }
+
+  simple_subdirs.each { |dir| import_notifier.watch(dir, :close_write, &call_debouncer) }
+  stfolders.each { |dir| import_notifier.watch(dir, :moved_to, &call_debouncer) }
+
+  import_notifier.run
+end
+
+Thread.new do
+  sync_deletions = ->(event) do
+    LOGGER.info("#{event.name} has disappeared!")
+
+    deleted_file   = Pathname(event.absolute_name).expand_path
+
+    sister_file    = if deleted_file.to_s.start_with?(File.expand_path(params['LIB_ORIG']))
+                      deleted_file.sub(*params.values_at('LIB_ORIG', 'LIB_WEB'))
+                    else
+                      deleted_file.sub(*params.values_at('LIB_WEB', 'LIB_ORIG'))
+                    end
+
+    sister_formats = FORMAT_GROUPS.find { |group| group.include?(deleted_file.extname) }
+
+    related_files  = [deleted_file, sister_file]
+      .product(sister_formats)
+      .map { |file, ext| file.sub_ext(ext) }
+      .select(&:file?)
+
+    # (Why mv to tmpdir first? Why not rm straight away?
+    # Because rm would recursively trigger this inotify callback.)
+    related_files.each { |f| LOGGER.info("deleting #{f.realpath}") }
+    FileUtils.mv(related_files, Dir.tmpdir)
+    FileUtils.rm(related_files.map { |f| File.join(Dir.tmpdir, f.basename) })
+  end
+
+  deletion_notifier = INotify::Notifier.new
+
+  # NOTE: inotify is not recursive,
+  # so subdirectories must be watched separately!
+  (Dir["#{params.fetch(:'lib-orig')}/**/*"] + Dir["#{params.fetch(:'lib-web')}/**/*"])
+    .select(&File.method(:directory?))
+    .each { |dir| deletion_notifier.watch(dir, :delete, &sync_deletions) }
+
+  deletion_notifier.run
+end.join

data/lib/photein/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Photein
-  VERSION = '0.0.2'
+  VERSION = '0.0.4'
 end

metadata

@@ -1,15 +1,29 @@
 --- !ruby/object:Gem::Specification
 name: photein
 version: !ruby/object:Gem::Version
-  version: 0.0.2
+  version: 0.0.4
 platform: ruby
 authors:
 - Ryan Lue
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2021-05-25 00:00:00.000000000 Z
+date: 2021-05-28 00:00:00.000000000 Z
 dependencies:
+- !ruby/object:Gem::Dependency
+  name: debouncer
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.2'
 - !ruby/object:Gem::Dependency
   name: mediainfo
   requirement: !ruby/object:Gem::Requirement
@@ -52,6 +66,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '4.11'
+- !ruby/object:Gem::Dependency
+  name: nokogiri
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.11'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.11'
 - !ruby/object:Gem::Dependency
   name: optipng
   requirement: !ruby/object:Gem::Requirement
@@ -67,19 +95,19 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0.2'
 - !ruby/object:Gem::Dependency
-  name: nokogiri
+  name: rb-inotify
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.11'
+        version: '0.10'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '1.11'
+        version: '0.10'
 - !ruby/object:Gem::Dependency
   name: streamio-ffmpeg
   requirement: !ruby/object:Gem::Requirement
@@ -126,11 +154,13 @@ description: ''
 email: hello@ryanlue.com
 executables:
 - photein
+- xferase
 extensions: []
 extra_rdoc_files: []
 files:
 - README.md
 - bin/photein
+- bin/xferase
 - lib/photein.rb
 - lib/photein/config.rb
 - lib/photein/image.rb