mark-twin 0.2.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 39f52a8341611b4a17bbeaffae64e21e7ffc2123f1b766df494eb422db697cb6
-  data.tar.gz: 224e40d39832efc3542c1c3de00e4c6fe30d01c72ca4f84ff5bc9868f15cedd3
+  metadata.gz: 0a1efe3b9e90b401c9c6f8a25652db649ddea26054ede30f2c221d0d73246297
+  data.tar.gz: b85e21f28805ade66696c9997b986b0472f2d3028971075a30f15d35ede113b7
 SHA512:
-  metadata.gz: 8f217e753b18810cad48b2e6c2f3360a3cc84e4f6449dd14f9f62aae4243b55d44a4d2b8321a8f60228c4ca696e97106f9cdca1e6a54f404436af284130609dd
-  data.tar.gz: 301088d4bd68082461e9e8d0730d1c921336256e8fe3f3a755f918c6e189fc4238fc4bd7c3f7e4e8879474f9736ef4272a5117ce8b3b97316983031e0656201b
+  metadata.gz: cf13dd6e51d53af11adc5b49c5e183ab6dd07ec9a49e4d83cafa60c712ca2bb492bb944f00b1afca864e051c09f7c054465d725d1da858d0960c48f6507778ff
+  data.tar.gz: d7a00b682a1e8b833c7aceda323da4c69625fa3538def0810334dd1ed757696aca7d92d34d1e914bb06b60d1539ed6b990f5b24fdb94a59847bee2404272e51e

data/ARCHITECTURE.md

@@ -27,10 +27,13 @@ sync-files (.md)
 ```
 lib/twin/
   version.rb
+  remote.rb     ssh targets: detection, reachability, batched stat, mkdir
   template.rb   {{token}} substitution + render-file helper
   config.rb     ~/.config/twin/config.yaml loader; host table → var_map
   scanner.rb    Job, Program structs; grubber + template + stat → grouped Programs
   sync.rb       rsync / render execution, mount check, post-sync hook
+  journal.rb    append-only sync journal (~/.local/state/twin/log.jsonl)
+  add.rb        `twin add` — interactive scaffolding of new sync entries
   picker.rb     fzf wrapper with apex preview
   cli.rb        subcommand dispatcher
 
@@ -48,8 +51,8 @@ delete, render, render_outdated, target_path_field, sync_file,
 source_exists, target_exists, source_mtime, target_mtime, conflict
 ```
 
-`Job#status` → one of `disabled / both_missing / missing_source / missing_target /
-target_newer / in_sync / source_newer`. Render jobs derive status from content
+`Job#status` → one of `disabled / unreachable / both_missing / missing_source /
+missing_target / target_newer / in_sync / source_newer`. Render jobs derive status from content
 (`render_outdated`), not mtime; non-render jobs ignore those fields.
 `Job#target_path` joins `target` with `target_path_field || path`.
 
@@ -151,17 +154,60 @@ File argument resolution (`twin <arg>` and `--file=<arg>`):
 Unknown options (anything starting with `-` that isn't `--help`) print an
 error pointing at `twin --help` and exit 1.
 
+`twin sync` returns exit 1 when any job failed. `--quiet` suppresses output
+for successful no-op jobs (conflicts, errors and real transfers still print);
+`--skip-unavailable` skips unmounted/unreachable targets instead of aborting.
+The combination is the unattended-run mode (launchd/cron).
+
+Every non-dry-run job lands in the journal (`Journal.record`): one JSON line
+in `~/.local/state/twin/log.jsonl` (`TWIN_STATE_DIR` overrides the directory)
+with timestamp, program, path, target, `ok`, `changed`, and a truncated error
+line on failure. `twin log [-n N] [--json]` reads it back. Journal write
+failures warn once and never break a sync.
+
+`twin add <path>` (`add.rb`) scaffolds a new entry: it matches the expanded
+path against the token-substituted `Source:` frontmatter of every sync-file
+(files with no or foreign roots drop out), computes `Path:` relative to the
+chosen root, suggests excludes from a fixed list of generated/heavy dirs found
+in the source (`SUGGEST_EXCLUDES`), rejects paths the file already has a
+block for, and appends heading + prose stub + YAML block. With no covering
+sync-file it can create one (frontmatter from prompts). The pure helpers
+(`frontmatter`, `candidates`, `relative_path`, `suggest_excludes`,
+`build_block`) are unit-tested; the prompt flow reads plain stdin, so it is
+scriptable by piping answers.
+
 `twin doctor` checks required tools (grubber, rsync, fzf), optional renderers
 (apex, glow, bat), templating (host/target resolve, every `{{token}}` resolves),
 and whether all configured sync targets are mounted. Exits 1 if any required
 check fails.
 
+## Remote targets
+
+`Target: user@host:/path` (rsync notation; colon before the first slash) makes
+a job remote — `Job#remote?`. Sources stay local, twin pushes.
+
+- **Stat**: remote paths can't be `File.stat`ed, so `build_job` leaves them
+  "missing" and `Scanner.fill_remote_stats` fills them in afterwards — one
+  `ssh` round-trip per host for all its paths (`Remote.stat_paths`: paths over
+  stdin, `path\tepoch` back; BSD `stat -f %m` with GNU `stat -c %Y` fallback).
+  A failed ssh sets `target_unreachable` → status `:unreachable`; the scan
+  itself never fails on a dead host.
+- **Reachability** replaces the mount check (`Remote.reachable?`,
+  `ssh -o BatchMode=yes … true` — key auth only, never prompts).
+- **rsync** needs no changes: the target string is already in its remote
+  syntax. Parent directories are created via `ssh host mkdir -p` first.
+- **`Cmd`** still runs locally (`sh -c`); acting on the server means writing
+  an `ssh host '…'` command in the sync-file.
+- **`Render: true` + remote raises** at scan time — render reads/writes target
+  content, which twin only does on local (mounted) paths.
+
 ## Sync
 
 Before syncing:
 
-1. **Mount check** — every unique target root must be a mount point
-   (`File.stat.dev != parent.dev`). Aborts if unmounted.
+1. **Mount check** — every unique local target root must be a mount point
+   (`File.stat.dev != parent.dev`); remote targets must be ssh-reachable.
+   Aborts otherwise.
 2. **Conflict warning** — emits stderr listing jobs where the target is
    newer than the source. Continues anyway (`rsync --update` skips them).
 
@@ -171,7 +217,13 @@ Then per Job, **rsync path** (non-render):
 rsync -av --itemize-changes --update [--delete] [--exclude=...]* src/ tgt/
 ```
 
-`--delete` is added when the Job has `delete: true` (from `Delete: true`).
+`--delete` is added when the Job has `delete: true` (from `Delete: true`),
+together with `--backup --backup-dir=<target>/.twin-backup/<run-stamp>` —
+deleted and overwritten files are moved aside, not destroyed. The stamp is
+per-process, so one run shares a backup dir; rsync only creates it when it
+actually backs something up. `--exclude=.twin-backup/` protects the backup
+dir from a `Path: "."` sync deleting it. For remote targets the backup dir
+is the path part of the target (it lives on the receiving side).
 `--itemize-changes` makes change detection deterministic: `Sync.transferred?`
 matches itemize lines (`/\A[<>ch*][fdLDS]/` — `>f…`, `cd…`, `*deleting`),
 covering files, directories and deletions, with no scraping of rsync's prose.

data/README.md

@@ -119,10 +119,58 @@ twin status                  # listing with source/target mtimes
 twin sync -p grubber         # sync one program by name pattern
 twin sync --file=repos       # sync all programs from a sync-file
 twin sync --dry-run          # preview without writing
+twin log                     # recent journal entries (-n N, --json)
 twin doctor                  # check tools, renderers, and sync targets
 twin --help                  # show usage
 ```
 
+### Adding a sync entry
+
+`twin add <path>` scaffolds a new entry interactively, so the judgment calls
+of setting up a sync become prompts with defaults:
+
+```
+$ twin add ~/.config/fish
+sync-file: home_macbook.md  (/Users/admin → /Volumes/macbook/Users/admin)
+Program name [fish]:
+Why is this synced? (one line of prose): Shell config incl. abbreviations.
+Description (short, for listings) [fish]: Fish Shell configuration
+Exclude (comma-separated) [.git/]:
+Mirror deletions on target (Delete: true)? (y/N) [n]:
+Post-sync Cmd (empty for none):
+
+added "fish" to home_macbook.md
+Run a dry-run now? (Y/n) [y]:
+```
+
+twin matches the path against the `Source:` roots of your sync-files (asking
+which to use when several match), derives `Path:` relative to that root,
+suggests excludes for what it finds in the directory (`.git/`,
+`node_modules/`, `.venv/`, …), refuses duplicates, and appends a Markdown
+block — prose stub included. If no sync-file covers the path, it offers to
+create one (frontmatter and all), which is also the quickest way to start
+syncing to a new SSH target.
+
+Every synced job is journaled to `~/.local/state/twin/log.jsonl` (one JSON
+line per job: timestamp, program, path, outcome). `twin log` shows the recent
+history; `twin sync` exits non-zero when any job failed.
+
+### Unattended syncs
+
+For a scheduled run (launchd, cron), combine two flags:
+
+```bash
+twin sync --quiet --skip-unavailable
+```
+
+`--quiet` prints only conflicts, errors, and jobs that actually changed
+something — a no-op run is silent. `--skip-unavailable` skips targets that
+are currently unmounted or unreachable instead of aborting, so a laptop that
+isn't docked doesn't turn into an error. Combined, the run produces output
+(and a non-zero exit) only when something genuinely needs attention, which is
+exactly what launchd's stdout/stderr logging wants; the journal still records
+every job.
+
 File argument resolution:
 
 - bare name (no `/`)  → looked up by substring in `sync_dir`
@@ -192,7 +240,55 @@ transferred bytes; no-op syncs skip it. See the Helix entry in
 
 The optional `Delete: true` field adds `--delete` to the rsync invocation,
 so files removed from the source are also removed on the target. Useful for
-directory syncs where the target should mirror the source exactly.
+directory syncs where the target should mirror the source exactly. As a
+safety net, deleted and overwritten files are moved to a per-run backup
+directory on the target (`<target>/.twin-backup/<timestamp>/`) instead of
+being destroyed — prune it occasionally.
+
+## SSH targets
+
+`Target:` accepts remote destinations in rsync notation — `user@host:/path` or
+`host:/path`. Everything else stays the same: the YAML block, excludes,
+`Delete:`, the `Cmd` hook.
+
+````markdown
+---
+Active: 1
+Label: mini → server
+Source: /Volumes/lightning/Git/Website
+Target: ralf@server:/srv/www
+---
+
+## Website
+
+Static site, deployed straight from the build directory.
+
+```yaml
+Program: website
+Path: public
+Description: static site
+Exclude: .git/
+Cmd: ssh ralf@server 'sudo systemctl reload caddy'
+```
+````
+
+Details:
+
+- **Push only.** `Source:` stays local; twin syncs *to* the remote host.
+- **Key-based auth required.** twin probes and stats hosts with
+  `ssh -o BatchMode=yes`, which never prompts for a password. Set up an SSH
+  key (`ssh-copy-id host`) first; `twin doctor` shows whether a host is
+  reachable.
+- **Status works remotely.** `twin status` and the picker stat all remote
+  paths of a host in a single ssh round-trip (macOS and Linux targets both
+  supported). An unreachable host shows as `?` instead of failing the scan.
+- **The mount check becomes a reachability check** — sync aborts if the host
+  doesn't answer.
+- **`Cmd` runs locally**, exactly as for mounted targets. To act on the
+  server, make the command an `ssh host '…'` call (see example above).
+- **`Render: true` is not supported** for remote targets (twin would have to
+  read and write remote file contents). Render locally or keep rendered files
+  on mounted targets.
 
 ## Templating
 
@@ -265,3 +361,7 @@ See [ARCHITECTURE.md](ARCHITECTURE.md) for the data model and internals.
 ```bash
 rake test
 ```
+
+---
+
+*Part of a family of plain-text tools — the [profile page](https://github.com/rhsev) has the map.*

data/bin/twin

@@ -1,5 +1,10 @@
 #!/usr/bin/env ruby
 
+# Sync-files are UTF-8 regardless of the environment's locale — without this,
+# a launchd/cron context (no LANG) reads them as US-ASCII and chokes on "→".
+Encoding.default_external = Encoding::UTF_8
+Encoding.default_internal = Encoding::UTF_8
+
 $LOAD_PATH.unshift File.expand_path("../lib", __dir__)
 
 require "twin"

data/lib/twin/add.rb

@@ -0,0 +1,177 @@
+require "yaml"
+
+require_relative "template"
+
+module Twin
+  # `twin add <path>` — guided scaffolding of a new sync entry. Turns the
+  # judgment calls of the "add a sync" recipe (which sync-file? which Path?
+  # what to exclude? deploy hook?) into prompts with sensible defaults, then
+  # appends a Markdown block to the chosen sync-file.
+  module Add
+    module_function
+
+    # Directories that are usually machine-generated or heavy — offered as
+    # exclude defaults when present in the source directory.
+    SUGGEST_EXCLUDES = %w[.git node_modules .venv __pycache__ dist build target].freeze
+
+    # ── pure helpers (unit-tested) ────────────────────────────────────────────
+
+    # Frontmatter of a sync-file as a Hash (Source/Target token-substituted),
+    # or nil when the file has none / it isn't a Hash.
+    def frontmatter(file, vars = {})
+      text = File.read(file)
+      return nil unless text.start_with?("---\n")
+      body = text[4..].split(/^---\s*$/, 2).first
+      data = begin
+        YAML.safe_load(body.to_s)
+      rescue Psych::SyntaxError
+        nil
+      end
+      return nil unless data.is_a?(Hash)
+      %w[Source Target].each do |k|
+        next unless data[k].is_a?(String)
+        data[k] = Twin::Template.substitute(data[k], vars, context: File.basename(file))
+      end
+      data
+    rescue Errno::ENOENT
+      nil
+    end
+
+    # Sync-files in dir whose Source is an ancestor of path.
+    # Returns [[file, frontmatter], …].
+    def candidates(dir, path, vars = {})
+      Dir.glob(File.join(dir, "*.md")).sort.filter_map do |f|
+        fm = frontmatter(f, vars)
+        next unless fm && fm["Source"].is_a?(String) && !fm["Source"].empty?
+        root = File.expand_path(fm["Source"])
+        next unless path == root || path.start_with?(root + "/")
+        [f, fm]
+      end
+    end
+
+    def relative_path(root, path)
+      path == root ? "." : path[(root.length + 1)..]
+    end
+
+    def suggest_excludes(path)
+      return [] unless File.directory?(path)
+      SUGGEST_EXCLUDES.filter_map do |e|
+        full = File.join(path, e)
+        next unless File.exist?(full)
+        File.directory?(full) ? "#{e}/" : e
+      end
+    end
+
+    def build_block(program:, path:, description: "", excludes: [], delete: false, cmd: "", prose: "")
+      yaml = ["Program: #{program}", "Path: #{path}"]
+      yaml << "Description: #{description}"        unless description.empty?
+      yaml << "Exclude: #{excludes.join(',')}"     unless excludes.empty?
+      yaml << "Delete: true"                       if delete
+      yaml << "Cmd: #{cmd}"                        unless cmd.empty?
+      prose = "TODO: document why this path is synced." if prose.empty?
+      "\n## #{program}\n\n#{prose}\n\n```yaml\n#{yaml.join("\n")}\n```\n"
+    end
+
+    def frontmatter_text(source:, target:, label: "")
+      lines = ["---", "Active: 1"]
+      lines << "Label: #{label}" unless label.empty?
+      lines << "Source: #{source}" << "Target: #{target}" << "---" << ""
+      lines.join("\n")
+    end
+
+    # ── interactive flow ──────────────────────────────────────────────────────
+
+    # Returns {program:, file:, dry_run:} on success (dry_run: whether the
+    # user asked for one), nil when nothing was written.
+    def run(cfg, args)
+      raw = args.first
+      raise "usage: twin add <path>" if raw.nil? || raw.empty?
+      path = File.expand_path(raw)
+      raise "not found: #{path}" unless File.exist?(path)
+
+      vars = cfg.var_map
+      picked = pick_sync_file(cfg, path, vars)
+      return nil unless picked
+      file, fm = picked
+
+      root = File.expand_path(fm["Source"])
+      rel  = relative_path(root, path)
+
+      if File.exist?(file) && File.read(file).match?(/^Path:\s*#{Regexp.escape(rel)}\s*$/)
+        raise "#{File.basename(file)} already has a block with Path: #{rel}"
+      end
+
+      program = ask("Program name", File.basename(path))
+      prose   = ask("Why is this synced? (one line of prose)")
+      desc    = ask("Description (short, for listings)", program)
+      excl    = ask("Exclude (comma-separated)", suggest_excludes(path).join(","))
+                  .split(",").map(&:strip).reject(&:empty?)
+      delete  = yes?(ask("Mirror deletions on target (Delete: true)? (y/N)", "n"))
+      cmd     = ask("Post-sync Cmd (empty for none)")
+
+      block = build_block(program: program, path: rel, description: desc,
+                          excludes: excl, delete: delete, cmd: cmd, prose: prose)
+      File.open(file, "a") { |f| f.write(block) }
+
+      puts "\nadded #{program.inspect} to #{File.basename(file)}"
+      puts "  #{File.join(root, rel)}  →  #{File.join(fm['Target'].to_s, rel)}"
+
+      dry = yes?(ask("Run a dry-run now? (Y/n)", "y"))
+      { program: program, file: file, dry_run: dry }
+    end
+
+    # Choose (or create) the sync-file covering path.
+    # Returns [file, frontmatter] or nil.
+    def pick_sync_file(cfg, path, vars)
+      cands = candidates(cfg.sync_dir, path, vars)
+      case cands.size
+      when 0 then offer_new_sync_file(cfg, path, vars)
+      when 1
+        file, fm = cands.first
+        puts "sync-file: #{File.basename(file)}  (#{fm['Source']} → #{fm['Target']})"
+        cands.first
+      else
+        puts "multiple sync-files cover #{path}:"
+        cands.each_with_index do |(f, fm), i|
+          puts "  #{i + 1}) #{File.basename(f)}  (#{fm['Source']} → #{fm['Target']})"
+        end
+        n = ask("Which one?", "1").to_i
+        cands[n - 1] or raise "invalid choice: #{n}"
+      end
+    end
+
+    def offer_new_sync_file(cfg, path, vars)
+      puts "no sync-file in #{cfg.sync_dir} covers #{path}"
+      return nil unless yes?(ask("Create a new sync-file? (y/N)", "n"))
+
+      name = ask("File name", "#{File.basename(path).sub(/\A\./, '')}.md")
+      name += ".md" unless name.end_with?(".md")
+      file = File.join(cfg.sync_dir, name)
+      raise "already exists: #{file}" if File.exist?(file)
+
+      source = ask("Source base on this machine", File.dirname(path))
+      target = ask("Target base (mount path or user@host:/path)")
+      raise "Target is required" if target.empty?
+      label  = ask("Label (e.g. mini → server)")
+
+      File.write(file, frontmatter_text(source: source, target: target, label: label))
+      puts "created #{File.basename(file)}"
+      fm = { "Source" => source, "Target" => target }
+      %w[Source Target].each do |k|
+        fm[k] = Twin::Template.substitute(fm[k], vars, context: name)
+      end
+      [file, fm]
+    end
+
+    def ask(prompt, default = "")
+      print default.empty? ? "#{prompt}: " : "#{prompt} [#{default}]: "
+      $stdout.flush
+      ans = $stdin.gets
+      raise "aborted (stdin closed)" if ans.nil?
+      ans = ans.strip
+      ans.empty? ? default : ans
+    end
+
+    def yes?(answer) = answer.match?(/\Ay/i)
+  end
+end

data/lib/twin/cli.rb

@@ -6,6 +6,8 @@ require "time"  # Time#iso8601 for --json output
 require_relative "config"
 require_relative "scanner"
 require_relative "sync"
+require_relative "journal"
+require_relative "add"
 require_relative "picker"
 
 module Twin
@@ -22,6 +24,9 @@ module Twin
         twin list   [--all] [--label X] [--file X] [--json]
         twin status [--all] [--label X] [--file X] [--json]
         twin sync   [-p PATTERN] [--label X] [--file X] [--all] [--dry-run]
+                    [--quiet] [--skip-unavailable]
+        twin add <path>            scaffold a new sync entry for a local path
+        twin log    [-n N] [--json]  recent journal entries (default 20)
         twin doctor                check tools, renderers, and sync targets
         twin --help                show this message
 
@@ -46,6 +51,8 @@ module Twin
       when "list"     then cmd_list(cfg, argv.drop(1))
       when "status"   then cmd_status(cfg, argv.drop(1))
       when "sync"     then cmd_sync(cfg, argv.drop(1))
+      when "add"      then cmd_add(cfg, argv.drop(1))
+      when "log"      then cmd_log(argv.drop(1))
       when "doctor"   then cmd_doctor(cfg)
       when "-h", "--help", "help"
         puts USAGE
@@ -137,6 +144,7 @@ module Twin
         p.jobs.each do |j|
           src = j.source_exists ? j.source_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
           tgt = j.target_exists ? j.target_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
+          tgt = "(unreachable)" if j.target_unreachable
           conflict = j.conflict ? (tty ? "  #{Picker.colorize(:target_newer, "!")}" : "  !") : ""
           puts "    #{j.path}#{conflict}"
           puts "      src #{src}"
@@ -156,31 +164,40 @@ module Twin
       end
 
       if programs.empty?
-        puts "no matching programs"
+        puts "no matching programs" unless opts[:quiet]
         return
       end
 
-      programs.each { |p| sync_program(cfg, p, dry_run: opts[:dry_run]) }
-    end
-
-    def sync_program(cfg, program, dry_run: false)
-      sync_jobs(cfg, program, program.active_jobs, dry_run: dry_run)
+      results = programs.map do |p|
+        sync_jobs(cfg, p, p.active_jobs,
+                  dry_run: opts[:dry_run], quiet: opts[:quiet],
+                  skip_unavailable: opts[:skip_unavailable])
+      end
+      exit 1 unless results.all?
     end
 
-    def sync_jobs(cfg, program, jobs, dry_run: false)
+    # Sync the given jobs. Returns true when every attempted job succeeded.
+    # quiet:            print only conflicts, errors, and jobs that changed something
+    # skip_unavailable: skip jobs whose target is unmounted/unreachable instead of aborting
+    def sync_jobs(cfg, program, jobs, dry_run: false, quiet: false, skip_unavailable: false)
       jobs = jobs.select { |j| j.active == 1 }
-      return if jobs.empty?
-
-      # one mount check per unique target root
-      checked = Set.new
-      jobs.each do |j|
-        next if checked.include?(j.target)
-        unless Twin::Sync.mounted?(j.target)
-          warn "abort: #{j.target} is not a mounted volume"
+      return true if jobs.empty?
+
+      # one availability check per unique target root:
+      # local targets must be mounted volumes, remote ones reachable via ssh
+      availability = {}
+      jobs.each { |j| availability[j.target] ||= target_availability(j) }
+      jobs, unavailable = jobs.partition { |j| availability[j.target] == :ok }
+
+      unavailable.map { |j| availability[j.target] }.uniq.each do |reason|
+        if skip_unavailable
+          puts "skipped: #{reason}" unless quiet
+        else
+          warn "abort: #{reason}"
           exit 1
         end
-        checked << j.target
       end
+      return true if jobs.empty?
 
       conflicts = jobs.select(&:conflict)
       unless conflicts.empty?
@@ -189,13 +206,74 @@ module Twin
         warn "continuing sync (--update skips newer files on target)."
       end
 
-      puts "→ #{program.name}"
+      header_printed = false
+      all_ok = true
       jobs.each do |job|
-        success, output, = Twin::Sync.run_job(cfg, job, dry_run: dry_run)
+        success, output, transferred = Twin::Sync.run_job(cfg, job, dry_run: dry_run)
+        Twin::Journal.record(job, success: success, transferred: transferred, output: output) unless dry_run
+        all_ok &&= success
+        next if quiet && success && !transferred
+
+        unless header_printed
+          puts "→ #{program.name}"
+          header_printed = true
+        end
         puts "  • #{job.path}"
         puts output.gsub(/^/, "    ") if output && !output.strip.empty?
         warn "  error syncing #{job.path}" unless success
       end
+      all_ok
+    end
+
+    # :ok, or a human-readable reason the target can't be synced right now.
+    def target_availability(job)
+      if job.remote?
+        host, = Twin::Remote.split(job.target)
+        return :ok if Twin::Remote.reachable?(host)
+        "#{host} is not reachable via ssh"
+      else
+        return :ok if Twin::Sync.mounted?(job.target)
+        "#{job.target} is not a mounted volume"
+      end
+    end
+
+    # ── add ────────────────────────────────────────────────────────────────────
+
+    def cmd_add(cfg, args)
+      result = Twin::Add.run(cfg, args)
+      return unless result && result[:dry_run]
+      cmd_sync(cfg, ["-p", result[:program],
+                     "--file=#{File.basename(result[:file])}", "--dry-run"])
+    end
+
+    # ── log ────────────────────────────────────────────────────────────────────
+
+    def cmd_log(args)
+      n = 20
+      json = false
+      OptionParser.new do |o|
+        o.on("-n N", Integer) { |v| n = v }
+        o.on("--json")        { json = true }
+      end.parse!(args)
+
+      entries = Twin::Journal.tail(n)
+      if json
+        puts JSON.pretty_generate(entries)
+        return
+      end
+      if entries.empty?
+        puts "journal is empty (#{Twin::Journal.log_path})"
+        return
+      end
+
+      tty = $stdout.tty?
+      entries.each do |e|
+        ts   = Time.parse(e["ts"]).strftime("%Y-%m-%d %H:%M:%S")
+        mark = e["ok"] ? "✓" : "✗"
+        mark = Picker.colorize(e["ok"] ? :in_sync : :both_missing, mark) if tty
+        note = e["ok"] ? (e["changed"] ? "changed" : "no-op") : "error: #{e["error"]}"
+        puts "#{ts}  #{mark}  #{e["program"]}  #{e["path"]}  (#{note})"
+      end
     end
 
     # ── doctor ─────────────────────────────────────────────────────────────────
@@ -252,7 +330,15 @@ module Twin
           puts "  (no programs loaded)"
         else
           targets.each do |tgt|
-            if Twin::Sync.mounted?(tgt)
+            if Twin::Remote.remote?(tgt)
+              host, = Twin::Remote.split(tgt)
+              if Twin::Remote.reachable?(host)
+                puts "  ✓  #{tgt}  (ssh)"
+              else
+                puts "  ✗  #{tgt}  (ssh: #{host} not reachable)"
+                ok = false
+              end
+            elsif Twin::Sync.mounted?(tgt)
               puts "  ✓  #{tgt}"
             else
               puts "  ✗  #{tgt}  (not mounted)"
@@ -287,13 +373,16 @@ module Twin
     end
 
     def parse_sync_opts(args)
-      opts = { show_all: false, label: nil, file: nil, pattern: nil, dry_run: false }
+      opts = { show_all: false, label: nil, file: nil, pattern: nil, dry_run: false,
+               quiet: false, skip_unavailable: false }
       OptionParser.new do |o|
         o.on("--all")          { opts[:show_all] = true }
         o.on("--label=L")      { |v| opts[:label] = v }
         o.on("--file=F")       { |v| opts[:file] = v }
         o.on("-p", "--pattern=P") { |v| opts[:pattern] = v }
         o.on("--dry-run")      { opts[:dry_run] = true }
+        o.on("-q", "--quiet")  { opts[:quiet] = true }
+        o.on("--skip-unavailable") { opts[:skip_unavailable] = true }
       end.parse!(args)
       opts
     end

data/lib/twin/journal.rb

@@ -0,0 +1,49 @@
+require "json"
+require "fileutils"
+require "time"
+
+module Twin
+  # Append-only sync journal: one JSON line per synced job in
+  # ~/.local/state/twin/log.jsonl. Answers "did yesterday's sync actually
+  # run, and what did it do?" — and stays machine-readable (jq/grubber).
+  # Journal failures never break a sync; they degrade to a warning.
+  module Journal
+    module_function
+
+    def state_dir
+      ENV["TWIN_STATE_DIR"] || File.join(Dir.home, ".local", "state", "twin")
+    end
+
+    def log_path = File.join(state_dir, "log.jsonl")
+
+    # Record one job result. Dry-runs are not journaled.
+    def record(job, success:, transferred:, output: nil)
+      entry = {
+        ts:      Time.now.iso8601,
+        program: job.program,
+        path:    job.path,
+        target:  job.target,
+        ok:      success,
+        changed: transferred,
+      }
+      unless success
+        entry[:error] = output.to_s.lines.map(&:strip).reject(&:empty?).last.to_s[0, 200]
+      end
+      FileUtils.mkdir_p(state_dir)
+      File.open(log_path, "a") { |f| f.puts(JSON.generate(entry)) }
+    rescue SystemCallError => e
+      warn "journal: #{e.message}" unless @warned
+      @warned = true
+    end
+
+    # Last n entries, oldest first. Unparseable lines are skipped.
+    def tail(n)
+      return [] unless File.exist?(log_path)
+      File.readlines(log_path).last(n).filter_map do |line|
+        JSON.parse(line)
+      rescue JSON::ParserError
+        nil
+      end
+    end
+  end
+end

data/lib/twin/picker.rb

@@ -17,6 +17,7 @@ module Twin
       missing_target: "!",
       missing_source: "!",
       both_missing:   "✗",
+      unreachable:    "?",
       disabled:       "·",
     }.freeze
 
@@ -27,6 +28,7 @@ module Twin
       missing_target: "\e[31m",   # red
       missing_source: "\e[31m",   # red
       both_missing:   "\e[31m",   # red
+      unreachable:    "\e[31m",   # red
       disabled:       "\e[2m",    # dim
     }.freeze
 

data/lib/twin/remote.rb

@@ -0,0 +1,72 @@
+require "open3"
+
+module Twin
+  # Remote (ssh) targets, written exactly as rsync understands them:
+  # "user@host:/path" or "host:/path". A target counts as remote when a colon
+  # appears before the first slash. Sources stay local — twin pushes.
+  module Remote
+    module_function
+
+    SSH_OPTS = ["-o", "BatchMode=yes", "-o", "ConnectTimeout=5"].freeze
+
+    def remote?(target)
+      %r{\A[^/]+:}.match?(target.to_s)
+    end
+
+    # "user@host:/path" → ["user@host", "/path"]
+    def split(target)
+      host, path = target.to_s.split(":", 2)
+      [host, path.to_s]
+    end
+
+    # Non-interactive reachability probe (BatchMode: never asks for a password).
+    def reachable?(host)
+      system("ssh", *SSH_OPTS, host, "true", out: File::NULL, err: File::NULL)
+    end
+
+    # Stat many paths in one ssh round-trip. Paths go over stdin (one per
+    # line), the remote loop answers "path<TAB>epoch" or "path<TAB>-" for
+    # missing ones. Tries BSD stat first, then GNU — covers macOS and Linux.
+    # Returns {path => Time or nil-if-missing}, or nil when ssh itself failed.
+    STAT_SCRIPT = <<~SH.freeze
+      while IFS= read -r p; do
+        if [ -e "$p" ]; then
+          printf '%s\t%s\n' "$p" "$(stat -f %m -- "$p" 2>/dev/null || stat -c %Y -- "$p")"
+        else
+          printf '%s\t-\n' "$p"
+        fi
+      done
+    SH
+
+    def stat_paths(host, paths)
+      return {} if paths.empty?
+      out, _err, status = Open3.capture3(
+        "ssh", *SSH_OPTS, host, STAT_SCRIPT,
+        stdin_data: paths.join("\n") + "\n"
+      )
+      return nil unless status.success?
+
+      result = {}
+      out.each_line do |line|
+        path, mtime = line.chomp.split("\t", 2)
+        next unless path && mtime
+        result[path] = mtime == "-" ? nil : Time.at(mtime.to_i)
+      end
+      result
+    rescue Errno::ENOENT
+      nil # ssh not installed
+    end
+
+    # Create a directory on the remote side (mkdir -p equivalent).
+    def mkdir_p(host, dir)
+      _out, _err, status = Open3.capture3("ssh", *SSH_OPTS, host, "mkdir", "-p", shellesc(dir))
+      status.success?
+    end
+
+    # Escape one argument for the remote shell (ssh joins args with spaces and
+    # hands the string to a shell — local exec-style arrays don't protect it).
+    def shellesc(s)
+      "'" + s.gsub("'", "'\\\\''") + "'"
+    end
+  end
+end

data/lib/twin/scanner.rb

@@ -1,19 +1,24 @@
 require "json"
 require "open3"
 
+require_relative "remote"
+
 module Twin
   # One YAML block from a sync-file, enriched with live filesystem state.
   Job = Struct.new(
     :program, :path, :description, :active, :excludes, :label,
     :source, :target, :cmd, :delete, :render, :render_outdated, :target_path_field, :sync_file,
     :source_exists, :target_exists, :source_mtime, :target_mtime, :conflict,
+    :target_unreachable,
     keyword_init: true,
   ) do
     def source_path = File.join(source, path)
     def target_path = File.join(target, target_path_field || path)
+    def remote?     = Twin::Remote.remote?(target)
 
     def status
       return :disabled if active != 1
+      return :unreachable if target_unreachable
       return :both_missing if !source_exists && !target_exists
       return :missing_source unless source_exists
       return :missing_target unless target_exists
@@ -41,7 +46,7 @@ module Twin
     # Aggregate status across jobs — worst first.
     def status
       states = jobs.map(&:status)
-      %i[both_missing missing_source missing_target target_newer source_newer disabled in_sync]
+      %i[unreachable both_missing missing_source missing_target target_newer source_newer disabled in_sync]
         .find { |s| states.include?(s) } || :in_sync
     end
 
@@ -69,10 +74,35 @@ module Twin
         raise "grubber returned invalid JSON: #{e.message}"
       end
       vars = cfg.var_map
-      records.filter_map do |r|
+      jobs = records.filter_map do |r|
         context = "#{r["Program"]} in #{File.basename(r["_note_file"].to_s)}"
         build_job(Twin::Template.substitute_record(r, vars, context: context), vars)
       end
+      fill_remote_stats(jobs)
+      jobs
+    end
+
+    # Remote targets can't be stat'ed locally — batch them into one ssh
+    # round-trip per host. A failed ssh marks the jobs unreachable instead
+    # of aborting the scan (local jobs stay usable).
+    def fill_remote_stats(jobs)
+      jobs.select { |j| j.remote? && j.active == 1 }
+          .group_by { |j| Twin::Remote.split(j.target).first }
+          .each do |host, host_jobs|
+        stats = Twin::Remote.stat_paths(host, host_jobs.map { |j| Twin::Remote.split(j.target_path).last })
+        host_jobs.each do |j|
+          rpath = Twin::Remote.split(j.target_path).last
+          if stats.nil?
+            j.target_unreachable = true
+            next
+          end
+          mtime = stats[rpath]
+          j.target_exists = !mtime.nil?
+          j.target_mtime  = mtime
+          j.conflict      = j.source_exists && mtime && j.source_mtime &&
+                            mtime - j.source_mtime >= 60
+        end
+      end
     end
 
     def load_programs(cfg, file: nil, label: nil, show_all: false)
@@ -114,11 +144,18 @@ module Twin
 
       render   = r["Render"] == true
       excludes = (r["Exclude"] || "").split(",").map(&:strip).reject(&:empty?)
+      remote   = Twin::Remote.remote?(target)
+
+      if render && remote
+        raise "#{r["Program"]}: Render is not supported for remote targets (#{target})"
+      end
 
       src_full = File.join(source, path)
       tgt_full = File.join(target, target_path_field || path)
       src_exists, src_mtime = stat(src_full)
-      tgt_exists, tgt_mtime = stat(tgt_full)
+      # Remote targets are stat'ed in one batched ssh call after all jobs are
+      # built (fill_remote_stats) — until then they read as missing.
+      tgt_exists, tgt_mtime = remote ? [false, nil] : stat(tgt_full)
 
       # Render jobs: status is content-based (mtime is meaningless for a rendered
       # target). conflict stays false so the mtime conflict-warning skips them.
@@ -148,6 +185,7 @@ module Twin
         source_mtime:     src_mtime,
         target_mtime:     tgt_mtime,
         conflict:         !!conflict,
+        target_unreachable: false,
       )
     end
 

data/lib/twin/sync.rb

@@ -1,5 +1,7 @@
 require "fileutils"
 
+require_relative "remote"
+
 module Twin
   module Sync
     module_function
@@ -35,6 +37,8 @@ module Twin
     # Render a template Job: read source, substitute {{vars}}, write if changed.
     # Returns [success, output, changed].
     def render_job(cfg, job, dry_run: false)
+      return [false, "render: remote targets are not supported", false] if job.remote?
+
       src = job.source_path
       tgt = job.target_path
 
@@ -89,21 +93,16 @@ module Twin
 
       return [false, "source not found: #{src}", false] unless File.exist?(src)
 
-      FileUtils.mkdir_p(File.dirname(tgt))
-
-      args = ["rsync", "-av", "--itemize-changes", "--update"]
-      args << "--delete" if job.delete
-      args << "--dry-run" if dry_run
-      cfg.global_excludes.each { |ex| args << "--exclude=#{ex}" }
-      job.excludes.each       { |ex| args << "--exclude=#{ex}" }
-
-      if File.directory?(src)
-        args << "#{src}/" << "#{tgt}/"
+      if job.remote?
+        host, rpath = Twin::Remote.split(tgt)
+        unless dry_run || Twin::Remote.mkdir_p(host, File.dirname(rpath))
+          return [false, "ssh: could not create #{File.dirname(rpath)} on #{host}", false]
+        end
       else
-        args << src << tgt
+        FileUtils.mkdir_p(File.dirname(tgt))
       end
 
-      output, status = run(args)
+      output, status = run(rsync_args(cfg, job, dry_run: dry_run))
       return [false, output, false] unless status.success?
 
       xfr = !dry_run && transferred?(output)
@@ -128,6 +127,44 @@ module Twin
       [true, output, xfr]
     end
 
+    # Full rsync argument vector for a job.
+    def rsync_args(cfg, job, dry_run: false)
+      src = job.source_path
+      tgt = job.target_path
+
+      args = ["rsync", "-av", "--itemize-changes", "--update"]
+      if job.delete
+        args << "--delete"
+        args.concat(backup_args(job))
+      end
+      args << "--dry-run" if dry_run
+      cfg.global_excludes.each { |ex| args << "--exclude=#{ex}" }
+      job.excludes.each       { |ex| args << "--exclude=#{ex}" }
+
+      if File.directory?(src)
+        args << "#{src}/" << "#{tgt}/"
+      else
+        args << src << tgt
+      end
+      args
+    end
+
+    # Safety net for --delete: deleted and overwritten files land in a
+    # per-run backup dir on the target side (<target>/.twin-backup/<stamp>).
+    # rsync only creates the dir when it actually backs something up.
+    # The exclude keeps a backup dir inside the transfer root (Path: ".")
+    # from being deleted by the very sync it protects against.
+    def backup_args(job)
+      root = job.remote? ? Twin::Remote.split(job.target).last : job.target
+      dir  = File.join(root, ".twin-backup", run_stamp)
+      ["--backup", "--backup-dir=#{dir}", "--exclude=.twin-backup/"]
+    end
+
+    # One timestamp per twin process, so a multi-job run shares a backup dir.
+    def run_stamp
+      @run_stamp ||= Time.now.strftime("%Y-%m-%d_%H%M%S")
+    end
+
     # Sync all jobs in a Program. Returns array of [job, success, output].
     def run_program(cfg, program, dry_run: false)
       program.active_jobs.map { |job| [job, *run_job(cfg, job, dry_run: dry_run)] }

data/lib/twin/version.rb

@@ -1,3 +1,3 @@
 module Twin
-  VERSION = "0.2.0"
+  VERSION = "0.3.0"
 end

data/lib/twin.rb

@@ -1,8 +1,11 @@
 require_relative "twin/version"
+require_relative "twin/remote"
 require_relative "twin/template"
 require_relative "twin/config"
 require_relative "twin/scanner"
 require_relative "twin/sync"
+require_relative "twin/journal"
+require_relative "twin/add"
 require_relative "twin/preview"
 require_relative "twin/picker"
 require_relative "twin/cli"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mark-twin
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Ralf Hülsmann
@@ -37,10 +37,13 @@ files:
 - README.md
 - bin/twin
 - lib/twin.rb
+- lib/twin/add.rb
 - lib/twin/cli.rb
 - lib/twin/config.rb
+- lib/twin/journal.rb
 - lib/twin/picker.rb
 - lib/twin/preview.rb
+- lib/twin/remote.rb
 - lib/twin/scanner.rb
 - lib/twin/sync.rb
 - lib/twin/template.rb