mark-twin 0.4.2 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 07baefa770954e6be785cd591c727fb287b7a64dcf84d4c83eb3c747978da7ba
-  data.tar.gz: d29597f8f7b20380b658bd9127c5a72f904f8baea2e3dd20756a7431317e53bd
+  metadata.gz: d54022c14550637480eed5ecea2c7cf5b391d75724559c5c25cdac8cef134526
+  data.tar.gz: c5fc90db5a080109b60f330f591cb62055412ca3cf59f7d68d89e156b6d17f85
 SHA512:
-  metadata.gz: 43c56ce8ed70a445dd3dabf803e68d25651cafe7b4a04e90f4202680592ac447345ab0f55e0dbd09970004d553c84868ae6aec6a643f48cd71f5d2583e0c20a4
-  data.tar.gz: 25043aaa4ddf2f30ed8b546f52e68dccd008493414a9c8447d44c2c36acc45d32e375977b0d8b927623d2e5d6e937f0b2b56ed28cb8be53c2eeedeea08c73871
+  metadata.gz: 4511a3d7293d69be4cabd081c772ac252a0123f4207cabd007da52186b3a4cc0f7edff487b8530484ab083ecab0ced150060c3912c4605021689ad3fc9e5caf1
+  data.tar.gz: c4968f1c11bebaa8b7b883f9271e5b55495a8bde93f6557d8a5e2842bac5c263c8e859c0a74e35ee4b119f3d75056170a0016d4f3516804c1926d136709a62f8

data/ARCHITECTURE.md

@@ -54,17 +54,25 @@ test/test_pure.rb
 ```
 program, path, description, active, excludes, owned, label, source, target, cmd,
 delete, render, render_outdated, target_path_field, sync_file,
-source_exists, target_exists, source_mtime, target_mtime, conflict
+source_exists, target_exists, source_mtime, target_mtime, conflict,
+directory, content_equal, drift
 ```
 
 `excludes` and `owned` both become `--exclude` (via `Job#all_excludes`); they
-are kept apart so `status` can report intent. `conflict` is the coarse mtime
-flag used by `status` only — conflict *resolution* ignores it and asks rsync
-directly, because a directory's mtime says nothing about edits inside it.
+are kept apart so `status` can report intent.
+
+mtime is never the verdict, only a pre-filter for file jobs — and even there
+identical bytes under a drifted timestamp clear it (`content_equal`; remote
+targets get one batched md5 round per host). `conflict` is therefore
+content-verified when set. Directory jobs get no mtime judgment at all: a
+directory's mtime moves on every sync and on every excluded file, so
+`Job#status` reports `unverified` until `twin status` fills `drift` by asking
+rsync (`Conflict.drift` — paired dry-runs, itemize classification, checksums
+for timestamp-only candidates).
 
 `Job#status` → one of `disabled / unreachable / both_missing / missing_source /
-missing_target / target_newer / in_sync / source_newer`. Render jobs derive status from content
-(`render_outdated`), not mtime; non-render jobs ignore those fields.
+missing_target / target_newer / in_sync / source_newer / unverified`. Render
+jobs derive status from content (`render_outdated`), not mtime.
 `Job#target_path` joins `target` with `target_path_field || path`.
 
 **Program** — group of Jobs sharing a `program` name:

data/README.md

@@ -5,7 +5,7 @@
 [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
 
 Sync configuration between Macs (or to any host you can reach over ssh) from
-self-documenting Markdown files.
+self-documenting Markdown files. Designed for informed, interactive syncing.
 
 A sync-file is a normal Markdown document. The prose is for you, or for an AI
 assistant. twin only uses the fenced YAML blocks:
@@ -30,8 +30,11 @@ Own: conf.d/local.fish
 ```
 ````
 
-`twin` shows you what differs and syncs what you choose. At its core, it's just
-`rsync`, but a year later the file still tells you *why* you did it that way.
+twin runs when you ask, not in the background. It syncs what is unambiguous;
+where the target has changes of its own, it stops and asks, diff in hand
+(verified against content, never guessed from timestamps). At its core it's
+just `rsync`, but a year later the file still tells you *why* you did it that
+way.
 
 That is the whole idea, and for most entries it stays as small as the excerpt
 above. The rest of this README is long because twin also covers more complex
@@ -62,7 +65,7 @@ paths underneath:
 Stage 2 — multi-select over the paths of one program. The right pane shows a
 compact preview of the relevant sync-file section, rendered by apex:
 
-![Stage 2 — Fish Shell paths with apex preview](https://raw.githubusercontent.com/rhsev/mark-twin/main/docs/stage_2_fish.png)
+![Stage 2 — a rendered LaunchAgent entry with apex preview](https://raw.githubusercontent.com/rhsev/mark-twin/main/docs/stage_2_livesync.png)
 
 ## Install
 
@@ -144,13 +147,13 @@ twin                         # picker — all programs across all sync-files
 twin home.md                 # picker — one sync-file in sync_dir (by name)
 twin ./some/dir/             # picker — all sync-files in a directory
 twin list                    # plain listing
-twin status                  # listing with source/target mtimes
+twin status                  # what a sync would change, content-verified
 twin sync -p grubber         # sync one program by name pattern
 twin sync --file=repos       # sync all programs from a sync-file
 twin sync --dry-run          # preview without writing
 twin sync -v                 # rsync's full output instead of just the changes
 twin log                     # recent journal entries (-n N, --json)
-twin doctor                  # check tools, renderers, and targets
+twin doctor                  # check tools, renderers, targets, remote hosts
 twin --version               # which twin is actually running
 twin --help                  # show usage
 ```
@@ -265,8 +268,18 @@ pruning occasionally. It applies to `Delete` jobs only.
 ## Mounted volume or ssh
 
 Both are first-class. `twin status`, the picker, `Exclude`/`Own`, `Delete` and
-`Cmd` behave identically; remote paths are stat'ed in a single ssh round-trip
-per host, and an unreachable host shows as `?` instead of failing the scan.
+`Cmd` behave identically; remote paths are stat'ed (and, where timestamps
+disagree, checksummed via `md5`/`md5sum`) in batched ssh round-trips per host,
+and an unreachable host shows as `?` instead of failing the scan.
+
+The remote side needs `rsync` and `/bin/sh`; `stat` (or `date -r`) and
+`md5`/`md5sum` improve status from there, and `twin doctor` probes a host for
+all of them before the first sync fails halfway. No particular login shell is
+required — twin drives the far side through `/bin/sh` explicitly. That matters
+more than it sounds: until 0.4.5 the stat script went to whatever shell the
+account uses, so a host with fish (or any other non-POSIX shell) reported
+every path as unreachable. It looked exactly like a machine being switched
+off, which is why it went unnoticed for a while.
 
 Two differences are real:
 
@@ -361,8 +374,11 @@ Two properties make this bearable day to day:
   directory's mtime untouched, and `rsync -a` equalises those anyway. Twin asks
   rsync what it would actually transfer instead of guessing from a directory.
 
-`twin status` is still mtime-based and cannot see an in-place edit. It is the
-cheap overview; `twin sync` is what decides.
+`twin status` gives the same verdict without syncing: it runs the dry-runs per
+entry and reports what would flow, what merely differs in timestamp, and what
+changed on the target. Only the picker stays mtime-blind for directory entries
+— it marks them `∘` (unverified) rather than guessing, because the dry-runs
+would make it slow to open.
 
 ## Automation
 

data/lib/twin/cli.rb

@@ -28,7 +28,7 @@ module Twin
                     [--force] [--skip-conflicts]
         twin add <path>            scaffold a new sync entry for a local path
         twin log    [-n N] [--json]  recent journal entries (default 20)
-        twin doctor                check tools, renderers, and sync targets
+        twin doctor                check tools, renderers, targets, remote hosts
         twin --help                show this message
         twin --version             show the running version
 
@@ -150,6 +150,7 @@ module Twin
     def cmd_status(cfg, args)
       opts = parse_filter_opts(args)
       programs = Scanner.load_programs(cfg, **opts.slice(:file, :label, :show_all))
+      verify_drift(cfg, programs)
 
       if opts[:json]
         puts JSON.pretty_generate(programs.map { |p| program_to_hash(p) })
@@ -163,19 +164,68 @@ module Twin
         name = tty ? Picker.bold(p.name) : p.name
         puts "#{icon}  #{name}"
         p.jobs.each do |j|
-          src = j.source_exists ? j.source_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
-          tgt = j.target_exists ? j.target_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
-          tgt = "(unreachable)" if j.target_unreachable
           conflict = j.conflict ? (tty ? "  #{Picker.colorize(:target_newer, "!")}" : "  !") : ""
           puts "    #{j.path}#{conflict}"
-          puts "      src #{src}"
-          puts "      dst #{tgt}"
+          if j.directory
+            # Directory mtimes prove nothing — the drift verdict replaces them.
+            puts "      #{j.drift ? drift_summary(j.drift) : "(not checked — target unavailable)"}"
+          else
+            src = j.source_exists ? j.source_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
+            tgt = j.target_exists ? j.target_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
+            tgt = "(unreachable)" if j.target_unreachable
+            puts "      src #{src}"
+            puts "      dst #{tgt}"
+            puts "      content identical, timestamps differ" if j.content_equal
+          end
           # Named, not hidden: these belong to the target on purpose.
           puts "      own #{j.owned.join(', ')}" unless j.owned.nil? || j.owned.empty?
         end
       end
     end
 
+    # Ask rsync what a sync of each directory job would actually do — the
+    # dry-runs are subprocess I/O, so a few run in parallel. File jobs are
+    # already content-checked by the scanner and need no second look.
+    def verify_drift(cfg, programs)
+      jobs = programs.flat_map(&:jobs).select do |j|
+        j.directory && j.active == 1 && !j.target_unreachable &&
+          j.source_exists && j.target_exists
+      end
+      return if jobs.empty?
+
+      queue = Queue.new
+      jobs.each { |j| queue << j }
+      Array.new([4, jobs.size].min) do
+        Thread.new do
+          loop do
+            j = begin
+              queue.pop(true)
+            rescue ThreadError
+              break
+            end
+            j.drift = Twin::Conflict.drift(cfg, j)
+          end
+        end
+      end.each(&:join)
+    end
+
+    def drift_summary(d)
+      if d.in_sync?
+        note = d.time_only.empty? ? "" : " (#{d.time_only.size} timestamp-only)"
+        return "in sync#{note}"
+      end
+      parts = []
+      parts << "#{d.pending.size} to sync: #{list_some(d.pending)}" unless d.pending.empty?
+      unless d.conflicts.empty?
+        parts << "#{d.conflicts.size} changed on target: #{list_some(d.conflicts.map(&:rel))} (twin sync will ask)"
+      end
+      parts.join("; ")
+    end
+
+    def list_some(rels, max = 3)
+      rels.take(max).join(", ") + (rels.size > max ? ", …" : "")
+    end
+
     # ── sync ───────────────────────────────────────────────────────────────────
 
     def cmd_sync(cfg, args)
@@ -423,11 +473,13 @@ module Twin
         if targets.empty?
           puts "  (no programs loaded)"
         else
+          reachable_hosts = []
           targets.each do |tgt|
             if Twin::Remote.remote?(tgt)
               host, = Twin::Remote.split(tgt)
               if Twin::Remote.reachable?(host)
                 puts "  ✓  #{tgt}  (ssh)"
+                reachable_hosts << host
               else
                 puts "  ✗  #{tgt}  (ssh: #{host} not reachable)"
                 ok = false
@@ -439,6 +491,7 @@ module Twin
               ok = false
             end
           end
+          ok = doctor_remote_tools(reachable_hosts.uniq) && ok
         end
       rescue => e
         puts "  ✗  #{e.message}"
@@ -453,6 +506,42 @@ module Twin
       system("command -v #{name} > /dev/null 2>&1")
     end
 
+    # Probe each reachable ssh host once for what the far side must provide.
+    # A missing rsync fails doctor — the first sync would die mid-run with a
+    # raw protocol error. Missing stat/date or md5/md5sum only degrade
+    # (unknown mtimes, conservative conflicts), so they warn and say so.
+    def doctor_remote_tools(hosts)
+      return true if hosts.empty?
+      ok = true
+      puts "\nRemote tools"
+      hosts.each do |host|
+        tools = Twin::Remote.preflight(host)
+        if tools.nil?
+          puts "  ✗  #{host}  (probe failed)"
+          ok = false
+          next
+        end
+        problems = []
+        unless tools["rsync"]
+          problems << "rsync missing — syncs will fail (OpenWrt: opkg install rsync)"
+          ok = false
+        end
+        unless tools["stat"] || tools["date"]
+          problems << "no stat or date — remote mtimes read as unknown"
+        end
+        unless tools["md5"] || tools["md5sum"]
+          problems << "no md5 or md5sum — timestamp-only files stay flagged as conflicts"
+        end
+        if problems.empty?
+          have = ["rsync", tools["stat"] ? "stat" : "date", tools["md5"] ? "md5" : "md5sum"]
+          puts "  ✓  #{host}  (#{have.join(', ')})"
+        else
+          puts "  #{tools["rsync"] ? "⚠" : "✗"}  #{host}  #{problems.join('; ')}"
+        end
+      end
+      ok
+    end
+
     # ── option parsing ─────────────────────────────────────────────────────────
 
     def parse_filter_opts(args)
@@ -501,6 +590,11 @@ module Twin
       h[:status]       = j.status
       h[:source_mtime] = j.source_mtime&.iso8601
       h[:target_mtime] = j.target_mtime&.iso8601
+      h[:drift]        = j.drift && {
+        pending:   j.drift.pending,
+        time_only: j.drift.time_only,
+        conflicts: j.drift.conflicts.map(&:rel),
+      }
       h
     end
   end

data/lib/twin/conflict.rb

@@ -1,40 +1,43 @@
 require "digest"
+require "set"
 
 require_relative "remote"
 
 module Twin
-  # Finding out which files on the target would be silently skipped by rsync's
-  # --update, and whether that actually matters.
+  # What would a sync actually change? mtimes cannot answer that — a
+  # directory's mtime records the last entry added or removed (after a sync:
+  # the sync itself), a file's mtime moves on a `cat >` copy that changed
+  # nothing, and rsync equalises directory mtimes on every run anyway. So we
+  # ask rsync, which has the answer already and knows its own matching rules
+  # better than any reimplementation would:
   #
-  # `Job#conflict` is no help here. It compares the mtime of the job's own path,
-  # and for a directory job that is the directory's mtime — which says nothing
-  # about the files inside it. Worse, it is wrong in exactly the case that
-  # matters: editing a file in place leaves its directory's mtime untouched, and
-  # `rsync -a` equalises directory mtimes on every run anyway. A hand-edit on
-  # the target is therefore invisible to it.
+  #   1. What would a *forced* run transfer?
+  #      One dry-run without --update. Empty means fully in sync — the common
+  #      case and the cheap exit: one stat-walk per job and no more.
   #
-  # So we ask rsync, which has the answer already and knows its own matching
-  # rules better than any reimplementation would:
+  #   2. Of that, what does --update hold back?
+  #      A second dry-run with --update. Everything the forced run would move
+  #      but the normal one would not is exactly the set --update protects —
+  #      files the target owns more recently.
   #
-  #   1. Which files does --update hold back?
-  #      Dry-run twice, once with --update and once without. Everything the
-  #      second run would transfer but the first would not is exactly the set
-  #      --update protects.
-  #
-  #   2. Of those, which differ in content?
-  #      Only these are worth asking about. A file that is merely newer — same
-  #      bytes, later timestamp — is noise, and noise is what turns a prompt
-  #      into a reflex. Sync both sides of a tree in either order and you get
-  #      dozens of them.
-  #
-  # The first dry-run is also the cheap exit: when a forced run would move
-  # nothing, the job is fully in sync and the second run is skipped. That is the
-  # common case, so a quiet sync costs one extra stat-walk per job and no more.
+  #   3. What differs only in timestamp?
+  #      rsync's itemize flags say so (">f..t" — time, same size). Those files
+  #      are checksummed; identical content is noise, not drift, and noise is
+  #      what turns a prompt into a reflex. Sync both sides of a tree in
+  #      either order and you get dozens of them.
   module Conflict
-    # One file the target owns more recently than the source, with content that
-    # actually differs. `same_content` is nil when it could not be determined
-    # (remote targets) — treated as a conflict, because guessing in the other
-    # direction would overwrite work.
+    # The classified outcome of a forced dry-run for one job.
+    #   pending   — relative paths a sync would genuinely change
+    #   time_only — same bytes, different timestamp; a sync merely aligns them
+    #   conflicts — Entry list: target-side changes whose content differs
+    Drift = Struct.new(:pending, :time_only, :conflicts, keyword_init: true) do
+      def in_sync? = pending.empty? && conflicts.empty?
+    end
+
+    # One file the target owns more recently than the source, with content
+    # that actually differs (or could not be checked — remote md5 failed —
+    # which is treated as differing, because guessing the other way would
+    # overwrite work). target_mtime is nil for remote targets.
     Entry = Struct.new(:job, :rel, :source_path, :target_path,
                        :source_mtime, :target_mtime, keyword_init: true) do
       def age_delta
@@ -43,55 +46,113 @@ module Twin
       end
     end
 
-    # rsync --itemize-changes line → relative path. Change lines start with an
-    # update type and a file type (">f.st...... lib/foo.rb"); "*deleting" and
-    # the surrounding prose do not match.
-    ITEMIZE_LINE = /\A[<>ch][fdLDS]\S*\s+(.+?)\s*\z/
+    # rsync --itemize-changes line → [flags, relative path]. Change lines
+    # start with an update type and a file type (">f.st...... lib/foo.rb") or
+    # "*deleting"; attribute-only lines (leading ".") and the surrounding
+    # prose do not match — they describe no change a sync would make.
+    ENTRY_LINE = /\A(\*deleting|[<>ch][fdLDS]\S*)\s+(.+?)\s*\z/
 
     module_function
 
-    # Real conflicts for one job, in the order rsync reports them.
-    # Empty when --update holds nothing back, or holds back only identical files.
-    def detect(cfg, job)
-      # Render jobs compare by content already and never use --update.
-      return [] if job.render
-      return [] unless job.source_exists && job.target_exists
+    # Classify one job's drift by asking rsync. nil for render jobs (they
+    # compare content already and never use --update) and when a side is
+    # missing (status reports that on its own).
+    def drift(cfg, job)
+      return nil if job.render
+      return nil unless job.source_exists && job.target_exists
+
+      forced = itemized(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: true))
+      return Drift.new(pending: [], time_only: [], conflicts: []) if forced.empty?
+
+      normal_rels = itemized(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: false))
+                    .map { |e| e[:rel] }.to_set
+      equal = equality_map(job, forced.filter_map { |e| e[:rel] if e[:kind] == :time })
 
-      held_back_paths(cfg, job).filter_map { |rel| entry_for(job, rel) }
+      assemble(forced, normal_rels, equal) do |rel|
+        conflict_entry(job, rel)
+      end
     end
 
-    # Relative paths that --update would skip: (would transfer forced) minus
-    # (would transfer normally).
-    def held_back_paths(cfg, job)
-      forced = itemized_paths(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: true))
-      return [] if forced.empty?   # nothing to move at all — no need to ask rsync twice
+    # Target-side changes worth asking about, in the order rsync reports them.
+    # Empty when --update holds nothing back, or holds back only identical files.
+    def detect(cfg, job)
+      drift(cfg, job)&.conflicts || []
+    end
 
-      normal = itemized_paths(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: false))
-      forced - normal
+    # Pure assembly of a Drift from parsed entries. A file the normal run
+    # would also transfer flows source→target as intended; one only the
+    # forced run would touch is being held back by --update — the target owns
+    # it more recently. Content decides whether that is a conflict or noise.
+    def assemble(forced, normal_rels, equal)
+      d = Drift.new(pending: [], time_only: [], conflicts: [])
+      forced.each do |e|
+        rel = e[:rel]
+        case e[:kind]
+        when :deleted, :new then d.pending << rel
+        when :content
+          normal_rels.include?(rel) ? d.pending << rel : d.conflicts << yield(rel)
+        when :time
+          if equal[rel]
+            d.time_only << rel
+          elsif normal_rels.include?(rel)
+            d.pending << rel
+          else
+            d.conflicts << yield(rel)
+          end
+        end
+      end
+      d
     end
 
-    def itemized_paths(args)
+    # Run rsync and parse its itemize output into [{rel:, kind:}, ...].
+    def itemized(args)
       output, status = Twin::Sync.run(args)
       return [] unless status.success?
       output.lines.filter_map do |line|
-        m = ITEMIZE_LINE.match(line)
+        m = ENTRY_LINE.match(line)
         next unless m
-        rel = m[1]
-        next if rel == "./" || rel.end_with?("/")   # directories carry no content
-        rel
+        kind = classify(m[1])
+        next if kind == :attrs
+        { rel: m[2], kind: kind }
       end
     end
 
-    # Build an Entry unless source and target hold the same bytes.
-    def entry_for(job, rel)
-      src = resolve(job.source_path, rel)
-      tgt = resolve(job.target_path, rel)
+    # Itemize flags → what kind of change this is.
+    #   :deleted — target-only file, removed by Delete: true
+    #   :new     — does not exist on the target yet (files and directories)
+    #   :content — size differs, so the bytes certainly do
+    #   :time    — timestamp only; content equality still to be determined
+    #   :attrs   — permissions/owner, no change a sync-file cares about
+    def classify(flags)
+      return :deleted if flags == "*deleting"
+      body = flags[2..].to_s
+      return :new     if body.include?("+")
+      return :content if body.include?("s")
+      return :time    if body.include?("t") || body.include?("T")
+      :attrs
+    end
 
-      # Remote targets can't be read here; report them rather than assume.
-      unless job.remote?
-        return nil if same_content?(src, tgt)
+    # Content equality for the :time candidates, {rel => bool}. Local pairs
+    # are compared directly; remote targets get one batched md5 round per job
+    # (STAT_SCRIPT-style), and an unanswered path counts as differing.
+    def equality_map(job, rels)
+      return {} if rels.empty?
+      if job.remote?
+        _host, rbase = Twin::Remote.split(job.target_path)
+        remote_paths = rels.to_h { |r| [r, job.directory ? File.join(rbase, r) : rbase] }
+        sums = Twin::Remote.md5_paths(Twin::Remote.split(job.target)[0], remote_paths.values) || {}
+        rels.to_h do |r|
+          local = local_md5(resolve(job.source_path, r))
+          [r, !local.nil? && sums[remote_paths[r]] == local]
+        end
+      else
+        rels.to_h { |r| [r, same_content?(resolve(job.source_path, r), resolve(job.target_path, r))] }
       end
+    end
 
+    def conflict_entry(job, rel)
+      src = resolve(job.source_path, rel)
+      tgt = job.remote? ? job.target_path : resolve(job.target_path, rel)
       Entry.new(
         job: job, rel: rel, source_path: src, target_path: tgt,
         source_mtime: mtime(src), target_mtime: job.remote? ? nil : mtime(tgt),
@@ -114,6 +175,12 @@ module Twin
 
     def digest(path) = Digest::SHA256.file(path).hexdigest
 
+    def local_md5(path)
+      Digest::MD5.file(path).hexdigest
+    rescue Errno::ENOENT, Errno::EACCES, Errno::EISDIR
+      nil
+    end
+
     def mtime(path)
       File.mtime(path)
     rescue Errno::ENOENT, Errno::EACCES

data/lib/twin/picker.rb

@@ -14,6 +14,10 @@ module Twin
       source_newer:   "→",
       target_newer:   "←",
       in_sync:        "✓",
+      # Directory jobs in the picker: their mtimes prove nothing, and the
+      # dry-run that would prove something is too slow for a picker to open
+      # with. No claim instead of a wrong one — `twin status` has the verdict.
+      unverified:     "∘",
       missing_target: "!",
       missing_source: "!",
       both_missing:   "✗",
@@ -25,6 +29,7 @@ module Twin
       source_newer:   "\e[33m",   # yellow
       target_newer:   "\e[36m",   # cyan
       in_sync:        "\e[32m",   # green
+      unverified:     "\e[2m",    # dim
       missing_target: "\e[31m",   # red
       missing_source: "\e[31m",   # red
       both_missing:   "\e[31m",   # red
@@ -91,8 +96,7 @@ module Twin
 
       rows = jobs.each_with_index.map do |j, i|
         icon  = STATUS_ICONS[j.status] || "?"
-        delta = format_delta(j.source_mtime, j.target_mtime)
-        line  = "#{icon}  #{j.path.ljust(path_width)}  #{delta}"
+        line  = "#{icon}  #{j.path.ljust(path_width)}  #{job_delta(j)}"
         "#{i}\t#{colorize(j.status, line)}"
       end
 
@@ -186,14 +190,19 @@ module Twin
 
       body = program.jobs.map do |j|
         icon  = STATUS_ICONS[j.status] || "?"
-        delta = format_delta(j.source_mtime, j.target_mtime)
-        line  = "    #{icon}  #{j.path.ljust(path_width)}  #{delta}"
+        line  = "    #{icon}  #{j.path.ljust(path_width)}  #{job_delta(j)}"
         colorize(j.status, line)
       end
 
       ([header] + body).join("\n")
     end
 
+    # A directory's mtime delta would mislead (it moves on every sync), so
+    # directory jobs show none.
+    def job_delta(job)
+      job.directory ? "" : format_delta(job.source_mtime, job.target_mtime)
+    end
+
     def format_delta(sm, tm)
       return "" if sm.nil? || tm.nil?
       seconds = (sm - tm).to_i

data/lib/twin/remote.rb

@@ -26,22 +26,40 @@ module Twin
 
     # Stat many paths in one ssh round-trip. Paths go over stdin (one per
     # line), the remote loop answers "path<TAB>epoch" or "path<TAB>-" for
-    # missing ones. Tries BSD stat first, then GNU — covers macOS and Linux.
+    # missing ones. Tries BSD stat, then GNU, then BusyBox `date -r` — covers
+    # macOS, Linux and OpenWrt. Auf dem Brume (25.08.2026) fehlte `stat`
+    # komplett: die leere Substitution wurde als Epoche 0 geparst und `twin
+    # status` zeigte 1970 statt "unbekannt". `date -r` kennt kein `--`;
+    # verkraftbar, weil hier nur absolute Zielpfade ankommen.
     # Returns {path => Time or nil-if-missing}, or nil when ssh itself failed.
+    # POSIX sh, and it must stay free of single quotes: it is handed to the
+    # remote side wrapped in '...' so that *any* login shell passes it through
+    # literally. The usual POSIX escape for an embedded quote ('\'') is parsed
+    # differently by fish, so the rule here is simply not to need it — hence
+    # double quotes around the printf formats.
     STAT_SCRIPT = <<~SH.freeze
       while IFS= read -r p; do
         if [ -e "$p" ]; then
-          printf '%s\t%s\n' "$p" "$(stat -f %m -- "$p" 2>/dev/null || stat -c %Y -- "$p")"
+          m=$(stat -f %m -- "$p" 2>/dev/null || stat -c %Y -- "$p" 2>/dev/null || date -r "$p" +%s)
+          printf "%s\t%s\n" "$p" "$m"
         else
-          printf '%s\t-\n' "$p"
+          printf "%s\t-\n" "$p"
         fi
       done
     SH
 
     def stat_paths(host, paths)
       return {} if paths.empty?
+      # Explicitly through /bin/sh. ssh hands the command to the *login shell*
+      # on the far side, and that is not necessarily POSIX: on a machine whose
+      # shell is fish, the bare script dies on "while IFS= read -r p; do" and
+      # twin read the failure as "host not answering" — every ssh target showed
+      # as :unreachable in status and the picker, for as long as SSH targets
+      # existed (#8, 2026-07-17). Found and fixed 2026-08-25.
+      raise "STAT_SCRIPT must not contain single quotes" if STAT_SCRIPT.include?("'")
+
       out, _err, status = Open3.capture3(
-        "ssh", *SSH_OPTS, host, STAT_SCRIPT,
+        "ssh", *SSH_OPTS, host, "/bin/sh -c '#{STAT_SCRIPT}'",
         stdin_data: paths.join("\n") + "\n"
       )
       return nil unless status.success?
@@ -50,13 +68,90 @@ module Twin
       out.each_line do |line|
         path, mtime = line.chomp.split("\t", 2)
         next unless path && mtime
-        result[path] = mtime == "-" ? nil : Time.at(mtime.to_i)
+        # Nur echte Epochen als Zeit werten. Scheitert die ganze stat-Kette,
+        # ist das Feld leer — das ist "unbekannt", nicht 1970.
+        result[path] = mtime.match?(/\A\d+\z/) ? Time.at(mtime.to_i) : nil
       end
       result
     rescue Errno::ENOENT
       nil # ssh not installed
     end
 
+    # Checksum many paths in one ssh round-trip, same shape as stat_paths:
+    # paths over stdin, "path<TAB>md5" back, "-" for anything that is not a
+    # regular file. Tries BSD md5 first, then md5sum (GNU, BusyBox) — covers
+    # macOS, Linux and OpenWrt. md5sum prints "hash  path"; the parameter
+    # expansion keeps only the first word. Same single-quote rule as
+    # STAT_SCRIPT, and MD5 is drift detection here, not cryptography.
+    # Returns {path => hex or nil-if-unreadable}, or nil when ssh itself failed.
+    MD5_SCRIPT = <<~SH.freeze
+      while IFS= read -r p; do
+        if [ -f "$p" ]; then
+          m=$(md5 -q "$p" 2>/dev/null || md5sum "$p" 2>/dev/null)
+          m=${m%% *}
+          printf "%s\t%s\n" "$p" "$m"
+        else
+          printf "%s\t-\n" "$p"
+        fi
+      done
+    SH
+
+    def md5_paths(host, paths)
+      return {} if paths.empty?
+      raise "MD5_SCRIPT must not contain single quotes" if MD5_SCRIPT.include?("'")
+
+      out, _err, status = Open3.capture3(
+        "ssh", *SSH_OPTS, host, "/bin/sh -c '#{MD5_SCRIPT}'",
+        stdin_data: paths.join("\n") + "\n"
+      )
+      return nil unless status.success?
+
+      result = {}
+      out.each_line do |line|
+        path, sum = line.chomp.split("\t", 2)
+        next unless path && sum
+        result[path] = sum.match?(/\A\h{32}\z/) ? sum.downcase : nil
+      end
+      result
+    rescue Errno::ENOENT
+      nil # ssh not installed
+    end
+
+    # Which of the tools twin relies on exist on the far side? One ssh
+    # round-trip per host, for `twin doctor`. rsync carries the sync itself;
+    # stat/date feed the batched mtime round; md5/md5sum feed the content
+    # check. Same single-quote rule as the other batch scripts.
+    # Returns {tool => present?}, or nil when ssh itself failed.
+    PREFLIGHT_SCRIPT = <<~SH.freeze
+      for t in rsync stat date md5 md5sum; do
+        if command -v "$t" >/dev/null 2>&1; then
+          printf "%s\tok\n" "$t"
+        else
+          printf "%s\t-\n" "$t"
+        fi
+      done
+    SH
+
+    def preflight(host)
+      raise "PREFLIGHT_SCRIPT must not contain single quotes" if PREFLIGHT_SCRIPT.include?("'")
+
+      out, _err, status = Open3.capture3("ssh", *SSH_OPTS, host, "/bin/sh -c '#{PREFLIGHT_SCRIPT}'")
+      return nil unless status.success?
+      parse_preflight(out)
+    rescue Errno::ENOENT
+      nil # ssh not installed
+    end
+
+    def parse_preflight(out)
+      result = {}
+      out.each_line do |line|
+        tool, state = line.chomp.split("\t", 2)
+        next unless tool && state
+        result[tool] = state == "ok"
+      end
+      result
+    end
+
     # Create a directory on the remote side (mkdir -p equivalent).
     def mkdir_p(host, dir)
       _out, _err, status = Open3.capture3("ssh", *SSH_OPTS, host, "mkdir", "-p", shellesc(dir))

data/lib/twin/scanner.rb

@@ -2,6 +2,7 @@ require "json"
 require "open3"
 
 require_relative "remote"
+require_relative "conflict"
 
 module Twin
   # One YAML block from a sync-file, enriched with live filesystem state.
@@ -9,7 +10,7 @@ module Twin
     :program, :path, :description, :active, :excludes, :owned, :label,
     :source, :target, :cmd, :delete, :render, :render_outdated, :target_path_field, :sync_file,
     :source_exists, :target_exists, :source_mtime, :target_mtime, :conflict,
-    :target_unreachable,
+    :target_unreachable, :directory, :content_equal, :drift,
     keyword_init: true,
   ) do
     def source_path = File.join(source, path)
@@ -29,6 +30,19 @@ module Twin
       # Render jobs compare by content, not mtime — a rendered target's mtime
       # bears no relation to the template's.
       return render_outdated ? :source_newer : :in_sync if render
+      # Directory jobs carry no mtime verdict at all: a directory's mtime
+      # records the last entry added or removed — after a sync that is the
+      # sync itself, and even an *excluded* file moves it. `twin status`
+      # fills in `drift` by asking rsync; without it the honest answer is
+      # "not checked", not a guess.
+      if directory
+        return :unverified unless drift
+        return :target_newer if drift.conflicts.any?
+        return :source_newer if drift.pending.any?
+        return :in_sync
+      end
+      # Same bytes under a newer timestamp (a `cat >` copy) is not drift.
+      return :in_sync if content_equal
       return :target_newer if conflict
       return :in_sync if source_mtime.nil? || target_mtime.nil?
       delta = source_mtime - target_mtime
@@ -50,7 +64,7 @@ module Twin
     # Aggregate status across jobs — worst first.
     def status
       states = jobs.map(&:status)
-      %i[unreachable both_missing missing_source missing_target target_newer source_newer disabled in_sync]
+      %i[unreachable both_missing missing_source missing_target target_newer source_newer unverified disabled in_sync]
         .find { |s| states.include?(s) } || :in_sync
     end
 
@@ -103,9 +117,31 @@ module Twin
           mtime = stats[rpath]
           j.target_exists = !mtime.nil?
           j.target_mtime  = mtime
-          j.conflict      = j.source_exists && mtime && j.source_mtime &&
+          j.conflict      = !j.directory && j.source_exists && mtime && j.source_mtime &&
                             mtime - j.source_mtime >= 60
         end
+        verify_remote_file_content(host, host_jobs)
+      end
+    end
+
+    # Remote counterpart of the local content check in build_job: file jobs
+    # whose mtimes drifted get one batched md5 round per host. Identical
+    # content clears the conflict — the timestamps merely disagree.
+    def verify_remote_file_content(host, host_jobs)
+      candidates = host_jobs.select do |j|
+        !j.directory && !j.render && j.source_exists && j.target_exists &&
+          j.source_mtime && j.target_mtime && (j.target_mtime - j.source_mtime).abs >= 60
+      end
+      return if candidates.empty?
+
+      sums = Twin::Remote.md5_paths(host, candidates.map { |j| Twin::Remote.split(j.target_path).last })
+      return if sums.nil?
+
+      candidates.each do |j|
+        remote_sum = sums[Twin::Remote.split(j.target_path).last]
+        next if remote_sum.nil?
+        j.content_equal = remote_sum == Twin::Conflict.local_md5(j.source_path)
+        j.conflict      = false if j.content_equal
       end
     end
 
@@ -170,8 +206,22 @@ module Twin
       # target). conflict stays false so the mtime conflict-warning skips them.
       render_outdated = render ? render_outdated?(src_full, tgt_full, vars, path) : nil
 
+      # rsync mirrors directories, so a directory source means a directory
+      # target — also for remote jobs, whose far side can't be inspected here.
+      directory = src_exists && File.directory?(src_full)
+
+      # A file job whose mtimes drifted apart may still hold the same bytes
+      # (a `cat >` copy before the first twin run). Check before judging;
+      # a directory's own mtime is judged not at all (see Job#status).
+      content_equal = nil
+      if !render && !remote && !directory && src_exists && tgt_exists &&
+         src_mtime && tgt_mtime && (tgt_mtime - src_mtime).abs >= 60
+        content_equal = Twin::Conflict.same_content?(src_full, tgt_full)
+      end
+
       # Same 60s tolerance as Job#status, so mtime jitter never flags a conflict.
-      conflict = !render && src_exists && tgt_exists && tgt_mtime && src_mtime &&
+      conflict = !render && !directory && !content_equal &&
+                 src_exists && tgt_exists && tgt_mtime && src_mtime &&
                  tgt_mtime - src_mtime >= 60
 
       Job.new(
@@ -196,6 +246,8 @@ module Twin
         target_mtime:     tgt_mtime,
         conflict:         !!conflict,
         target_unreachable: false,
+        directory:        directory,
+        content_equal:    content_equal,
       )
     end
 

data/lib/twin/sync.rb

@@ -130,6 +130,9 @@ module Twin
 
       xfr = !dry_run && transferred?(output)
 
+      # conflict is content-verified by the scanner and only ever set on file
+      # jobs — a directory's mtime says nothing, so directory jobs surface
+      # target-side changes through the pre-run prompt instead.
       if job.conflict && !xfr && !dry_run && !force
         output += "\nskipped: target is newer, source not synced"
       end

data/lib/twin/version.rb

@@ -1,3 +1,3 @@
 module Twin
-  VERSION = "0.4.2"
+  VERSION = "0.5.0"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mark-twin
 version: !ruby/object:Gem::Version
-  version: 0.4.2
+  version: 0.5.0
 platform: ruby
 authors:
 - Ralf Hülsmann
@@ -24,19 +24,14 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '5.0'
 description: |
-  Keeping two machines' configuration aligned usually ends in a shell script
-  nobody dares touch, because the reasons behind it were never written down.
+  twin helps keep two machines' configuration aligned.
 
-  twin puts the reasons and the instructions in the same file. A sync-file is
-  ordinary Markdown: the prose explains why a path is synced, and fenced YAML
-  blocks say what to do. rsync does the copying.
+  A sync-file is just Markdown: the prose says why a path is synced, and
+  fenced YAML blocks define what to do. rsync does the copying.
 
-  It knows the awkward parts. Paths the other machine owns stay untouched and
-  are named as such, not buried among build artefacts. When both sides changed
-  since the last sync, twin shows a diff and asks before overwriting anything,
-  once for the whole group, so a half-applied deploy is not a state it can
-  leave you in. Targets are a local path, a mounted volume, or user@host:/path
-  over ssh, with no difference in how they are used.
+  When both sides changed since the last sync, twin shows a diff and asks
+  before overwriting. A target is a local path, a mounted volume, or
+  user@host:/path over ssh, and twin treats all three the same.
 
   Interactive selection runs through fzf with a rendered Markdown preview; the
   same sync-files drive scriptable status, dry-run and sync commands.