mark-twin 0.4.3 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8f9146653ecf4288e9bd05297300344cc68b0606291d2bbc61c73f13aa067706
4
- data.tar.gz: 30e5c66372ae4511a550ed522701505463586b72eaa7ddb79cf2b266b5d47681
3
+ metadata.gz: 9e643b14b2fe0dd153539ff16d10a0950f3c478d327620dec1b9b04d7af4f340
4
+ data.tar.gz: 4eb24c0280ee5f1085024c1357a6d7b8dd0b7f16198e3cce8b4c9ce5849705b5
5
5
  SHA512:
6
- metadata.gz: a4d702bfe0443951977d6db35e61697140a97d3c12c278a6eebcb58665e8aa35a1e3eb646c9fbdbac0187ca8368dc161a24ab9db12de0ac698c2e23a92f0a01a
7
- data.tar.gz: bb56c6892a1b93b8d6108cbfcbdd82b7021cd89116a59b896d755646c699a805c5a08e73825d5e2297d20056de1b1f04f0ae8d3035006d34a2a8169cff3af5bf
6
+ metadata.gz: 7d5567dc0637bc767f7ddff5716a2e5031902e787958e1ecd69f54931e737b2946d82c999637d6610d732e5063898c3b717b49148f8e495af328ea2f0b716d03
7
+ data.tar.gz: 94df8bd72ea5548e668b1173e17b9e9bf54efc1fbca4576c109ed64740794decb94eb23fab78c50ed2b0654bc2160bb1a4be85c74a2f9281bd2a178e0f33b270
data/ARCHITECTURE.md CHANGED
@@ -53,18 +53,29 @@ test/test_pure.rb
53
53
 
54
54
  ```
55
55
  program, path, description, active, excludes, owned, label, source, target, cmd,
56
- delete, render, render_outdated, target_path_field, sync_file,
57
- source_exists, target_exists, source_mtime, target_mtime, conflict
56
+ delete, render, render_outdated, target_path_field, sync_file, verify,
57
+ source_exists, target_exists, source_mtime, target_mtime, conflict,
58
+ directory, content_equal, drift
58
59
  ```
59
60
 
60
61
  `excludes` and `owned` both become `--exclude` (via `Job#all_excludes`); they
61
- are kept apart so `status` can report intent. `conflict` is the coarse mtime
62
- flag used by `status` only — conflict *resolution* ignores it and asks rsync
63
- directly, because a directory's mtime says nothing about edits inside it.
62
+ are kept apart so `status` can report intent.
63
+
64
+ mtime is never the verdict, only a pre-filter for file jobs and even there
65
+ identical bytes under a drifted timestamp clear it (`content_equal`; remote
66
+ targets get one batched md5 round per host). `conflict` is therefore
67
+ content-verified when set. Directory jobs get no mtime judgment at all: a
68
+ directory's mtime moves on every sync and on every excluded file, so
69
+ `Job#status` reports `unverified` until `twin status` fills `drift` by asking
70
+ rsync (`Conflict.drift` — paired dry-runs, itemize classification, checksums
71
+ for timestamp-only candidates). `Verify: false` opts a job out of every
72
+ content round — scanner md5, drift dry-runs, pre-sync conflict detection —
73
+ for entries where the walk itself is the cost; its status stays mtime-based
74
+ (files) or `unverified` (directories).
64
75
 
65
76
  `Job#status` → one of `disabled / unreachable / both_missing / missing_source /
66
- missing_target / target_newer / in_sync / source_newer`. Render jobs derive status from content
67
- (`render_outdated`), not mtime; non-render jobs ignore those fields.
77
+ missing_target / target_newer / in_sync / source_newer / unverified`. Render
78
+ jobs derive status from content (`render_outdated`), not mtime.
68
79
  `Job#target_path` joins `target` with `target_path_field || path`.
69
80
 
70
81
  **Program** — group of Jobs sharing a `program` name:
@@ -156,16 +167,28 @@ rationale.
156
167
  Two stages:
157
168
 
158
169
  1. **Stage 1 — program picker.** Multi-line NUL-separated entries (`--read0`).
159
- Each entry has a header (icon, program name, job count, sync-file) and
170
+ Each entry has a header (icon, program name, job count, sync-files) and
160
171
  indented body lines (one per job). No preview. Single-select. ESC exits.
172
+ Same-named programs (case-insensitive) from different sync-files are merged
173
+ into one entry, and a trailing bracket group joins by convention:
174
+ `livesync [agent]` merges under `livesync`. A picker-only view
175
+ (`Picker.merge_programs`); the data model, `status`, `sync -p` and JSON
176
+ keep the per-file programs. Jobs stay grouped per file in document order,
177
+ so a file's closing `Cmd` block still fires after its own paths.
161
178
  2. **Stage 2 — path multi-picker.** Tab-delimited rows (`id\tdisplay`),
162
- `--with-nth=2` hides the `id`. Multi-select via Tab. Preview pane shows
163
- the *compact* view (frontmatter + intro + the heading section containing
164
- the YAML block for the highlighted path), rendered via
179
+ `--with-nth=2` hides the `id`. Multi-select via Tab. On open, directory
180
+ jobs get their drift verified (`Conflict.fill_drift` affordable for one
181
+ program, unlike for the whole of stage 1). A merged program's rows are
182
+ sectioned per sync-file by dim header rows whose id is `-`; they map to no
183
+ job, so toggling one is inert. Preview pane shows the *compact* view
184
+ (frontmatter + intro + the heading section containing the YAML block for
185
+ the highlighted path) from the job's own sync-file, rendered via
165
186
  `apex --plugins -t terminal256`. ESC returns to Stage 1.
166
187
 
167
188
  Compact previews are pre-rendered to per-job tempfiles before fzf launches.
168
- An `awk` lookup maps `{1}` (the id) → tempfile path.
189
+ An `awk` lookup maps `{1}` (the id) → tempfile path. The Tempfile objects
190
+ stay referenced while fzf runs — a GC'd Tempfile unlinks its file under the
191
+ running preview.
169
192
 
170
193
  ## CLI
171
194
 
data/README.md CHANGED
@@ -5,7 +5,7 @@
5
5
  [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
6
6
 
7
7
  Sync configuration between Macs (or to any host you can reach over ssh) from
8
- self-documenting Markdown files.
8
+ self-documenting Markdown files. Designed for informed, interactive syncing.
9
9
 
10
10
  A sync-file is a normal Markdown document. The prose is for you, or for an AI
11
11
  assistant. twin only uses the fenced YAML blocks:
@@ -30,8 +30,11 @@ Own: conf.d/local.fish
30
30
  ```
31
31
  ````
32
32
 
33
- `twin` shows you what differs and syncs what you choose. At its core, it's just
34
- `rsync`, but a year later the file still tells you *why* you did it that way.
33
+ twin runs when you ask, not in the background. It syncs what is unambiguous;
34
+ where the target has changes of its own, it stops and asks, diff in hand
35
+ (verified against content, never guessed from timestamps). At its core it's
36
+ just `rsync`, but a year later the file still tells you *why* you did it that
37
+ way.
35
38
 
36
39
  That is the whole idea, and for most entries it stays as small as the excerpt
37
40
  above. The rest of this README is long because twin also covers more complex
@@ -55,14 +58,17 @@ on both sides since the last sync, and paths that differ per host.
55
58
  ## Screenshots
56
59
 
57
60
  Stage 1 — program picker. One row per program, color-coded status, indented
58
- paths underneath:
61
+ paths underneath. A name that appears in several sync-files
62
+ (the app in one, its config in another) is one entry, and a trailing bracket
63
+ group ties variants to their base name — `livesync [agent]` lists under
64
+ `livesync`. Stage 2 sections the merged paths per origin:
59
65
 
60
66
  ![Stage 1 — program picker](https://raw.githubusercontent.com/rhsev/mark-twin/main/docs/stage_1.png)
61
67
 
62
68
  Stage 2 — multi-select over the paths of one program. The right pane shows a
63
69
  compact preview of the relevant sync-file section, rendered by apex:
64
70
 
65
- ![Stage 2 — Fish Shell paths with apex preview](https://raw.githubusercontent.com/rhsev/mark-twin/main/docs/stage_2_fish.png)
71
+ ![Stage 2 — a rendered LaunchAgent entry with apex preview](https://raw.githubusercontent.com/rhsev/mark-twin/main/docs/stage_2_livesync.png)
66
72
 
67
73
  ## Install
68
74
 
@@ -144,13 +150,13 @@ twin # picker — all programs across all sync-files
144
150
  twin home.md # picker — one sync-file in sync_dir (by name)
145
151
  twin ./some/dir/ # picker — all sync-files in a directory
146
152
  twin list # plain listing
147
- twin status # listing with source/target mtimes
153
+ twin status # what a sync would change, content-verified
148
154
  twin sync -p grubber # sync one program by name pattern
149
155
  twin sync --file=repos # sync all programs from a sync-file
150
156
  twin sync --dry-run # preview without writing
151
157
  twin sync -v # rsync's full output instead of just the changes
152
158
  twin log # recent journal entries (-n N, --json)
153
- twin doctor # check tools, renderers, and targets
159
+ twin doctor # check tools, renderers, targets, remote hosts
154
160
  twin --version # which twin is actually running
155
161
  twin --help # show usage
156
162
  ```
@@ -210,7 +216,7 @@ rather than as an error.
210
216
 
211
217
  | Field | Where | Meaning |
212
218
  |---|---|---|
213
- | `Program` | block | Group name; blocks sharing it sync together |
219
+ | `Program` | block | Group name; blocks sharing it sync together. A trailing `[…]` lists under the base name in the picker |
214
220
  | `Path` | block | Path relative to `Source` (file or directory) |
215
221
  | `Source` | either | Absolute base path on this machine |
216
222
  | `Target` | either | Absolute base path, or `user@host:/path` for ssh |
@@ -223,6 +229,7 @@ rather than as an error.
223
229
  | `Delete` | block | `true` mirrors deletions, with backups |
224
230
  | `Cmd` | block | Shell command, run only when bytes actually moved |
225
231
  | `Render` | block | `true` substitutes `{{tokens}}` instead of copying |
232
+ | `Verify` | block | `false` skips content verification — for entries too big (see below) |
226
233
 
227
234
  ### Exclude or Own?
228
235
 
@@ -265,8 +272,18 @@ pruning occasionally. It applies to `Delete` jobs only.
265
272
  ## Mounted volume or ssh
266
273
 
267
274
  Both are first-class. `twin status`, the picker, `Exclude`/`Own`, `Delete` and
268
- `Cmd` behave identically; remote paths are stat'ed in a single ssh round-trip
269
- per host, and an unreachable host shows as `?` instead of failing the scan.
275
+ `Cmd` behave identically; remote paths are stat'ed (and, where timestamps
276
+ disagree, checksummed via `md5`/`md5sum`) in batched ssh round-trips per host,
277
+ and an unreachable host shows as `?` instead of failing the scan.
278
+
279
+ The remote side needs `rsync` and `/bin/sh`; `stat` (or `date -r`) and
280
+ `md5`/`md5sum` improve status from there, and `twin doctor` probes a host for
281
+ all of them before the first sync fails halfway. No particular login shell is
282
+ required — twin drives the far side through `/bin/sh` explicitly. That matters
283
+ more than it sounds: until 0.4.5 the stat script went to whatever shell the
284
+ account uses, so a host with fish (or any other non-POSIX shell) reported
285
+ every path as unreachable. It looked exactly like a machine being switched
286
+ off, which is why it went unnoticed for a while.
270
287
 
271
288
  Two differences are real:
272
289
 
@@ -361,8 +378,21 @@ Two properties make this bearable day to day:
361
378
  directory's mtime untouched, and `rsync -a` equalises those anyway. Twin asks
362
379
  rsync what it would actually transfer instead of guessing from a directory.
363
380
 
364
- `twin status` is still mtime-based and cannot see an in-place edit. It is the
365
- cheap overview; `twin sync` is what decides.
381
+ `twin status` gives the same verdict without syncing: it runs the dry-runs per
382
+ entry and reports what would flow, what merely differs in timestamp, and what
383
+ changed on the target. The picker's first stage stays mtime-blind for
384
+ directory entries — it marks them `∘` (unverified) rather than guessing,
385
+ because the dry-runs would make it slow to open. Entering a program runs them
386
+ for just that program, so its directory rows show a real verdict.
387
+
388
+ One escape hatch: `Verify: false` opts an entry out of every content round —
389
+ the md5 checks, the status dry-runs, the picker's verification, the pre-sync
390
+ conflict listing. It exists for entries where the walk itself is the cost: a
391
+ `node_modules` tree on an SMB mount stats tens of thousands of files for one
392
+ verdict. Such an entry stays `∘`/mtime-based, and the picker's second stage
393
+ says so in its header rather than letting the `∘` pass as pending; a sync
394
+ still leaves newer target files alone (`--update` holds), they just aren't
395
+ itemised first.
366
396
 
367
397
  ## Automation
368
398
 
@@ -427,7 +457,7 @@ it differs from the current target, so a `Cmd` hook fires only on a real change.
427
457
  ## LiveSync LaunchAgent
428
458
 
429
459
  ```yaml
430
- Program: livesync-agent
460
+ Program: livesync [agent]
431
461
  Source: "{{src.home}}/Automation/launchd"
432
462
  Path: com.ralf.livesync.plist
433
463
  Target: "{{dst.mount}}"
data/lib/twin/cli.rb CHANGED
@@ -28,7 +28,7 @@ module Twin
28
28
  [--force] [--skip-conflicts]
29
29
  twin add <path> scaffold a new sync entry for a local path
30
30
  twin log [-n N] [--json] recent journal entries (default 20)
31
- twin doctor check tools, renderers, and sync targets
31
+ twin doctor check tools, renderers, targets, remote hosts
32
32
  twin --help show this message
33
33
  twin --version show the running version
34
34
 
@@ -93,7 +93,7 @@ module Twin
93
93
  exit 1
94
94
  end
95
95
 
96
- programs = Scanner.load_programs(cfg, file: file, show_all: false)
96
+ programs = Picker.merge_programs(Scanner.load_programs(cfg, file: file, show_all: false))
97
97
  if programs.empty?
98
98
  warn "no active programs found#{" in #{file}" if file}"
99
99
  return
@@ -125,7 +125,7 @@ module Twin
125
125
  break if $stdin.gets&.strip == "q"
126
126
 
127
127
  # reload so status reflects what was just synced, stay on this program
128
- programs = Scanner.load_programs(cfg, file: file, show_all: false)
128
+ programs = Picker.merge_programs(Scanner.load_programs(cfg, file: file, show_all: false))
129
129
  selected_key = [program.name, program.sync_file]
130
130
  end
131
131
  end
@@ -150,6 +150,7 @@ module Twin
150
150
  def cmd_status(cfg, args)
151
151
  opts = parse_filter_opts(args)
152
152
  programs = Scanner.load_programs(cfg, **opts.slice(:file, :label, :show_all))
153
+ verify_drift(cfg, programs)
153
154
 
154
155
  if opts[:json]
155
156
  puts JSON.pretty_generate(programs.map { |p| program_to_hash(p) })
@@ -163,19 +164,51 @@ module Twin
163
164
  name = tty ? Picker.bold(p.name) : p.name
164
165
  puts "#{icon} #{name}"
165
166
  p.jobs.each do |j|
166
- src = j.source_exists ? j.source_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
167
- tgt = j.target_exists ? j.target_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
168
- tgt = "(unreachable)" if j.target_unreachable
169
167
  conflict = j.conflict ? (tty ? " #{Picker.colorize(:target_newer, "!")}" : " !") : ""
170
168
  puts " #{j.path}#{conflict}"
171
- puts " src #{src}"
172
- puts " dst #{tgt}"
169
+ if j.directory
170
+ # Directory mtimes prove nothing — the drift verdict replaces them.
171
+ note = if j.drift then drift_summary(j.drift)
172
+ elsif j.verify == false then "(not checked — Verify: false)"
173
+ else "(not checked — target unavailable)"
174
+ end
175
+ puts " #{note}"
176
+ else
177
+ src = j.source_exists ? j.source_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
178
+ tgt = j.target_exists ? j.target_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
179
+ tgt = "(unreachable)" if j.target_unreachable
180
+ puts " src #{src}"
181
+ puts " dst #{tgt}"
182
+ puts " content identical, timestamps differ" if j.content_equal
183
+ end
173
184
  # Named, not hidden: these belong to the target on purpose.
174
185
  puts " own #{j.owned.join(', ')}" unless j.owned.nil? || j.owned.empty?
175
186
  end
176
187
  end
177
188
  end
178
189
 
190
+ # Ask rsync what a sync of each directory job would actually do.
191
+ def verify_drift(cfg, programs)
192
+ Twin::Conflict.fill_drift(cfg, programs.flat_map(&:jobs))
193
+ end
194
+
195
+ def drift_summary(d)
196
+ if d.in_sync?
197
+ note = d.time_only.empty? ? "" : " (#{d.time_only.size} timestamp-only)"
198
+ return "in sync#{note}"
199
+ end
200
+ parts = []
201
+ parts << "#{d.pending.size} to sync: #{list_some(d.pending)}" unless d.pending.empty?
202
+ unless d.conflicts.empty?
203
+ parts << "#{d.conflicts.size} changed on target: #{list_some(d.conflicts.map(&:rel))} (twin sync will ask)"
204
+ end
205
+ parts.join("; ")
206
+ end
207
+
208
+ def list_some(rels, max = 3)
209
+ rels.take(max).join(", ") + (rels.size > max ? ", …" : "")
210
+ end
211
+
179
212
  # ── sync ───────────────────────────────────────────────────────────────────
180
213
 
181
214
  def cmd_sync(cfg, args)
@@ -264,7 +297,10 @@ module Twin
264
297
  return true if force
265
298
  return false if skip_conflicts || dry_run
266
299
 
267
- conflicts = jobs.flat_map { |j| Twin::Conflict.detect(cfg, j) }
300
+ # Verify: false jobs skip the detection round (too big to walk); rsync's
301
+ # --update still keeps newer target files, they just aren't listed here.
302
+ conflicts = jobs.reject { |j| j.verify == false }
303
+ .flat_map { |j| Twin::Conflict.detect(cfg, j) }
268
304
  return false if conflicts.empty?
269
305
 
270
306
  report_conflicts(conflicts)
@@ -423,11 +459,13 @@ module Twin
423
459
  if targets.empty?
424
460
  puts " (no programs loaded)"
425
461
  else
462
+ reachable_hosts = []
426
463
  targets.each do |tgt|
427
464
  if Twin::Remote.remote?(tgt)
428
465
  host, = Twin::Remote.split(tgt)
429
466
  if Twin::Remote.reachable?(host)
430
467
  puts " ✓ #{tgt} (ssh)"
468
+ reachable_hosts << host
431
469
  else
432
470
  puts " ✗ #{tgt} (ssh: #{host} not reachable)"
433
471
  ok = false
@@ -439,6 +477,7 @@ module Twin
439
477
  ok = false
440
478
  end
441
479
  end
480
+ ok = doctor_remote_tools(reachable_hosts.uniq) && ok
442
481
  end
443
482
  rescue => e
444
483
  puts " ✗ #{e.message}"
@@ -453,6 +492,42 @@ module Twin
453
492
  system("command -v #{name} > /dev/null 2>&1")
454
493
  end
455
494
 
495
+ # Probe each reachable ssh host once for what the far side must provide.
496
+ # A missing rsync fails doctor — the first sync would die mid-run with a
497
+ # raw protocol error. Missing stat/date or md5/md5sum only degrade
498
+ # (unknown mtimes, conservative conflicts), so they warn and say so.
499
+ def doctor_remote_tools(hosts)
500
+ return true if hosts.empty?
501
+ ok = true
502
+ puts "\nRemote tools"
503
+ hosts.each do |host|
504
+ tools = Twin::Remote.preflight(host)
505
+ if tools.nil?
506
+ puts " ✗ #{host} (probe failed)"
507
+ ok = false
508
+ next
509
+ end
510
+ problems = []
511
+ unless tools["rsync"]
512
+ problems << "rsync missing — syncs will fail (OpenWrt: opkg install rsync)"
513
+ ok = false
514
+ end
515
+ unless tools["stat"] || tools["date"]
516
+ problems << "no stat or date — remote mtimes read as unknown"
517
+ end
518
+ unless tools["md5"] || tools["md5sum"]
519
+ problems << "no md5 or md5sum — timestamp-only files stay flagged as conflicts"
520
+ end
521
+ if problems.empty?
522
+ have = ["rsync", tools["stat"] ? "stat" : "date", tools["md5"] ? "md5" : "md5sum"]
523
+ puts " ✓ #{host} (#{have.join(', ')})"
524
+ else
525
+ puts " #{tools["rsync"] ? "⚠" : "✗"} #{host} #{problems.join('; ')}"
526
+ end
527
+ end
528
+ ok
529
+ end
530
+
456
531
  # ── option parsing ─────────────────────────────────────────────────────────
457
532
 
458
533
  def parse_filter_opts(args)
@@ -501,6 +576,11 @@ module Twin
501
576
  h[:status] = j.status
502
577
  h[:source_mtime] = j.source_mtime&.iso8601
503
578
  h[:target_mtime] = j.target_mtime&.iso8601
579
+ h[:drift] = j.drift && {
580
+ pending: j.drift.pending,
581
+ time_only: j.drift.time_only,
582
+ conflicts: j.drift.conflicts.map(&:rel),
583
+ }
504
584
  h
505
585
  end
506
586
  end
data/lib/twin/conflict.rb CHANGED
@@ -1,40 +1,43 @@
1
1
  require "digest"
2
+ require "set"
2
3
 
3
4
  require_relative "remote"
4
5
 
5
6
  module Twin
6
- # Finding out which files on the target would be silently skipped by rsync's
7
- # --update, and whether that actually matters.
7
+ # What would a sync actually change? mtimes cannot answer that a
8
+ # directory's mtime records the last entry added or removed (after a sync:
9
+ # the sync itself), a file's mtime moves on a `cat >` copy that changed
10
+ # nothing, and rsync equalises directory mtimes on every run anyway. So we
11
+ # ask rsync, which has the answer already and knows its own matching rules
12
+ # better than any reimplementation would:
8
13
  #
9
- # `Job#conflict` is no help here. It compares the mtime of the job's own path,
10
- # and for a directory job that is the directory's mtime which says nothing
11
- # about the files inside it. Worse, it is wrong in exactly the case that
12
- # matters: editing a file in place leaves its directory's mtime untouched, and
13
- # `rsync -a` equalises directory mtimes on every run anyway. A hand-edit on
14
- # the target is therefore invisible to it.
14
+ # 1. What would a *forced* run transfer?
15
+ # One dry-run without --update. Empty means fully in syncthe common
16
+ # case and the cheap exit: one stat-walk per job and no more.
15
17
  #
16
- # So we ask rsync, which has the answer already and knows its own matching
17
- # rules better than any reimplementation would:
18
+ # 2. Of that, what does --update hold back?
19
+ # A second dry-run with --update. Everything the forced run would move
20
+ # but the normal one would not is exactly the set --update protects —
21
+ # files the target owns more recently.
18
22
  #
19
- # 1. Which files does --update hold back?
20
- # Dry-run twice, once with --update and once without. Everything the
21
- # second run would transfer but the first would not is exactly the set
22
- # --update protects.
23
- #
24
- # 2. Of those, which differ in content?
25
- # Only these are worth asking about. A file that is merely newer — same
26
- # bytes, later timestamp — is noise, and noise is what turns a prompt
27
- # into a reflex. Sync both sides of a tree in either order and you get
28
- # dozens of them.
29
- #
30
- # The first dry-run is also the cheap exit: when a forced run would move
31
- # nothing, the job is fully in sync and the second run is skipped. That is the
32
- # common case, so a quiet sync costs one extra stat-walk per job and no more.
23
+ # 3. What differs only in timestamp?
24
+ # rsync's itemize flags say so (">f..t" time, same size). Those files
25
+ # are checksummed; identical content is noise, not drift, and noise is
26
+ # what turns a prompt into a reflex. Sync both sides of a tree in
27
+ # either order and you get dozens of them.
33
28
  module Conflict
34
- # One file the target owns more recently than the source, with content that
35
- # actually differs. `same_content` is nil when it could not be determined
36
- # (remote targets) treated as a conflict, because guessing in the other
37
- # direction would overwrite work.
29
+ # The classified outcome of a forced dry-run for one job.
30
+ # pending — relative paths a sync would genuinely change
31
+ # time_only same bytes, different timestamp; a sync merely aligns them
32
+ # conflicts Entry list: target-side changes whose content differs
33
+ Drift = Struct.new(:pending, :time_only, :conflicts, keyword_init: true) do
34
+ def in_sync? = pending.empty? && conflicts.empty?
35
+ end
36
+
37
+ # One file the target owns more recently than the source, with content
38
+ # that actually differs (or could not be checked — remote md5 failed —
39
+ # which is treated as differing, because guessing the other way would
40
+ # overwrite work). target_mtime is nil for remote targets.
38
41
  Entry = Struct.new(:job, :rel, :source_path, :target_path,
39
42
  :source_mtime, :target_mtime, keyword_init: true) do
40
43
  def age_delta
@@ -43,55 +46,145 @@ module Twin
43
46
  end
44
47
  end
45
48
 
46
- # rsync --itemize-changes line → relative path. Change lines start with an
47
- # update type and a file type (">f.st...... lib/foo.rb"); "*deleting" and
48
- # the surrounding prose do not match.
49
- ITEMIZE_LINE = /\A[<>ch][fdLDS]\S*\s+(.+?)\s*\z/
49
+ # rsync --itemize-changes line → [flags, relative path]. Change lines
50
+ # start with an update type and a file type (">f.st...... lib/foo.rb") or
51
+ # "*deleting"; attribute-only lines (leading ".") and the surrounding
52
+ # prose do not match — they describe no change a sync would make.
53
+ ENTRY_LINE = /\A(\*deleting|[<>ch][fdLDS]\S*)\s+(.+?)\s*\z/
50
54
 
51
55
  module_function
52
56
 
53
- # Real conflicts for one job, in the order rsync reports them.
57
+ # Classify one job's drift by asking rsync. nil for render jobs (they
58
+ # compare content already and never use --update) and when a side is
59
+ # missing (status reports that on its own).
60
+ def drift(cfg, job)
61
+ return nil if job.render
62
+ return nil unless job.source_exists && job.target_exists
63
+
64
+ forced = itemized(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: true))
65
+ return Drift.new(pending: [], time_only: [], conflicts: []) if forced.empty?
66
+
67
+ normal_rels = itemized(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: false))
68
+ .map { |e| e[:rel] }.to_set
69
+ equal = equality_map(job, forced.filter_map { |e| e[:rel] if e[:kind] == :time })
70
+
71
+ assemble(forced, normal_rels, equal) do |rel|
72
+ conflict_entry(job, rel)
73
+ end
74
+ end
75
+
76
+ # Target-side changes worth asking about, in the order rsync reports them.
54
77
  # Empty when --update holds nothing back, or holds back only identical files.
55
78
  def detect(cfg, job)
56
- # Render jobs compare by content already and never use --update.
57
- return [] if job.render
58
- return [] unless job.source_exists && job.target_exists
79
+ drift(cfg, job)&.conflicts || []
80
+ end
59
81
 
60
- held_back_paths(cfg, job).filter_map { |rel| entry_for(job, rel) }
82
+ # Directory jobs whose drift is still unknown and answerable — both sides
83
+ # present, target reachable, not opted out via Verify: false. File jobs
84
+ # are content-checked by the scanner and need no second look.
85
+ def drift_candidates(jobs)
86
+ jobs.select do |j|
87
+ j.drift.nil? && j.verify != false && j.directory && j.active == 1 &&
88
+ !j.target_unreachable && j.source_exists && j.target_exists
89
+ end
61
90
  end
62
91
 
63
- # Relative paths that --update would skip: (would transfer forced) minus
64
- # (would transfer normally).
65
- def held_back_paths(cfg, job)
66
- forced = itemized_paths(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: true))
67
- return [] if forced.empty? # nothing to move at all — no need to ask rsync twice
92
+ # Fill `drift` on every candidate by asking rsync the dry-runs are
93
+ # subprocess I/O, so a few run in parallel.
94
+ def fill_drift(cfg, jobs)
95
+ jobs = drift_candidates(jobs)
96
+ return if jobs.empty?
97
+
98
+ queue = Queue.new
99
+ jobs.each { |j| queue << j }
100
+ Array.new([4, jobs.size].min) do
101
+ Thread.new do
102
+ loop do
103
+ j = begin
104
+ queue.pop(true)
105
+ rescue ThreadError
106
+ break
107
+ end
108
+ j.drift = drift(cfg, j)
109
+ end
110
+ end
111
+ end.each(&:join)
112
+ end
68
113
 
69
- normal = itemized_paths(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: false))
70
- forced - normal
114
+ # Pure assembly of a Drift from parsed entries. A file the normal run
115
+ # would also transfer flows source→target as intended; one only the
116
+ # forced run would touch is being held back by --update — the target owns
117
+ # it more recently. Content decides whether that is a conflict or noise.
118
+ def assemble(forced, normal_rels, equal)
119
+ d = Drift.new(pending: [], time_only: [], conflicts: [])
120
+ forced.each do |e|
121
+ rel = e[:rel]
122
+ case e[:kind]
123
+ when :deleted, :new then d.pending << rel
124
+ when :content
125
+ normal_rels.include?(rel) ? d.pending << rel : d.conflicts << yield(rel)
126
+ when :time
127
+ if equal[rel]
128
+ d.time_only << rel
129
+ elsif normal_rels.include?(rel)
130
+ d.pending << rel
131
+ else
132
+ d.conflicts << yield(rel)
133
+ end
134
+ end
135
+ end
136
+ d
71
137
  end
72
138
 
73
- def itemized_paths(args)
139
+ # Run rsync and parse its itemize output into [{rel:, kind:}, ...].
140
+ def itemized(args)
74
141
  output, status = Twin::Sync.run(args)
75
142
  return [] unless status.success?
76
143
  output.lines.filter_map do |line|
77
- m = ITEMIZE_LINE.match(line)
144
+ m = ENTRY_LINE.match(line)
78
145
  next unless m
79
- rel = m[1]
80
- next if rel == "./" || rel.end_with?("/") # directories carry no content
81
- rel
146
+ kind = classify(m[1])
147
+ next if kind == :attrs
148
+ { rel: m[2], kind: kind }
82
149
  end
83
150
  end
84
151
 
85
- # Build an Entry unless source and target hold the same bytes.
86
- def entry_for(job, rel)
87
- src = resolve(job.source_path, rel)
88
- tgt = resolve(job.target_path, rel)
152
+ # Itemize flags what kind of change this is.
153
+ # :deleted — target-only file, removed by Delete: true
154
+ # :new — does not exist on the target yet (files and directories)
155
+ # :content size differs, so the bytes certainly do
156
+ # :time — timestamp only; content equality still to be determined
157
+ # :attrs — permissions/owner, no change a sync-file cares about
158
+ def classify(flags)
159
+ return :deleted if flags == "*deleting"
160
+ body = flags[2..].to_s
161
+ return :new if body.include?("+")
162
+ return :content if body.include?("s")
163
+ return :time if body.include?("t") || body.include?("T")
164
+ :attrs
165
+ end
89
166
 
90
- # Remote targets can't be read here; report them rather than assume.
91
- unless job.remote?
92
- return nil if same_content?(src, tgt)
167
+ # Content equality for the :time candidates, {rel => bool}. Local pairs
168
+ # are compared directly; remote targets get one batched md5 round per job
169
+ # (STAT_SCRIPT-style), and an unanswered path counts as differing.
170
+ def equality_map(job, rels)
171
+ return {} if rels.empty?
172
+ if job.remote?
173
+ _host, rbase = Twin::Remote.split(job.target_path)
174
+ remote_paths = rels.to_h { |r| [r, job.directory ? File.join(rbase, r) : rbase] }
175
+ sums = Twin::Remote.md5_paths(Twin::Remote.split(job.target)[0], remote_paths.values) || {}
176
+ rels.to_h do |r|
177
+ local = local_md5(resolve(job.source_path, r))
178
+ [r, !local.nil? && sums[remote_paths[r]] == local]
179
+ end
180
+ else
181
+ rels.to_h { |r| [r, same_content?(resolve(job.source_path, r), resolve(job.target_path, r))] }
93
182
  end
183
+ end
94
184
 
185
+ def conflict_entry(job, rel)
186
+ src = resolve(job.source_path, rel)
187
+ tgt = job.remote? ? job.target_path : resolve(job.target_path, rel)
95
188
  Entry.new(
96
189
  job: job, rel: rel, source_path: src, target_path: tgt,
97
190
  source_mtime: mtime(src), target_mtime: job.remote? ? nil : mtime(tgt),
@@ -114,6 +207,12 @@ module Twin
114
207
 
115
208
  def digest(path) = Digest::SHA256.file(path).hexdigest
116
209
 
210
+ def local_md5(path)
211
+ Digest::MD5.file(path).hexdigest
212
+ rescue Errno::ENOENT, Errno::EACCES, Errno::EISDIR
213
+ nil
214
+ end
215
+
117
216
  def mtime(path)
118
217
  File.mtime(path)
119
218
  rescue Errno::ENOENT, Errno::EACCES
data/lib/twin/picker.rb CHANGED
@@ -2,6 +2,7 @@ require "open3"
2
2
  require "tempfile"
3
3
 
4
4
  require_relative "preview"
5
+ require_relative "conflict"
5
6
 
6
7
  module Twin
7
8
  # Two-step fzf picker:
@@ -14,6 +15,11 @@ module Twin
14
15
  source_newer: "→",
15
16
  target_newer: "←",
16
17
  in_sync: "✓",
18
+ # Directory jobs in stage 1: their mtimes prove nothing, and the
19
+ # dry-runs that would prove something are too slow for the full picker.
20
+ # No claim instead of a wrong one — stage 2 verifies its one program
21
+ # on open, and `twin status` has the full verdict.
22
+ unverified: "∘",
17
23
  missing_target: "!",
18
24
  missing_source: "!",
19
25
  both_missing: "✗",
@@ -25,6 +31,7 @@ module Twin
25
31
  source_newer: "\e[33m", # yellow
26
32
  target_newer: "\e[36m", # cyan
27
33
  in_sync: "\e[32m", # green
34
+ unverified: "\e[2m", # dim
28
35
  missing_target: "\e[31m", # red
29
36
  missing_source: "\e[31m", # red
30
37
  both_missing: "\e[31m", # red
@@ -44,6 +51,29 @@ module Twin
44
51
 
45
52
  # ── Stufe 1: program picker ───────────────────────────────────────────────
46
53
 
54
+ # A trailing bracket group ties a program to its base name by convention:
55
+ # "livesync [agent]" lists under "livesync". Only the suffix form counts —
56
+ # brackets elsewhere are just a name.
57
+ BRACKET_SUFFIX = /\s*\[[^\]]*\]\z/
58
+
59
+ def base_name(name) = name.sub(BRACKET_SUFFIX, "").strip
60
+
61
+ # One stage-1 entry per base name (case-insensitive), even when it appears
62
+ # in several sync-files or as bracketed variants — "everything of
63
+ # fileview" is one decision. The list is sorted by name; document order
64
+ # only orders the *jobs*, program order carries no meaning. Purely a
65
+ # picker view: the data model, `status`, `sync -p` and JSON keep the
66
+ # per-file programs. Jobs stay grouped per file in document order, so a
67
+ # file's closing Cmd block still fires after its own paths.
68
+ def merge_programs(programs)
69
+ programs.group_by { |p| base_name(p.name).downcase }.values.map do |group|
70
+ next group.first if group.size == 1
71
+ name = group.find { |p| !p.name.match?(BRACKET_SUFFIX) }&.name ||
72
+ base_name(group.first.name)
73
+ Program.new(name: name, jobs: group.flat_map(&:jobs))
74
+ end.sort_by { |p| p.name.downcase }
75
+ end
76
+
47
77
  # Multi-line entries (NUL-separated). Header line per program, indented
48
78
  # body lines per job. Returns the selected Program or nil.
49
79
  def pick_program(programs)
@@ -85,15 +115,45 @@ module Twin
85
115
  jobs = program.jobs
86
116
  return [] if jobs.empty?
87
117
 
118
+ # Stage 2 covers one program, so the dry-runs that are too slow for the
119
+ # whole picker are affordable here — directory rows get a real verdict
120
+ # instead of `∘`. Already-verified jobs are skipped on re-entry.
121
+ if Conflict.drift_candidates(jobs).any?
122
+ $stderr.print dim("verifying directories …")
123
+ Conflict.fill_drift(cfg, jobs)
124
+ $stderr.print "\r\e[K"
125
+ end
126
+
88
127
  path_width = jobs.map { |j| j.path.length }.max
89
128
  tempfiles = write_compact_previews(program, jobs)
90
129
  preview_cmd, mapfile = build_apex_preview_cmd(tempfiles, cfg)
91
130
 
92
- rows = jobs.each_with_index.map do |j, i|
93
- icon = STATUS_ICONS[j.status] || "?"
94
- delta = format_delta(j.source_mtime, j.target_mtime)
95
- line = "#{icon} #{j.path.ljust(path_width)} #{delta}"
96
- "#{i}\t#{colorize(j.status, line)}"
131
+ # A merged program spans sync-files (and bracketed variants); a dim
132
+ # section row marks each block, named by its file — plus the original
133
+ # program name when it differs from the merged one. Section rows carry
134
+ # "-" instead of an index — fzf can't make them unselectable, but they
135
+ # map to no job, so toggling one is inert.
136
+ section = ->(j) { [j.program, j.sync_file] }
137
+ multi = jobs.map(&section).uniq.size > 1
138
+ rows = []
139
+ prev = nil
140
+ jobs.each_with_index do |j, i|
141
+ if multi && section.(j) != prev
142
+ file = File.basename(j.sync_file.to_s)
143
+ label = j.program == program.name ? file : "#{j.program} · #{file}"
144
+ rows << "-\t#{dim("[#{label}]")}"
145
+ prev = section.(j)
146
+ end
147
+ icon = STATUS_ICONS[j.status] || "?"
148
+ line = "#{icon} #{j.path.ljust(path_width)} #{job_delta(j)}"
149
+ rows << "#{i}\t#{colorize(j.status, line)}"
150
+ end
151
+
152
+ header = "#{program.name} — Tab toggles, Enter confirms"
153
+ unverified = jobs.select { |j| j.verify == false }
154
+ unless unverified.empty?
155
+ header += "\n\e[33m⚠ content checks off (Verify: false): " \
156
+ "#{unverified.map(&:path).join(", ")}\e[0m"
97
157
  end
98
158
 
99
159
  fzf = [
@@ -101,7 +161,7 @@ module Twin
101
161
  "--multi", "--ansi",
102
162
  "--delimiter=\t", "--with-nth=2",
103
163
  "--prompt=#{program.name} > ",
104
- "--header=#{program.name} — Tab toggles, Enter confirms",
164
+ "--header=#{header}",
105
165
  "--preview=#{preview_cmd}",
106
166
  "--preview-window=right:60%:wrap",
107
167
  "--height=100%", "--reverse",
@@ -109,45 +169,55 @@ module Twin
109
169
  "--color=bg+:-1,hl+:reverse",
110
170
  ]
111
171
 
112
- output, status = Open3.capture2(SH_ENV, *fzf, stdin_data: rows.join("\n"))
113
- return :back if status.exitstatus == 130 # ESC / Ctrl-C
114
- return [] unless status.success?
115
- return [] if output.strip.empty?
172
+ loop do
173
+ output, status = Open3.capture2(SH_ENV, *fzf, stdin_data: rows.join("\n"))
174
+ return :back if status.exitstatus == 130 # ESC / Ctrl-C
175
+ return [] unless status.success?
176
+ return [] if output.strip.empty?
116
177
 
117
- output.lines.filter_map do |line|
118
- idx = line.split("\t", 2).first&.to_i
119
- jobs[idx] if idx
178
+ selected = output.lines.filter_map do |line|
179
+ idx = line.split("\t", 2).first
180
+ jobs[idx.to_i] if idx&.match?(/\A\d+\z/)
181
+ end
182
+ return selected unless selected.empty?
183
+ # Enter landed on a section header (or only headers were toggled):
184
+ # nothing real was accepted — reopen rather than read it as "exit".
120
185
  end
121
186
  ensure
122
- tempfiles&.each_value { |path| File.unlink(path) rescue nil }
123
- File.unlink(mapfile) rescue nil if mapfile
187
+ tempfiles&.each_value { |f| f.close! rescue nil }
188
+ mapfile&.close! rescue nil
124
189
  end
125
190
 
126
- # Write per-job compact-preview markdown to tempfiles. Returns {idx => path}.
191
+ # Write per-job compact-preview markdown to tempfiles. Returns
192
+ # {idx => Tempfile}. The Tempfile objects (not just their paths) must stay
193
+ # referenced while fzf runs: a GC'd Tempfile unlinks its file, and the
194
+ # preview command would find nothing to render. Each job's excerpt comes
195
+ # from its own sync-file — a merged program spans several.
127
196
  def write_compact_previews(program, jobs)
128
197
  result = {}
129
198
  jobs.each_with_index do |job, i|
130
- compact = Preview.extract_compact(program.sync_file, job.path)
199
+ compact = Preview.extract_compact(job.sync_file, job.path)
131
200
  f = Tempfile.new(["twin-#{i}-", ".md"])
132
201
  f.write(compact)
133
202
  f.close
134
- result[i] = f.path
203
+ result[i] = f
135
204
  end
136
205
  result
137
206
  end
138
207
 
139
- # Returns [preview_cmd, mapfile_path]; the caller unlinks the mapfile.
208
+ # Returns [preview_cmd, mapfile]; the caller holds the Tempfile until fzf
209
+ # is done (GC would unlink it) and closes it afterwards.
140
210
  def build_apex_preview_cmd(tempfiles, cfg)
141
211
  # Map idx → file via a small TSV, awk picks the right one for {1}.
142
212
  mapfile = Tempfile.new(["twin-map-", ".tsv"])
143
- tempfiles.each { |i, path| mapfile.puts("#{i}\t#{path}") }
213
+ tempfiles.each { |i, f| mapfile.puts("#{i}\t#{f.path}") }
144
214
  mapfile.close
145
215
 
146
216
  render_cmd = pick_renderer(cfg)
147
217
 
148
218
  cmd = %(F=$(awk -v id={1} -F'\\t' '$1==id {print $2}' #{mapfile.path}); ) +
149
219
  %([ -n "$F" ] && #{render_cmd})
150
- [cmd, mapfile.path]
220
+ [cmd, mapfile]
151
221
  end
152
222
 
153
223
  # Pick the first available markdown renderer.
@@ -178,22 +248,27 @@ module Twin
178
248
 
179
249
  def render_program_entry(program, name_width, path_width)
180
250
  icon = colorize(program.status, STATUS_ICONS[program.status] || "?")
181
- sync_file = File.basename(program.sync_file.to_s)
251
+ files = program.jobs.map { |j| File.basename(j.sync_file.to_s) }.uniq.join(" · ")
182
252
  count = program.active_jobs.size
183
253
  total = program.jobs.size
184
254
  header = "#{icon} #{bold(program.name.ljust(name_width))} " \
185
- "#{dim("(#{count}/#{total})")} #{dim("[#{sync_file}]")}"
255
+ "#{dim("(#{count}/#{total})")} #{dim("[#{files}]")}"
186
256
 
187
257
  body = program.jobs.map do |j|
188
258
  icon = STATUS_ICONS[j.status] || "?"
189
- delta = format_delta(j.source_mtime, j.target_mtime)
190
- line = " #{icon} #{j.path.ljust(path_width)} #{delta}"
259
+ line = " #{icon} #{j.path.ljust(path_width)} #{job_delta(j)}"
191
260
  colorize(j.status, line)
192
261
  end
193
262
 
194
263
  ([header] + body).join("\n")
195
264
  end
196
265
 
266
+ # A directory's mtime delta would mislead (it moves on every sync), so
267
+ # directory jobs show none.
268
+ def job_delta(job)
269
+ job.directory ? "" : format_delta(job.source_mtime, job.target_mtime)
270
+ end
271
+
197
272
  def format_delta(sm, tm)
198
273
  return "" if sm.nil? || tm.nil?
199
274
  seconds = (sm - tm).to_i
data/lib/twin/remote.rb CHANGED
@@ -26,22 +26,40 @@ module Twin
26
26
 
27
27
  # Stat many paths in one ssh round-trip. Paths go over stdin (one per
28
28
  # line), the remote loop answers "path<TAB>epoch" or "path<TAB>-" for
29
- # missing ones. Tries BSD stat first, then GNU covers macOS and Linux.
29
+ # missing ones. Tries BSD stat, then GNU, then BusyBox `date -r` — covers
30
+ # macOS, Linux and OpenWrt. Auf dem Brume (25.08.2026) fehlte `stat`
31
+ # komplett: die leere Substitution wurde als Epoche 0 geparst und `twin
32
+ # status` zeigte 1970 statt "unbekannt". `date -r` kennt kein `--`;
33
+ # verkraftbar, weil hier nur absolute Zielpfade ankommen.
30
34
  # Returns {path => Time or nil-if-missing}, or nil when ssh itself failed.
35
+ # POSIX sh, and it must stay free of single quotes: it is handed to the
36
+ # remote side wrapped in '...' so that *any* login shell passes it through
37
+ # literally. The usual POSIX escape for an embedded quote ('\'') is parsed
38
+ # differently by fish, so the rule here is simply not to need it — hence
39
+ # double quotes around the printf formats.
31
40
  STAT_SCRIPT = <<~SH.freeze
32
41
  while IFS= read -r p; do
33
42
  if [ -e "$p" ]; then
34
- printf '%s\t%s\n' "$p" "$(stat -f %m -- "$p" 2>/dev/null || stat -c %Y -- "$p")"
43
+ m=$(stat -f %m -- "$p" 2>/dev/null || stat -c %Y -- "$p" 2>/dev/null || date -r "$p" +%s)
44
+ printf "%s\t%s\n" "$p" "$m"
35
45
  else
36
- printf '%s\t-\n' "$p"
46
+ printf "%s\t-\n" "$p"
37
47
  fi
38
48
  done
39
49
  SH
40
50
 
41
51
  def stat_paths(host, paths)
42
52
  return {} if paths.empty?
53
+ # Explicitly through /bin/sh. ssh hands the command to the *login shell*
54
+ # on the far side, and that is not necessarily POSIX: on a machine whose
55
+ # shell is fish, the bare script dies on "while IFS= read -r p; do" and
56
+ # twin read the failure as "host not answering" — every ssh target showed
57
+ # as :unreachable in status and the picker, for as long as SSH targets
58
+ # existed (#8, 2026-07-17). Found and fixed 2026-08-25.
59
+ raise "STAT_SCRIPT must not contain single quotes" if STAT_SCRIPT.include?("'")
60
+
43
61
  out, _err, status = Open3.capture3(
44
- "ssh", *SSH_OPTS, host, STAT_SCRIPT,
62
+ "ssh", *SSH_OPTS, host, "/bin/sh -c '#{STAT_SCRIPT}'",
45
63
  stdin_data: paths.join("\n") + "\n"
46
64
  )
47
65
  return nil unless status.success?
@@ -50,13 +68,90 @@ module Twin
50
68
  out.each_line do |line|
51
69
  path, mtime = line.chomp.split("\t", 2)
52
70
  next unless path && mtime
53
- result[path] = mtime == "-" ? nil : Time.at(mtime.to_i)
71
+ # Nur echte Epochen als Zeit werten. Scheitert die ganze stat-Kette,
72
+ # ist das Feld leer — das ist "unbekannt", nicht 1970.
73
+ result[path] = mtime.match?(/\A\d+\z/) ? Time.at(mtime.to_i) : nil
54
74
  end
55
75
  result
56
76
  rescue Errno::ENOENT
57
77
  nil # ssh not installed
58
78
  end
59
79
 
80
+ # Checksum many paths in one ssh round-trip, same shape as stat_paths:
81
+ # paths over stdin, "path<TAB>md5" back, "-" for anything that is not a
82
+ # regular file. Tries BSD md5 first, then md5sum (GNU, BusyBox) — covers
83
+ # macOS, Linux and OpenWrt. md5sum prints "hash path"; the parameter
84
+ # expansion keeps only the first word. Same single-quote rule as
85
+ # STAT_SCRIPT, and MD5 is drift detection here, not cryptography.
86
+ # Returns {path => hex or nil-if-unreadable}, or nil when ssh itself failed.
87
+ MD5_SCRIPT = <<~SH.freeze
88
+ while IFS= read -r p; do
89
+ if [ -f "$p" ]; then
90
+ m=$(md5 -q "$p" 2>/dev/null || md5sum "$p" 2>/dev/null)
91
+ m=${m%% *}
92
+ printf "%s\t%s\n" "$p" "$m"
93
+ else
94
+ printf "%s\t-\n" "$p"
95
+ fi
96
+ done
97
+ SH
98
+
99
+ def md5_paths(host, paths)
100
+ return {} if paths.empty?
101
+ raise "MD5_SCRIPT must not contain single quotes" if MD5_SCRIPT.include?("'")
102
+
103
+ out, _err, status = Open3.capture3(
104
+ "ssh", *SSH_OPTS, host, "/bin/sh -c '#{MD5_SCRIPT}'",
105
+ stdin_data: paths.join("\n") + "\n"
106
+ )
107
+ return nil unless status.success?
108
+
109
+ result = {}
110
+ out.each_line do |line|
111
+ path, sum = line.chomp.split("\t", 2)
112
+ next unless path && sum
113
+ result[path] = sum.match?(/\A\h{32}\z/) ? sum.downcase : nil
114
+ end
115
+ result
116
+ rescue Errno::ENOENT
117
+ nil # ssh not installed
118
+ end
119
+
120
+ # Which of the tools twin relies on exist on the far side? One ssh
121
+ # round-trip per host, for `twin doctor`. rsync carries the sync itself;
122
+ # stat/date feed the batched mtime round; md5/md5sum feed the content
123
+ # check. Same single-quote rule as the other batch scripts.
124
+ # Returns {tool => present?}, or nil when ssh itself failed.
125
+ PREFLIGHT_SCRIPT = <<~SH.freeze
126
+ for t in rsync stat date md5 md5sum; do
127
+ if command -v "$t" >/dev/null 2>&1; then
128
+ printf "%s\tok\n" "$t"
129
+ else
130
+ printf "%s\t-\n" "$t"
131
+ fi
132
+ done
133
+ SH
134
+
135
+ def preflight(host)
136
+ raise "PREFLIGHT_SCRIPT must not contain single quotes" if PREFLIGHT_SCRIPT.include?("'")
137
+
138
+ out, _err, status = Open3.capture3("ssh", *SSH_OPTS, host, "/bin/sh -c '#{PREFLIGHT_SCRIPT}'")
139
+ return nil unless status.success?
140
+ parse_preflight(out)
141
+ rescue Errno::ENOENT
142
+ nil # ssh not installed
143
+ end
144
+
145
+ def parse_preflight(out)
146
+ result = {}
147
+ out.each_line do |line|
148
+ tool, state = line.chomp.split("\t", 2)
149
+ next unless tool && state
150
+ result[tool] = state == "ok"
151
+ end
152
+ result
153
+ end
154
+
60
155
  # Create a directory on the remote side (mkdir -p equivalent).
61
156
  def mkdir_p(host, dir)
62
157
  _out, _err, status = Open3.capture3("ssh", *SSH_OPTS, host, "mkdir", "-p", shellesc(dir))
data/lib/twin/scanner.rb CHANGED
@@ -2,6 +2,7 @@ require "json"
2
2
  require "open3"
3
3
 
4
4
  require_relative "remote"
5
+ require_relative "conflict"
5
6
 
6
7
  module Twin
7
8
  # One YAML block from a sync-file, enriched with live filesystem state.
@@ -9,7 +10,7 @@ module Twin
9
10
  :program, :path, :description, :active, :excludes, :owned, :label,
10
11
  :source, :target, :cmd, :delete, :render, :render_outdated, :target_path_field, :sync_file,
11
12
  :source_exists, :target_exists, :source_mtime, :target_mtime, :conflict,
12
- :target_unreachable,
13
+ :target_unreachable, :directory, :content_equal, :drift, :verify,
13
14
  keyword_init: true,
14
15
  ) do
15
16
  def source_path = File.join(source, path)
@@ -29,6 +30,19 @@ module Twin
29
30
  # Render jobs compare by content, not mtime — a rendered target's mtime
30
31
  # bears no relation to the template's.
31
32
  return render_outdated ? :source_newer : :in_sync if render
33
+ # Directory jobs carry no mtime verdict at all: a directory's mtime
34
+ # records the last entry added or removed — after a sync that is the
35
+ # sync itself, and even an *excluded* file moves it. `twin status`
36
+ # fills in `drift` by asking rsync; without it the honest answer is
37
+ # "not checked", not a guess.
38
+ if directory
39
+ return :unverified unless drift
40
+ return :target_newer if drift.conflicts.any?
41
+ return :source_newer if drift.pending.any?
42
+ return :in_sync
43
+ end
44
+ # Same bytes under a newer timestamp (a `cat >` copy) is not drift.
45
+ return :in_sync if content_equal
32
46
  return :target_newer if conflict
33
47
  return :in_sync if source_mtime.nil? || target_mtime.nil?
34
48
  delta = source_mtime - target_mtime
@@ -50,7 +64,7 @@ module Twin
50
64
  # Aggregate status across jobs — worst first.
51
65
  def status
52
66
  states = jobs.map(&:status)
53
- %i[unreachable both_missing missing_source missing_target target_newer source_newer disabled in_sync]
67
+ %i[unreachable both_missing missing_source missing_target target_newer source_newer unverified disabled in_sync]
54
68
  .find { |s| states.include?(s) } || :in_sync
55
69
  end
56
70
 
@@ -103,9 +117,32 @@ module Twin
103
117
  mtime = stats[rpath]
104
118
  j.target_exists = !mtime.nil?
105
119
  j.target_mtime = mtime
106
- j.conflict = j.source_exists && mtime && j.source_mtime &&
120
+ j.conflict = !j.directory && j.source_exists && mtime && j.source_mtime &&
107
121
  mtime - j.source_mtime >= 60
108
122
  end
123
+ verify_remote_file_content(host, host_jobs)
124
+ end
125
+ end
126
+
127
+ # Remote counterpart of the local content check in build_job: file jobs
128
+ # whose mtimes drifted get one batched md5 round per host. Identical
129
+ # content clears the conflict — the timestamps merely disagree.
130
+ def verify_remote_file_content(host, host_jobs)
131
+ candidates = host_jobs.select do |j|
132
+ j.verify != false && !j.directory && !j.render &&
133
+ j.source_exists && j.target_exists &&
134
+ j.source_mtime && j.target_mtime && (j.target_mtime - j.source_mtime).abs >= 60
135
+ end
136
+ return if candidates.empty?
137
+
138
+ sums = Twin::Remote.md5_paths(host, candidates.map { |j| Twin::Remote.split(j.target_path).last })
139
+ return if sums.nil?
140
+
141
+ candidates.each do |j|
142
+ remote_sum = sums[Twin::Remote.split(j.target_path).last]
143
+ next if remote_sum.nil?
144
+ j.content_equal = remote_sum == Twin::Conflict.local_md5(j.source_path)
145
+ j.conflict = false if j.content_equal
109
146
  end
110
147
  end
111
148
 
@@ -147,6 +184,11 @@ module Twin
147
184
  return nil if path.empty? || source.empty? || target.empty?
148
185
 
149
186
  render = r["Render"] == true
187
+ # Verify: false — no content verification for this entry, ever: too big
188
+ # or too remote for md5/dry-run rounds (a node_modules tree over SMB).
189
+ # Status falls back to mtime for files and stays ∘ for directories; the
190
+ # pre-sync conflict check skips it (--update still protects the target).
191
+ verify = r["Verify"] != false
150
192
  excludes = split_list(r["Exclude"])
151
193
  # Own: paths inside the sync scope that the TARGET owns — machine-specific
152
194
  # config the source must never clobber. Same rsync effect as Exclude, but
@@ -170,8 +212,22 @@ module Twin
170
212
  # target). conflict stays false so the mtime conflict-warning skips them.
171
213
  render_outdated = render ? render_outdated?(src_full, tgt_full, vars, path) : nil
172
214
 
215
+ # rsync mirrors directories, so a directory source means a directory
216
+ # target — also for remote jobs, whose far side can't be inspected here.
217
+ directory = src_exists && File.directory?(src_full)
218
+
219
+ # A file job whose mtimes drifted apart may still hold the same bytes
220
+ # (a `cat >` copy before the first twin run). Check before judging;
221
+ # a directory's own mtime is judged not at all (see Job#status).
222
+ content_equal = nil
223
+ if verify && !render && !remote && !directory && src_exists && tgt_exists &&
224
+ src_mtime && tgt_mtime && (tgt_mtime - src_mtime).abs >= 60
225
+ content_equal = Twin::Conflict.same_content?(src_full, tgt_full)
226
+ end
227
+
173
228
  # Same 60s tolerance as Job#status, so mtime jitter never flags a conflict.
174
- conflict = !render && src_exists && tgt_exists && tgt_mtime && src_mtime &&
229
+ conflict = !render && !directory && !content_equal &&
230
+ src_exists && tgt_exists && tgt_mtime && src_mtime &&
175
231
  tgt_mtime - src_mtime >= 60
176
232
 
177
233
  Job.new(
@@ -196,6 +252,9 @@ module Twin
196
252
  target_mtime: tgt_mtime,
197
253
  conflict: !!conflict,
198
254
  target_unreachable: false,
255
+ directory: directory,
256
+ content_equal: content_equal,
257
+ verify: verify,
199
258
  )
200
259
  end
201
260
 
data/lib/twin/sync.rb CHANGED
@@ -130,6 +130,9 @@ module Twin
130
130
 
131
131
  xfr = !dry_run && transferred?(output)
132
132
 
133
+ # conflict is content-verified by the scanner and only ever set on file
134
+ # jobs — a directory's mtime says nothing, so directory jobs surface
135
+ # target-side changes through the pre-run prompt instead.
133
136
  if job.conflict && !xfr && !dry_run && !force
134
137
  output += "\nskipped: target is newer, source not synced"
135
138
  end
data/lib/twin/version.rb CHANGED
@@ -1,3 +1,3 @@
1
1
  module Twin
2
- VERSION = "0.4.3"
2
+ VERSION = "0.6.0"
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: mark-twin
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.4.3
4
+ version: 0.6.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Ralf Hülsmann