mark-twin 0.4.3 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/ARCHITECTURE.md +35 -12
- data/README.md +43 -13
- data/lib/twin/cli.rb +89 -9
- data/lib/twin/conflict.rb +155 -56
- data/lib/twin/picker.rb +100 -25
- data/lib/twin/remote.rb +100 -5
- data/lib/twin/scanner.rb +63 -4
- data/lib/twin/sync.rb +3 -0
- data/lib/twin/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9e643b14b2fe0dd153539ff16d10a0950f3c478d327620dec1b9b04d7af4f340
|
|
4
|
+
data.tar.gz: 4eb24c0280ee5f1085024c1357a6d7b8dd0b7f16198e3cce8b4c9ce5849705b5
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 7d5567dc0637bc767f7ddff5716a2e5031902e787958e1ecd69f54931e737b2946d82c999637d6610d732e5063898c3b717b49148f8e495af328ea2f0b716d03
|
|
7
|
+
data.tar.gz: 94df8bd72ea5548e668b1173e17b9e9bf54efc1fbca4576c109ed64740794decb94eb23fab78c50ed2b0654bc2160bb1a4be85c74a2f9281bd2a178e0f33b270
|
data/ARCHITECTURE.md
CHANGED
|
@@ -53,18 +53,29 @@ test/test_pure.rb
|
|
|
53
53
|
|
|
54
54
|
```
|
|
55
55
|
program, path, description, active, excludes, owned, label, source, target, cmd,
|
|
56
|
-
delete, render, render_outdated, target_path_field, sync_file,
|
|
57
|
-
source_exists, target_exists, source_mtime, target_mtime, conflict
|
|
56
|
+
delete, render, render_outdated, target_path_field, sync_file, verify,
|
|
57
|
+
source_exists, target_exists, source_mtime, target_mtime, conflict,
|
|
58
|
+
directory, content_equal, drift
|
|
58
59
|
```
|
|
59
60
|
|
|
60
61
|
`excludes` and `owned` both become `--exclude` (via `Job#all_excludes`); they
|
|
61
|
-
are kept apart so `status` can report intent.
|
|
62
|
-
|
|
63
|
-
|
|
62
|
+
are kept apart so `status` can report intent.
|
|
63
|
+
|
|
64
|
+
mtime is never the verdict, only a pre-filter for file jobs — and even there
|
|
65
|
+
identical bytes under a drifted timestamp clear it (`content_equal`; remote
|
|
66
|
+
targets get one batched md5 round per host). `conflict` is therefore
|
|
67
|
+
content-verified when set. Directory jobs get no mtime judgment at all: a
|
|
68
|
+
directory's mtime moves on every sync and on every excluded file, so
|
|
69
|
+
`Job#status` reports `unverified` until `twin status` fills `drift` by asking
|
|
70
|
+
rsync (`Conflict.drift` — paired dry-runs, itemize classification, checksums
|
|
71
|
+
for timestamp-only candidates). `Verify: false` opts a job out of every
|
|
72
|
+
content round — scanner md5, drift dry-runs, pre-sync conflict detection —
|
|
73
|
+
for entries where the walk itself is the cost; its status stays mtime-based
|
|
74
|
+
(files) or `unverified` (directories).
|
|
64
75
|
|
|
65
76
|
`Job#status` → one of `disabled / unreachable / both_missing / missing_source /
|
|
66
|
-
missing_target / target_newer / in_sync / source_newer`. Render
|
|
67
|
-
(`render_outdated`), not mtime
|
|
77
|
+
missing_target / target_newer / in_sync / source_newer / unverified`. Render
|
|
78
|
+
jobs derive status from content (`render_outdated`), not mtime.
|
|
68
79
|
`Job#target_path` joins `target` with `target_path_field || path`.
|
|
69
80
|
|
|
70
81
|
**Program** — group of Jobs sharing a `program` name:
|
|
@@ -156,16 +167,28 @@ rationale.
|
|
|
156
167
|
Two stages:
|
|
157
168
|
|
|
158
169
|
1. **Stage 1 — program picker.** Multi-line NUL-separated entries (`--read0`).
|
|
159
|
-
Each entry has a header (icon, program name, job count, sync-
|
|
170
|
+
Each entry has a header (icon, program name, job count, sync-files) and
|
|
160
171
|
indented body lines (one per job). No preview. Single-select. ESC exits.
|
|
172
|
+
Same-named programs (case-insensitive) from different sync-files are merged
|
|
173
|
+
into one entry, and a trailing bracket group joins by convention:
|
|
174
|
+
`livesync [agent]` merges under `livesync`. A picker-only view
|
|
175
|
+
(`Picker.merge_programs`); the data model, `status`, `sync -p` and JSON
|
|
176
|
+
keep the per-file programs. Jobs stay grouped per file in document order,
|
|
177
|
+
so a file's closing `Cmd` block still fires after its own paths.
|
|
161
178
|
2. **Stage 2 — path multi-picker.** Tab-delimited rows (`id\tdisplay`),
|
|
162
|
-
`--with-nth=2` hides the `id`. Multi-select via Tab.
|
|
163
|
-
|
|
164
|
-
|
|
179
|
+
`--with-nth=2` hides the `id`. Multi-select via Tab. On open, directory
|
|
180
|
+
jobs get their drift verified (`Conflict.fill_drift` — affordable for one
|
|
181
|
+
program, unlike for the whole of stage 1). A merged program's rows are
|
|
182
|
+
sectioned per sync-file by dim header rows whose id is `-`; they map to no
|
|
183
|
+
job, so toggling one is inert. Preview pane shows the *compact* view
|
|
184
|
+
(frontmatter + intro + the heading section containing the YAML block for
|
|
185
|
+
the highlighted path) from the job's own sync-file, rendered via
|
|
165
186
|
`apex --plugins -t terminal256`. ESC returns to Stage 1.
|
|
166
187
|
|
|
167
188
|
Compact previews are pre-rendered to per-job tempfiles before fzf launches.
|
|
168
|
-
An `awk` lookup maps `{1}` (the id) → tempfile path.
|
|
189
|
+
An `awk` lookup maps `{1}` (the id) → tempfile path. The Tempfile objects
|
|
190
|
+
stay referenced while fzf runs — a GC'd Tempfile unlinks its file under the
|
|
191
|
+
running preview.
|
|
169
192
|
|
|
170
193
|
## CLI
|
|
171
194
|
|
data/README.md
CHANGED
|
@@ -5,7 +5,7 @@
|
|
|
5
5
|
[](LICENSE)
|
|
6
6
|
|
|
7
7
|
Sync configuration between Macs (or to any host you can reach over ssh) from
|
|
8
|
-
self-documenting Markdown files.
|
|
8
|
+
self-documenting Markdown files. Designed for informed, interactive syncing.
|
|
9
9
|
|
|
10
10
|
A sync-file is a normal Markdown document. The prose is for you, or for an AI
|
|
11
11
|
assistant. twin only uses the fenced YAML blocks:
|
|
@@ -30,8 +30,11 @@ Own: conf.d/local.fish
|
|
|
30
30
|
```
|
|
31
31
|
````
|
|
32
32
|
|
|
33
|
-
|
|
34
|
-
|
|
33
|
+
twin runs when you ask, not in the background. It syncs what is unambiguous;
|
|
34
|
+
where the target has changes of its own, it stops and asks, diff in hand
|
|
35
|
+
(verified against content, never guessed from timestamps). At its core it's
|
|
36
|
+
just `rsync`, but a year later the file still tells you *why* you did it that
|
|
37
|
+
way.
|
|
35
38
|
|
|
36
39
|
That is the whole idea, and for most entries it stays as small as the excerpt
|
|
37
40
|
above. The rest of this README is long because twin also covers more complex
|
|
@@ -55,14 +58,17 @@ on both sides since the last sync, and paths that differ per host.
|
|
|
55
58
|
## Screenshots
|
|
56
59
|
|
|
57
60
|
Stage 1 — program picker. One row per program, color-coded status, indented
|
|
58
|
-
paths underneath
|
|
61
|
+
paths underneath. A name that appears in several sync-files
|
|
62
|
+
(the app in one, its config in another) is one entry, and a trailing bracket
|
|
63
|
+
group ties variants to their base name — `livesync [agent]` lists under
|
|
64
|
+
`livesync`. Stage 2 sections the merged paths per origin:
|
|
59
65
|
|
|
60
66
|

|
|
61
67
|
|
|
62
68
|
Stage 2 — multi-select over the paths of one program. The right pane shows a
|
|
63
69
|
compact preview of the relevant sync-file section, rendered by apex:
|
|
64
70
|
|
|
65
|
-

|
|
66
72
|
|
|
67
73
|
## Install
|
|
68
74
|
|
|
@@ -144,13 +150,13 @@ twin # picker — all programs across all sync-files
|
|
|
144
150
|
twin home.md # picker — one sync-file in sync_dir (by name)
|
|
145
151
|
twin ./some/dir/ # picker — all sync-files in a directory
|
|
146
152
|
twin list # plain listing
|
|
147
|
-
twin status #
|
|
153
|
+
twin status # what a sync would change, content-verified
|
|
148
154
|
twin sync -p grubber # sync one program by name pattern
|
|
149
155
|
twin sync --file=repos # sync all programs from a sync-file
|
|
150
156
|
twin sync --dry-run # preview without writing
|
|
151
157
|
twin sync -v # rsync's full output instead of just the changes
|
|
152
158
|
twin log # recent journal entries (-n N, --json)
|
|
153
|
-
twin doctor # check tools, renderers,
|
|
159
|
+
twin doctor # check tools, renderers, targets, remote hosts
|
|
154
160
|
twin --version # which twin is actually running
|
|
155
161
|
twin --help # show usage
|
|
156
162
|
```
|
|
@@ -210,7 +216,7 @@ rather than as an error.
|
|
|
210
216
|
|
|
211
217
|
| Field | Where | Meaning |
|
|
212
218
|
|---|---|---|
|
|
213
|
-
| `Program` | block | Group name; blocks sharing it sync together |
|
|
219
|
+
| `Program` | block | Group name; blocks sharing it sync together. A trailing `[…]` lists under the base name in the picker |
|
|
214
220
|
| `Path` | block | Path relative to `Source` (file or directory) |
|
|
215
221
|
| `Source` | either | Absolute base path on this machine |
|
|
216
222
|
| `Target` | either | Absolute base path, or `user@host:/path` for ssh |
|
|
@@ -223,6 +229,7 @@ rather than as an error.
|
|
|
223
229
|
| `Delete` | block | `true` mirrors deletions, with backups |
|
|
224
230
|
| `Cmd` | block | Shell command, run only when bytes actually moved |
|
|
225
231
|
| `Render` | block | `true` substitutes `{{tokens}}` instead of copying |
|
|
232
|
+
| `Verify` | block | `false` skips content verification — for entries too big (see below) |
|
|
226
233
|
|
|
227
234
|
### Exclude or Own?
|
|
228
235
|
|
|
@@ -265,8 +272,18 @@ pruning occasionally. It applies to `Delete` jobs only.
|
|
|
265
272
|
## Mounted volume or ssh
|
|
266
273
|
|
|
267
274
|
Both are first-class. `twin status`, the picker, `Exclude`/`Own`, `Delete` and
|
|
268
|
-
`Cmd` behave identically; remote paths are stat'ed
|
|
269
|
-
|
|
275
|
+
`Cmd` behave identically; remote paths are stat'ed (and, where timestamps
|
|
276
|
+
disagree, checksummed via `md5`/`md5sum`) in batched ssh round-trips per host,
|
|
277
|
+
and an unreachable host shows as `?` instead of failing the scan.
|
|
278
|
+
|
|
279
|
+
The remote side needs `rsync` and `/bin/sh`; `stat` (or `date -r`) and
|
|
280
|
+
`md5`/`md5sum` improve status from there, and `twin doctor` probes a host for
|
|
281
|
+
all of them before the first sync fails halfway. No particular login shell is
|
|
282
|
+
required — twin drives the far side through `/bin/sh` explicitly. That matters
|
|
283
|
+
more than it sounds: until 0.4.5 the stat script went to whatever shell the
|
|
284
|
+
account uses, so a host with fish (or any other non-POSIX shell) reported
|
|
285
|
+
every path as unreachable. It looked exactly like a machine being switched
|
|
286
|
+
off, which is why it went unnoticed for a while.
|
|
270
287
|
|
|
271
288
|
Two differences are real:
|
|
272
289
|
|
|
@@ -361,8 +378,21 @@ Two properties make this bearable day to day:
|
|
|
361
378
|
directory's mtime untouched, and `rsync -a` equalises those anyway. Twin asks
|
|
362
379
|
rsync what it would actually transfer instead of guessing from a directory.
|
|
363
380
|
|
|
364
|
-
`twin status`
|
|
365
|
-
|
|
381
|
+
`twin status` gives the same verdict without syncing: it runs the dry-runs per
|
|
382
|
+
entry and reports what would flow, what merely differs in timestamp, and what
|
|
383
|
+
changed on the target. The picker's first stage stays mtime-blind for
|
|
384
|
+
directory entries — it marks them `∘` (unverified) rather than guessing,
|
|
385
|
+
because the dry-runs would make it slow to open. Entering a program runs them
|
|
386
|
+
for just that program, so its directory rows show a real verdict.
|
|
387
|
+
|
|
388
|
+
One escape hatch: `Verify: false` opts an entry out of every content round —
|
|
389
|
+
the md5 checks, the status dry-runs, the picker's verification, the pre-sync
|
|
390
|
+
conflict listing. It exists for entries where the walk itself is the cost: a
|
|
391
|
+
`node_modules` tree on an SMB mount stats tens of thousands of files for one
|
|
392
|
+
verdict. Such an entry stays `∘`/mtime-based, and the picker's second stage
|
|
393
|
+
says so in its header rather than letting the `∘` pass as pending; a sync
|
|
394
|
+
still leaves newer target files alone (`--update` holds), they just aren't
|
|
395
|
+
itemised first.
|
|
366
396
|
|
|
367
397
|
## Automation
|
|
368
398
|
|
|
@@ -427,7 +457,7 @@ it differs from the current target, so a `Cmd` hook fires only on a real change.
|
|
|
427
457
|
## LiveSync LaunchAgent
|
|
428
458
|
|
|
429
459
|
```yaml
|
|
430
|
-
Program: livesync
|
|
460
|
+
Program: livesync [agent]
|
|
431
461
|
Source: "{{src.home}}/Automation/launchd"
|
|
432
462
|
Path: com.ralf.livesync.plist
|
|
433
463
|
Target: "{{dst.mount}}"
|
data/lib/twin/cli.rb
CHANGED
|
@@ -28,7 +28,7 @@ module Twin
|
|
|
28
28
|
[--force] [--skip-conflicts]
|
|
29
29
|
twin add <path> scaffold a new sync entry for a local path
|
|
30
30
|
twin log [-n N] [--json] recent journal entries (default 20)
|
|
31
|
-
twin doctor check tools, renderers,
|
|
31
|
+
twin doctor check tools, renderers, targets, remote hosts
|
|
32
32
|
twin --help show this message
|
|
33
33
|
twin --version show the running version
|
|
34
34
|
|
|
@@ -93,7 +93,7 @@ module Twin
|
|
|
93
93
|
exit 1
|
|
94
94
|
end
|
|
95
95
|
|
|
96
|
-
programs = Scanner.load_programs(cfg, file: file, show_all: false)
|
|
96
|
+
programs = Picker.merge_programs(Scanner.load_programs(cfg, file: file, show_all: false))
|
|
97
97
|
if programs.empty?
|
|
98
98
|
warn "no active programs found#{" in #{file}" if file}"
|
|
99
99
|
return
|
|
@@ -125,7 +125,7 @@ module Twin
|
|
|
125
125
|
break if $stdin.gets&.strip == "q"
|
|
126
126
|
|
|
127
127
|
# reload so status reflects what was just synced, stay on this program
|
|
128
|
-
programs = Scanner.load_programs(cfg, file: file, show_all: false)
|
|
128
|
+
programs = Picker.merge_programs(Scanner.load_programs(cfg, file: file, show_all: false))
|
|
129
129
|
selected_key = [program.name, program.sync_file]
|
|
130
130
|
end
|
|
131
131
|
end
|
|
@@ -150,6 +150,7 @@ module Twin
|
|
|
150
150
|
def cmd_status(cfg, args)
|
|
151
151
|
opts = parse_filter_opts(args)
|
|
152
152
|
programs = Scanner.load_programs(cfg, **opts.slice(:file, :label, :show_all))
|
|
153
|
+
verify_drift(cfg, programs)
|
|
153
154
|
|
|
154
155
|
if opts[:json]
|
|
155
156
|
puts JSON.pretty_generate(programs.map { |p| program_to_hash(p) })
|
|
@@ -163,19 +164,51 @@ module Twin
|
|
|
163
164
|
name = tty ? Picker.bold(p.name) : p.name
|
|
164
165
|
puts "#{icon} #{name}"
|
|
165
166
|
p.jobs.each do |j|
|
|
166
|
-
src = j.source_exists ? j.source_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
|
|
167
|
-
tgt = j.target_exists ? j.target_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
|
|
168
|
-
tgt = "(unreachable)" if j.target_unreachable
|
|
169
167
|
conflict = j.conflict ? (tty ? " #{Picker.colorize(:target_newer, "!")}" : " !") : ""
|
|
170
168
|
puts " #{j.path}#{conflict}"
|
|
171
|
-
|
|
172
|
-
|
|
169
|
+
if j.directory
|
|
170
|
+
# Directory mtimes prove nothing — the drift verdict replaces them.
|
|
171
|
+
note = if j.drift then drift_summary(j.drift)
|
|
172
|
+
elsif j.verify == false then "(not checked — Verify: false)"
|
|
173
|
+
else "(not checked — target unavailable)"
|
|
174
|
+
end
|
|
175
|
+
puts " #{note}"
|
|
176
|
+
else
|
|
177
|
+
src = j.source_exists ? j.source_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
|
|
178
|
+
tgt = j.target_exists ? j.target_mtime.strftime("%Y-%m-%d %H:%M:%S") : "(not found)"
|
|
179
|
+
tgt = "(unreachable)" if j.target_unreachable
|
|
180
|
+
puts " src #{src}"
|
|
181
|
+
puts " dst #{tgt}"
|
|
182
|
+
puts " content identical, timestamps differ" if j.content_equal
|
|
183
|
+
end
|
|
173
184
|
# Named, not hidden: these belong to the target on purpose.
|
|
174
185
|
puts " own #{j.owned.join(', ')}" unless j.owned.nil? || j.owned.empty?
|
|
175
186
|
end
|
|
176
187
|
end
|
|
177
188
|
end
|
|
178
189
|
|
|
190
|
+
# Ask rsync what a sync of each directory job would actually do.
|
|
191
|
+
def verify_drift(cfg, programs)
|
|
192
|
+
Twin::Conflict.fill_drift(cfg, programs.flat_map(&:jobs))
|
|
193
|
+
end
|
|
194
|
+
|
|
195
|
+
def drift_summary(d)
|
|
196
|
+
if d.in_sync?
|
|
197
|
+
note = d.time_only.empty? ? "" : " (#{d.time_only.size} timestamp-only)"
|
|
198
|
+
return "in sync#{note}"
|
|
199
|
+
end
|
|
200
|
+
parts = []
|
|
201
|
+
parts << "#{d.pending.size} to sync: #{list_some(d.pending)}" unless d.pending.empty?
|
|
202
|
+
unless d.conflicts.empty?
|
|
203
|
+
parts << "#{d.conflicts.size} changed on target: #{list_some(d.conflicts.map(&:rel))} (twin sync will ask)"
|
|
204
|
+
end
|
|
205
|
+
parts.join("; ")
|
|
206
|
+
end
|
|
207
|
+
|
|
208
|
+
def list_some(rels, max = 3)
|
|
209
|
+
rels.take(max).join(", ") + (rels.size > max ? ", …" : "")
|
|
210
|
+
end
|
|
211
|
+
|
|
179
212
|
# ── sync ───────────────────────────────────────────────────────────────────
|
|
180
213
|
|
|
181
214
|
def cmd_sync(cfg, args)
|
|
@@ -264,7 +297,10 @@ module Twin
|
|
|
264
297
|
return true if force
|
|
265
298
|
return false if skip_conflicts || dry_run
|
|
266
299
|
|
|
267
|
-
|
|
300
|
+
# Verify: false jobs skip the detection round (too big to walk); rsync's
|
|
301
|
+
# --update still keeps newer target files, they just aren't listed here.
|
|
302
|
+
conflicts = jobs.reject { |j| j.verify == false }
|
|
303
|
+
.flat_map { |j| Twin::Conflict.detect(cfg, j) }
|
|
268
304
|
return false if conflicts.empty?
|
|
269
305
|
|
|
270
306
|
report_conflicts(conflicts)
|
|
@@ -423,11 +459,13 @@ module Twin
|
|
|
423
459
|
if targets.empty?
|
|
424
460
|
puts " (no programs loaded)"
|
|
425
461
|
else
|
|
462
|
+
reachable_hosts = []
|
|
426
463
|
targets.each do |tgt|
|
|
427
464
|
if Twin::Remote.remote?(tgt)
|
|
428
465
|
host, = Twin::Remote.split(tgt)
|
|
429
466
|
if Twin::Remote.reachable?(host)
|
|
430
467
|
puts " ✓ #{tgt} (ssh)"
|
|
468
|
+
reachable_hosts << host
|
|
431
469
|
else
|
|
432
470
|
puts " ✗ #{tgt} (ssh: #{host} not reachable)"
|
|
433
471
|
ok = false
|
|
@@ -439,6 +477,7 @@ module Twin
|
|
|
439
477
|
ok = false
|
|
440
478
|
end
|
|
441
479
|
end
|
|
480
|
+
ok = doctor_remote_tools(reachable_hosts.uniq) && ok
|
|
442
481
|
end
|
|
443
482
|
rescue => e
|
|
444
483
|
puts " ✗ #{e.message}"
|
|
@@ -453,6 +492,42 @@ module Twin
|
|
|
453
492
|
system("command -v #{name} > /dev/null 2>&1")
|
|
454
493
|
end
|
|
455
494
|
|
|
495
|
+
# Probe each reachable ssh host once for what the far side must provide.
|
|
496
|
+
# A missing rsync fails doctor — the first sync would die mid-run with a
|
|
497
|
+
# raw protocol error. Missing stat/date or md5/md5sum only degrade
|
|
498
|
+
# (unknown mtimes, conservative conflicts), so they warn and say so.
|
|
499
|
+
def doctor_remote_tools(hosts)
|
|
500
|
+
return true if hosts.empty?
|
|
501
|
+
ok = true
|
|
502
|
+
puts "\nRemote tools"
|
|
503
|
+
hosts.each do |host|
|
|
504
|
+
tools = Twin::Remote.preflight(host)
|
|
505
|
+
if tools.nil?
|
|
506
|
+
puts " ✗ #{host} (probe failed)"
|
|
507
|
+
ok = false
|
|
508
|
+
next
|
|
509
|
+
end
|
|
510
|
+
problems = []
|
|
511
|
+
unless tools["rsync"]
|
|
512
|
+
problems << "rsync missing — syncs will fail (OpenWrt: opkg install rsync)"
|
|
513
|
+
ok = false
|
|
514
|
+
end
|
|
515
|
+
unless tools["stat"] || tools["date"]
|
|
516
|
+
problems << "no stat or date — remote mtimes read as unknown"
|
|
517
|
+
end
|
|
518
|
+
unless tools["md5"] || tools["md5sum"]
|
|
519
|
+
problems << "no md5 or md5sum — timestamp-only files stay flagged as conflicts"
|
|
520
|
+
end
|
|
521
|
+
if problems.empty?
|
|
522
|
+
have = ["rsync", tools["stat"] ? "stat" : "date", tools["md5"] ? "md5" : "md5sum"]
|
|
523
|
+
puts " ✓ #{host} (#{have.join(', ')})"
|
|
524
|
+
else
|
|
525
|
+
puts " #{tools["rsync"] ? "⚠" : "✗"} #{host} #{problems.join('; ')}"
|
|
526
|
+
end
|
|
527
|
+
end
|
|
528
|
+
ok
|
|
529
|
+
end
|
|
530
|
+
|
|
456
531
|
# ── option parsing ─────────────────────────────────────────────────────────
|
|
457
532
|
|
|
458
533
|
def parse_filter_opts(args)
|
|
@@ -501,6 +576,11 @@ module Twin
|
|
|
501
576
|
h[:status] = j.status
|
|
502
577
|
h[:source_mtime] = j.source_mtime&.iso8601
|
|
503
578
|
h[:target_mtime] = j.target_mtime&.iso8601
|
|
579
|
+
h[:drift] = j.drift && {
|
|
580
|
+
pending: j.drift.pending,
|
|
581
|
+
time_only: j.drift.time_only,
|
|
582
|
+
conflicts: j.drift.conflicts.map(&:rel),
|
|
583
|
+
}
|
|
504
584
|
h
|
|
505
585
|
end
|
|
506
586
|
end
|
data/lib/twin/conflict.rb
CHANGED
|
@@ -1,40 +1,43 @@
|
|
|
1
1
|
require "digest"
|
|
2
|
+
require "set"
|
|
2
3
|
|
|
3
4
|
require_relative "remote"
|
|
4
5
|
|
|
5
6
|
module Twin
|
|
6
|
-
#
|
|
7
|
-
#
|
|
7
|
+
# What would a sync actually change? mtimes cannot answer that — a
|
|
8
|
+
# directory's mtime records the last entry added or removed (after a sync:
|
|
9
|
+
# the sync itself), a file's mtime moves on a `cat >` copy that changed
|
|
10
|
+
# nothing, and rsync equalises directory mtimes on every run anyway. So we
|
|
11
|
+
# ask rsync, which has the answer already and knows its own matching rules
|
|
12
|
+
# better than any reimplementation would:
|
|
8
13
|
#
|
|
9
|
-
#
|
|
10
|
-
#
|
|
11
|
-
#
|
|
12
|
-
# matters: editing a file in place leaves its directory's mtime untouched, and
|
|
13
|
-
# `rsync -a` equalises directory mtimes on every run anyway. A hand-edit on
|
|
14
|
-
# the target is therefore invisible to it.
|
|
14
|
+
# 1. What would a *forced* run transfer?
|
|
15
|
+
# One dry-run without --update. Empty means fully in sync — the common
|
|
16
|
+
# case and the cheap exit: one stat-walk per job and no more.
|
|
15
17
|
#
|
|
16
|
-
#
|
|
17
|
-
#
|
|
18
|
+
# 2. Of that, what does --update hold back?
|
|
19
|
+
# A second dry-run with --update. Everything the forced run would move
|
|
20
|
+
# but the normal one would not is exactly the set --update protects —
|
|
21
|
+
# files the target owns more recently.
|
|
18
22
|
#
|
|
19
|
-
#
|
|
20
|
-
#
|
|
21
|
-
#
|
|
22
|
-
#
|
|
23
|
-
#
|
|
24
|
-
# 2. Of those, which differ in content?
|
|
25
|
-
# Only these are worth asking about. A file that is merely newer — same
|
|
26
|
-
# bytes, later timestamp — is noise, and noise is what turns a prompt
|
|
27
|
-
# into a reflex. Sync both sides of a tree in either order and you get
|
|
28
|
-
# dozens of them.
|
|
29
|
-
#
|
|
30
|
-
# The first dry-run is also the cheap exit: when a forced run would move
|
|
31
|
-
# nothing, the job is fully in sync and the second run is skipped. That is the
|
|
32
|
-
# common case, so a quiet sync costs one extra stat-walk per job and no more.
|
|
23
|
+
# 3. What differs only in timestamp?
|
|
24
|
+
# rsync's itemize flags say so (">f..t" — time, same size). Those files
|
|
25
|
+
# are checksummed; identical content is noise, not drift, and noise is
|
|
26
|
+
# what turns a prompt into a reflex. Sync both sides of a tree in
|
|
27
|
+
# either order and you get dozens of them.
|
|
33
28
|
module Conflict
|
|
34
|
-
#
|
|
35
|
-
#
|
|
36
|
-
#
|
|
37
|
-
#
|
|
29
|
+
# The classified outcome of a forced dry-run for one job.
|
|
30
|
+
# pending — relative paths a sync would genuinely change
|
|
31
|
+
# time_only — same bytes, different timestamp; a sync merely aligns them
|
|
32
|
+
# conflicts — Entry list: target-side changes whose content differs
|
|
33
|
+
Drift = Struct.new(:pending, :time_only, :conflicts, keyword_init: true) do
|
|
34
|
+
def in_sync? = pending.empty? && conflicts.empty?
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
# One file the target owns more recently than the source, with content
|
|
38
|
+
# that actually differs (or could not be checked — remote md5 failed —
|
|
39
|
+
# which is treated as differing, because guessing the other way would
|
|
40
|
+
# overwrite work). target_mtime is nil for remote targets.
|
|
38
41
|
Entry = Struct.new(:job, :rel, :source_path, :target_path,
|
|
39
42
|
:source_mtime, :target_mtime, keyword_init: true) do
|
|
40
43
|
def age_delta
|
|
@@ -43,55 +46,145 @@ module Twin
|
|
|
43
46
|
end
|
|
44
47
|
end
|
|
45
48
|
|
|
46
|
-
# rsync --itemize-changes line → relative path. Change lines
|
|
47
|
-
# update type and a file type (">f.st...... lib/foo.rb")
|
|
48
|
-
#
|
|
49
|
-
|
|
49
|
+
# rsync --itemize-changes line → [flags, relative path]. Change lines
|
|
50
|
+
# start with an update type and a file type (">f.st...... lib/foo.rb") or
|
|
51
|
+
# "*deleting"; attribute-only lines (leading ".") and the surrounding
|
|
52
|
+
# prose do not match — they describe no change a sync would make.
|
|
53
|
+
ENTRY_LINE = /\A(\*deleting|[<>ch][fdLDS]\S*)\s+(.+?)\s*\z/
|
|
50
54
|
|
|
51
55
|
module_function
|
|
52
56
|
|
|
53
|
-
#
|
|
57
|
+
# Classify one job's drift by asking rsync. nil for render jobs (they
|
|
58
|
+
# compare content already and never use --update) and when a side is
|
|
59
|
+
# missing (status reports that on its own).
|
|
60
|
+
def drift(cfg, job)
|
|
61
|
+
return nil if job.render
|
|
62
|
+
return nil unless job.source_exists && job.target_exists
|
|
63
|
+
|
|
64
|
+
forced = itemized(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: true))
|
|
65
|
+
return Drift.new(pending: [], time_only: [], conflicts: []) if forced.empty?
|
|
66
|
+
|
|
67
|
+
normal_rels = itemized(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: false))
|
|
68
|
+
.map { |e| e[:rel] }.to_set
|
|
69
|
+
equal = equality_map(job, forced.filter_map { |e| e[:rel] if e[:kind] == :time })
|
|
70
|
+
|
|
71
|
+
assemble(forced, normal_rels, equal) do |rel|
|
|
72
|
+
conflict_entry(job, rel)
|
|
73
|
+
end
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
# Target-side changes worth asking about, in the order rsync reports them.
|
|
54
77
|
# Empty when --update holds nothing back, or holds back only identical files.
|
|
55
78
|
def detect(cfg, job)
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
return [] unless job.source_exists && job.target_exists
|
|
79
|
+
drift(cfg, job)&.conflicts || []
|
|
80
|
+
end
|
|
59
81
|
|
|
60
|
-
|
|
82
|
+
# Directory jobs whose drift is still unknown and answerable — both sides
|
|
83
|
+
# present, target reachable, not opted out via Verify: false. File jobs
|
|
84
|
+
# are content-checked by the scanner and need no second look.
|
|
85
|
+
def drift_candidates(jobs)
|
|
86
|
+
jobs.select do |j|
|
|
87
|
+
j.drift.nil? && j.verify != false && j.directory && j.active == 1 &&
|
|
88
|
+
!j.target_unreachable && j.source_exists && j.target_exists
|
|
89
|
+
end
|
|
61
90
|
end
|
|
62
91
|
|
|
63
|
-
#
|
|
64
|
-
#
|
|
65
|
-
def
|
|
66
|
-
|
|
67
|
-
return
|
|
92
|
+
# Fill `drift` on every candidate by asking rsync — the dry-runs are
|
|
93
|
+
# subprocess I/O, so a few run in parallel.
|
|
94
|
+
def fill_drift(cfg, jobs)
|
|
95
|
+
jobs = drift_candidates(jobs)
|
|
96
|
+
return if jobs.empty?
|
|
97
|
+
|
|
98
|
+
queue = Queue.new
|
|
99
|
+
jobs.each { |j| queue << j }
|
|
100
|
+
Array.new([4, jobs.size].min) do
|
|
101
|
+
Thread.new do
|
|
102
|
+
loop do
|
|
103
|
+
j = begin
|
|
104
|
+
queue.pop(true)
|
|
105
|
+
rescue ThreadError
|
|
106
|
+
break
|
|
107
|
+
end
|
|
108
|
+
j.drift = drift(cfg, j)
|
|
109
|
+
end
|
|
110
|
+
end
|
|
111
|
+
end.each(&:join)
|
|
112
|
+
end
|
|
68
113
|
|
|
69
|
-
|
|
70
|
-
|
|
114
|
+
# Pure assembly of a Drift from parsed entries. A file the normal run
|
|
115
|
+
# would also transfer flows source→target as intended; one only the
|
|
116
|
+
# forced run would touch is being held back by --update — the target owns
|
|
117
|
+
# it more recently. Content decides whether that is a conflict or noise.
|
|
118
|
+
def assemble(forced, normal_rels, equal)
|
|
119
|
+
d = Drift.new(pending: [], time_only: [], conflicts: [])
|
|
120
|
+
forced.each do |e|
|
|
121
|
+
rel = e[:rel]
|
|
122
|
+
case e[:kind]
|
|
123
|
+
when :deleted, :new then d.pending << rel
|
|
124
|
+
when :content
|
|
125
|
+
normal_rels.include?(rel) ? d.pending << rel : d.conflicts << yield(rel)
|
|
126
|
+
when :time
|
|
127
|
+
if equal[rel]
|
|
128
|
+
d.time_only << rel
|
|
129
|
+
elsif normal_rels.include?(rel)
|
|
130
|
+
d.pending << rel
|
|
131
|
+
else
|
|
132
|
+
d.conflicts << yield(rel)
|
|
133
|
+
end
|
|
134
|
+
end
|
|
135
|
+
end
|
|
136
|
+
d
|
|
71
137
|
end
|
|
72
138
|
|
|
73
|
-
|
|
139
|
+
# Run rsync and parse its itemize output into [{rel:, kind:}, ...].
|
|
140
|
+
def itemized(args)
|
|
74
141
|
output, status = Twin::Sync.run(args)
|
|
75
142
|
return [] unless status.success?
|
|
76
143
|
output.lines.filter_map do |line|
|
|
77
|
-
m =
|
|
144
|
+
m = ENTRY_LINE.match(line)
|
|
78
145
|
next unless m
|
|
79
|
-
|
|
80
|
-
next if
|
|
81
|
-
rel
|
|
146
|
+
kind = classify(m[1])
|
|
147
|
+
next if kind == :attrs
|
|
148
|
+
{ rel: m[2], kind: kind }
|
|
82
149
|
end
|
|
83
150
|
end
|
|
84
151
|
|
|
85
|
-
#
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
152
|
+
# Itemize flags → what kind of change this is.
|
|
153
|
+
# :deleted — target-only file, removed by Delete: true
|
|
154
|
+
# :new — does not exist on the target yet (files and directories)
|
|
155
|
+
# :content — size differs, so the bytes certainly do
|
|
156
|
+
# :time — timestamp only; content equality still to be determined
|
|
157
|
+
# :attrs — permissions/owner, no change a sync-file cares about
|
|
158
|
+
def classify(flags)
|
|
159
|
+
return :deleted if flags == "*deleting"
|
|
160
|
+
body = flags[2..].to_s
|
|
161
|
+
return :new if body.include?("+")
|
|
162
|
+
return :content if body.include?("s")
|
|
163
|
+
return :time if body.include?("t") || body.include?("T")
|
|
164
|
+
:attrs
|
|
165
|
+
end
|
|
89
166
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
167
|
+
# Content equality for the :time candidates, {rel => bool}. Local pairs
|
|
168
|
+
# are compared directly; remote targets get one batched md5 round per job
|
|
169
|
+
# (STAT_SCRIPT-style), and an unanswered path counts as differing.
|
|
170
|
+
def equality_map(job, rels)
|
|
171
|
+
return {} if rels.empty?
|
|
172
|
+
if job.remote?
|
|
173
|
+
_host, rbase = Twin::Remote.split(job.target_path)
|
|
174
|
+
remote_paths = rels.to_h { |r| [r, job.directory ? File.join(rbase, r) : rbase] }
|
|
175
|
+
sums = Twin::Remote.md5_paths(Twin::Remote.split(job.target)[0], remote_paths.values) || {}
|
|
176
|
+
rels.to_h do |r|
|
|
177
|
+
local = local_md5(resolve(job.source_path, r))
|
|
178
|
+
[r, !local.nil? && sums[remote_paths[r]] == local]
|
|
179
|
+
end
|
|
180
|
+
else
|
|
181
|
+
rels.to_h { |r| [r, same_content?(resolve(job.source_path, r), resolve(job.target_path, r))] }
|
|
93
182
|
end
|
|
183
|
+
end
|
|
94
184
|
|
|
185
|
+
def conflict_entry(job, rel)
|
|
186
|
+
src = resolve(job.source_path, rel)
|
|
187
|
+
tgt = job.remote? ? job.target_path : resolve(job.target_path, rel)
|
|
95
188
|
Entry.new(
|
|
96
189
|
job: job, rel: rel, source_path: src, target_path: tgt,
|
|
97
190
|
source_mtime: mtime(src), target_mtime: job.remote? ? nil : mtime(tgt),
|
|
@@ -114,6 +207,12 @@ module Twin
|
|
|
114
207
|
|
|
115
208
|
def digest(path) = Digest::SHA256.file(path).hexdigest
|
|
116
209
|
|
|
210
|
+
def local_md5(path)
|
|
211
|
+
Digest::MD5.file(path).hexdigest
|
|
212
|
+
rescue Errno::ENOENT, Errno::EACCES, Errno::EISDIR
|
|
213
|
+
nil
|
|
214
|
+
end
|
|
215
|
+
|
|
117
216
|
def mtime(path)
|
|
118
217
|
File.mtime(path)
|
|
119
218
|
rescue Errno::ENOENT, Errno::EACCES
|
data/lib/twin/picker.rb
CHANGED
|
@@ -2,6 +2,7 @@ require "open3"
|
|
|
2
2
|
require "tempfile"
|
|
3
3
|
|
|
4
4
|
require_relative "preview"
|
|
5
|
+
require_relative "conflict"
|
|
5
6
|
|
|
6
7
|
module Twin
|
|
7
8
|
# Two-step fzf picker:
|
|
@@ -14,6 +15,11 @@ module Twin
|
|
|
14
15
|
source_newer: "→",
|
|
15
16
|
target_newer: "←",
|
|
16
17
|
in_sync: "✓",
|
|
18
|
+
# Directory jobs in stage 1: their mtimes prove nothing, and the
|
|
19
|
+
# dry-runs that would prove something are too slow for the full picker.
|
|
20
|
+
# No claim instead of a wrong one — stage 2 verifies its one program
|
|
21
|
+
# on open, and `twin status` has the full verdict.
|
|
22
|
+
unverified: "∘",
|
|
17
23
|
missing_target: "!",
|
|
18
24
|
missing_source: "!",
|
|
19
25
|
both_missing: "✗",
|
|
@@ -25,6 +31,7 @@ module Twin
|
|
|
25
31
|
source_newer: "\e[33m", # yellow
|
|
26
32
|
target_newer: "\e[36m", # cyan
|
|
27
33
|
in_sync: "\e[32m", # green
|
|
34
|
+
unverified: "\e[2m", # dim
|
|
28
35
|
missing_target: "\e[31m", # red
|
|
29
36
|
missing_source: "\e[31m", # red
|
|
30
37
|
both_missing: "\e[31m", # red
|
|
@@ -44,6 +51,29 @@ module Twin
|
|
|
44
51
|
|
|
45
52
|
# ── Stufe 1: program picker ───────────────────────────────────────────────
|
|
46
53
|
|
|
54
|
+
# A trailing bracket group ties a program to its base name by convention:
|
|
55
|
+
# "livesync [agent]" lists under "livesync". Only the suffix form counts —
|
|
56
|
+
# brackets elsewhere are just a name.
|
|
57
|
+
BRACKET_SUFFIX = /\s*\[[^\]]*\]\z/
|
|
58
|
+
|
|
59
|
+
def base_name(name) = name.sub(BRACKET_SUFFIX, "").strip
|
|
60
|
+
|
|
61
|
+
# One stage-1 entry per base name (case-insensitive), even when it appears
|
|
62
|
+
# in several sync-files or as bracketed variants — "everything of
|
|
63
|
+
# fileview" is one decision. The list is sorted by name; document order
|
|
64
|
+
# only orders the *jobs*, program order carries no meaning. Purely a
|
|
65
|
+
# picker view: the data model, `status`, `sync -p` and JSON keep the
|
|
66
|
+
# per-file programs. Jobs stay grouped per file in document order, so a
|
|
67
|
+
# file's closing Cmd block still fires after its own paths.
|
|
68
|
+
def merge_programs(programs)
|
|
69
|
+
programs.group_by { |p| base_name(p.name).downcase }.values.map do |group|
|
|
70
|
+
next group.first if group.size == 1
|
|
71
|
+
name = group.find { |p| !p.name.match?(BRACKET_SUFFIX) }&.name ||
|
|
72
|
+
base_name(group.first.name)
|
|
73
|
+
Program.new(name: name, jobs: group.flat_map(&:jobs))
|
|
74
|
+
end.sort_by { |p| p.name.downcase }
|
|
75
|
+
end
|
|
76
|
+
|
|
47
77
|
# Multi-line entries (NUL-separated). Header line per program, indented
|
|
48
78
|
# body lines per job. Returns the selected Program or nil.
|
|
49
79
|
def pick_program(programs)
|
|
@@ -85,15 +115,45 @@ module Twin
|
|
|
85
115
|
jobs = program.jobs
|
|
86
116
|
return [] if jobs.empty?
|
|
87
117
|
|
|
118
|
+
# Stage 2 covers one program, so the dry-runs that are too slow for the
|
|
119
|
+
# whole picker are affordable here — directory rows get a real verdict
|
|
120
|
+
# instead of `∘`. Already-verified jobs are skipped on re-entry.
|
|
121
|
+
if Conflict.drift_candidates(jobs).any?
|
|
122
|
+
$stderr.print dim("verifying directories …")
|
|
123
|
+
Conflict.fill_drift(cfg, jobs)
|
|
124
|
+
$stderr.print "\r\e[K"
|
|
125
|
+
end
|
|
126
|
+
|
|
88
127
|
path_width = jobs.map { |j| j.path.length }.max
|
|
89
128
|
tempfiles = write_compact_previews(program, jobs)
|
|
90
129
|
preview_cmd, mapfile = build_apex_preview_cmd(tempfiles, cfg)
|
|
91
130
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
131
|
+
# A merged program spans sync-files (and bracketed variants); a dim
|
|
132
|
+
# section row marks each block, named by its file — plus the original
|
|
133
|
+
# program name when it differs from the merged one. Section rows carry
|
|
134
|
+
# "-" instead of an index — fzf can't make them unselectable, but they
|
|
135
|
+
# map to no job, so toggling one is inert.
|
|
136
|
+
section = ->(j) { [j.program, j.sync_file] }
|
|
137
|
+
multi = jobs.map(§ion).uniq.size > 1
|
|
138
|
+
rows = []
|
|
139
|
+
prev = nil
|
|
140
|
+
jobs.each_with_index do |j, i|
|
|
141
|
+
if multi && section.(j) != prev
|
|
142
|
+
file = File.basename(j.sync_file.to_s)
|
|
143
|
+
label = j.program == program.name ? file : "#{j.program} · #{file}"
|
|
144
|
+
rows << "-\t#{dim("[#{label}]")}"
|
|
145
|
+
prev = section.(j)
|
|
146
|
+
end
|
|
147
|
+
icon = STATUS_ICONS[j.status] || "?"
|
|
148
|
+
line = "#{icon} #{j.path.ljust(path_width)} #{job_delta(j)}"
|
|
149
|
+
rows << "#{i}\t#{colorize(j.status, line)}"
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
header = "#{program.name} — Tab toggles, Enter confirms"
|
|
153
|
+
unverified = jobs.select { |j| j.verify == false }
|
|
154
|
+
unless unverified.empty?
|
|
155
|
+
header += "\n\e[33m⚠ content checks off (Verify: false): " \
|
|
156
|
+
"#{unverified.map(&:path).join(", ")}\e[0m"
|
|
97
157
|
end
|
|
98
158
|
|
|
99
159
|
fzf = [
|
|
@@ -101,7 +161,7 @@ module Twin
|
|
|
101
161
|
"--multi", "--ansi",
|
|
102
162
|
"--delimiter=\t", "--with-nth=2",
|
|
103
163
|
"--prompt=#{program.name} > ",
|
|
104
|
-
"--header=#{
|
|
164
|
+
"--header=#{header}",
|
|
105
165
|
"--preview=#{preview_cmd}",
|
|
106
166
|
"--preview-window=right:60%:wrap",
|
|
107
167
|
"--height=100%", "--reverse",
|
|
@@ -109,45 +169,55 @@ module Twin
|
|
|
109
169
|
"--color=bg+:-1,hl+:reverse",
|
|
110
170
|
]
|
|
111
171
|
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
172
|
+
loop do
|
|
173
|
+
output, status = Open3.capture2(SH_ENV, *fzf, stdin_data: rows.join("\n"))
|
|
174
|
+
return :back if status.exitstatus == 130 # ESC / Ctrl-C
|
|
175
|
+
return [] unless status.success?
|
|
176
|
+
return [] if output.strip.empty?
|
|
116
177
|
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
178
|
+
selected = output.lines.filter_map do |line|
|
|
179
|
+
idx = line.split("\t", 2).first
|
|
180
|
+
jobs[idx.to_i] if idx&.match?(/\A\d+\z/)
|
|
181
|
+
end
|
|
182
|
+
return selected unless selected.empty?
|
|
183
|
+
# Enter landed on a section header (or only headers were toggled):
|
|
184
|
+
# nothing real was accepted — reopen rather than read it as "exit".
|
|
120
185
|
end
|
|
121
186
|
ensure
|
|
122
|
-
tempfiles&.each_value { |
|
|
123
|
-
|
|
187
|
+
tempfiles&.each_value { |f| f.close! rescue nil }
|
|
188
|
+
mapfile&.close! rescue nil
|
|
124
189
|
end
|
|
125
190
|
|
|
126
|
-
# Write per-job compact-preview markdown to tempfiles. Returns
|
|
191
|
+
# Write per-job compact-preview markdown to tempfiles. Returns
|
|
192
|
+
# {idx => Tempfile}. The Tempfile objects (not just their paths) must stay
|
|
193
|
+
# referenced while fzf runs: a GC'd Tempfile unlinks its file, and the
|
|
194
|
+
# preview command would find nothing to render. Each job's excerpt comes
|
|
195
|
+
# from its own sync-file — a merged program spans several.
|
|
127
196
|
def write_compact_previews(program, jobs)
|
|
128
197
|
result = {}
|
|
129
198
|
jobs.each_with_index do |job, i|
|
|
130
|
-
compact = Preview.extract_compact(
|
|
199
|
+
compact = Preview.extract_compact(job.sync_file, job.path)
|
|
131
200
|
f = Tempfile.new(["twin-#{i}-", ".md"])
|
|
132
201
|
f.write(compact)
|
|
133
202
|
f.close
|
|
134
|
-
result[i] = f
|
|
203
|
+
result[i] = f
|
|
135
204
|
end
|
|
136
205
|
result
|
|
137
206
|
end
|
|
138
207
|
|
|
139
|
-
# Returns [preview_cmd,
|
|
208
|
+
# Returns [preview_cmd, mapfile]; the caller holds the Tempfile until fzf
|
|
209
|
+
# is done (GC would unlink it) and closes it afterwards.
|
|
140
210
|
def build_apex_preview_cmd(tempfiles, cfg)
|
|
141
211
|
# Map idx → file via a small TSV, awk picks the right one for {1}.
|
|
142
212
|
mapfile = Tempfile.new(["twin-map-", ".tsv"])
|
|
143
|
-
tempfiles.each { |i,
|
|
213
|
+
tempfiles.each { |i, f| mapfile.puts("#{i}\t#{f.path}") }
|
|
144
214
|
mapfile.close
|
|
145
215
|
|
|
146
216
|
render_cmd = pick_renderer(cfg)
|
|
147
217
|
|
|
148
218
|
cmd = %(F=$(awk -v id={1} -F'\\t' '$1==id {print $2}' #{mapfile.path}); ) +
|
|
149
219
|
%([ -n "$F" ] && #{render_cmd})
|
|
150
|
-
[cmd, mapfile
|
|
220
|
+
[cmd, mapfile]
|
|
151
221
|
end
|
|
152
222
|
|
|
153
223
|
# Pick the first available markdown renderer.
|
|
@@ -178,22 +248,27 @@ module Twin
|
|
|
178
248
|
|
|
179
249
|
def render_program_entry(program, name_width, path_width)
|
|
180
250
|
icon = colorize(program.status, STATUS_ICONS[program.status] || "?")
|
|
181
|
-
|
|
251
|
+
files = program.jobs.map { |j| File.basename(j.sync_file.to_s) }.uniq.join(" · ")
|
|
182
252
|
count = program.active_jobs.size
|
|
183
253
|
total = program.jobs.size
|
|
184
254
|
header = "#{icon} #{bold(program.name.ljust(name_width))} " \
|
|
185
|
-
"#{dim("(#{count}/#{total})")} #{dim("[#{
|
|
255
|
+
"#{dim("(#{count}/#{total})")} #{dim("[#{files}]")}"
|
|
186
256
|
|
|
187
257
|
body = program.jobs.map do |j|
|
|
188
258
|
icon = STATUS_ICONS[j.status] || "?"
|
|
189
|
-
|
|
190
|
-
line = " #{icon} #{j.path.ljust(path_width)} #{delta}"
|
|
259
|
+
line = " #{icon} #{j.path.ljust(path_width)} #{job_delta(j)}"
|
|
191
260
|
colorize(j.status, line)
|
|
192
261
|
end
|
|
193
262
|
|
|
194
263
|
([header] + body).join("\n")
|
|
195
264
|
end
|
|
196
265
|
|
|
266
|
+
# A directory's mtime delta would mislead (it moves on every sync), so
|
|
267
|
+
# directory jobs show none.
|
|
268
|
+
def job_delta(job)
|
|
269
|
+
job.directory ? "" : format_delta(job.source_mtime, job.target_mtime)
|
|
270
|
+
end
|
|
271
|
+
|
|
197
272
|
def format_delta(sm, tm)
|
|
198
273
|
return "" if sm.nil? || tm.nil?
|
|
199
274
|
seconds = (sm - tm).to_i
|
data/lib/twin/remote.rb
CHANGED
|
@@ -26,22 +26,40 @@ module Twin
|
|
|
26
26
|
|
|
27
27
|
# Stat many paths in one ssh round-trip. Paths go over stdin (one per
|
|
28
28
|
# line), the remote loop answers "path<TAB>epoch" or "path<TAB>-" for
|
|
29
|
-
# missing ones. Tries BSD stat
|
|
29
|
+
# missing ones. Tries BSD stat, then GNU, then BusyBox `date -r` — covers
|
|
30
|
+
# macOS, Linux and OpenWrt. Auf dem Brume (25.08.2026) fehlte `stat`
|
|
31
|
+
# komplett: die leere Substitution wurde als Epoche 0 geparst und `twin
|
|
32
|
+
# status` zeigte 1970 statt "unbekannt". `date -r` kennt kein `--`;
|
|
33
|
+
# verkraftbar, weil hier nur absolute Zielpfade ankommen.
|
|
30
34
|
# Returns {path => Time or nil-if-missing}, or nil when ssh itself failed.
|
|
35
|
+
# POSIX sh, and it must stay free of single quotes: it is handed to the
|
|
36
|
+
# remote side wrapped in '...' so that *any* login shell passes it through
|
|
37
|
+
# literally. The usual POSIX escape for an embedded quote ('\'') is parsed
|
|
38
|
+
# differently by fish, so the rule here is simply not to need it — hence
|
|
39
|
+
# double quotes around the printf formats.
|
|
31
40
|
STAT_SCRIPT = <<~SH.freeze
|
|
32
41
|
while IFS= read -r p; do
|
|
33
42
|
if [ -e "$p" ]; then
|
|
34
|
-
|
|
43
|
+
m=$(stat -f %m -- "$p" 2>/dev/null || stat -c %Y -- "$p" 2>/dev/null || date -r "$p" +%s)
|
|
44
|
+
printf "%s\t%s\n" "$p" "$m"
|
|
35
45
|
else
|
|
36
|
-
printf
|
|
46
|
+
printf "%s\t-\n" "$p"
|
|
37
47
|
fi
|
|
38
48
|
done
|
|
39
49
|
SH
|
|
40
50
|
|
|
41
51
|
def stat_paths(host, paths)
|
|
42
52
|
return {} if paths.empty?
|
|
53
|
+
# Explicitly through /bin/sh. ssh hands the command to the *login shell*
|
|
54
|
+
# on the far side, and that is not necessarily POSIX: on a machine whose
|
|
55
|
+
# shell is fish, the bare script dies on "while IFS= read -r p; do" and
|
|
56
|
+
# twin read the failure as "host not answering" — every ssh target showed
|
|
57
|
+
# as :unreachable in status and the picker, for as long as SSH targets
|
|
58
|
+
# existed (#8, 2026-07-17). Found and fixed 2026-08-25.
|
|
59
|
+
raise "STAT_SCRIPT must not contain single quotes" if STAT_SCRIPT.include?("'")
|
|
60
|
+
|
|
43
61
|
out, _err, status = Open3.capture3(
|
|
44
|
-
"ssh", *SSH_OPTS, host, STAT_SCRIPT,
|
|
62
|
+
"ssh", *SSH_OPTS, host, "/bin/sh -c '#{STAT_SCRIPT}'",
|
|
45
63
|
stdin_data: paths.join("\n") + "\n"
|
|
46
64
|
)
|
|
47
65
|
return nil unless status.success?
|
|
@@ -50,13 +68,90 @@ module Twin
|
|
|
50
68
|
out.each_line do |line|
|
|
51
69
|
path, mtime = line.chomp.split("\t", 2)
|
|
52
70
|
next unless path && mtime
|
|
53
|
-
|
|
71
|
+
# Nur echte Epochen als Zeit werten. Scheitert die ganze stat-Kette,
|
|
72
|
+
# ist das Feld leer — das ist "unbekannt", nicht 1970.
|
|
73
|
+
result[path] = mtime.match?(/\A\d+\z/) ? Time.at(mtime.to_i) : nil
|
|
54
74
|
end
|
|
55
75
|
result
|
|
56
76
|
rescue Errno::ENOENT
|
|
57
77
|
nil # ssh not installed
|
|
58
78
|
end
|
|
59
79
|
|
|
80
|
+
# Checksum many paths in one ssh round-trip, same shape as stat_paths:
|
|
81
|
+
# paths over stdin, "path<TAB>md5" back, "-" for anything that is not a
|
|
82
|
+
# regular file. Tries BSD md5 first, then md5sum (GNU, BusyBox) — covers
|
|
83
|
+
# macOS, Linux and OpenWrt. md5sum prints "hash path"; the parameter
|
|
84
|
+
# expansion keeps only the first word. Same single-quote rule as
|
|
85
|
+
# STAT_SCRIPT, and MD5 is drift detection here, not cryptography.
|
|
86
|
+
# Returns {path => hex or nil-if-unreadable}, or nil when ssh itself failed.
|
|
87
|
+
MD5_SCRIPT = <<~SH.freeze
|
|
88
|
+
while IFS= read -r p; do
|
|
89
|
+
if [ -f "$p" ]; then
|
|
90
|
+
m=$(md5 -q "$p" 2>/dev/null || md5sum "$p" 2>/dev/null)
|
|
91
|
+
m=${m%% *}
|
|
92
|
+
printf "%s\t%s\n" "$p" "$m"
|
|
93
|
+
else
|
|
94
|
+
printf "%s\t-\n" "$p"
|
|
95
|
+
fi
|
|
96
|
+
done
|
|
97
|
+
SH
|
|
98
|
+
|
|
99
|
+
def md5_paths(host, paths)
|
|
100
|
+
return {} if paths.empty?
|
|
101
|
+
raise "MD5_SCRIPT must not contain single quotes" if MD5_SCRIPT.include?("'")
|
|
102
|
+
|
|
103
|
+
out, _err, status = Open3.capture3(
|
|
104
|
+
"ssh", *SSH_OPTS, host, "/bin/sh -c '#{MD5_SCRIPT}'",
|
|
105
|
+
stdin_data: paths.join("\n") + "\n"
|
|
106
|
+
)
|
|
107
|
+
return nil unless status.success?
|
|
108
|
+
|
|
109
|
+
result = {}
|
|
110
|
+
out.each_line do |line|
|
|
111
|
+
path, sum = line.chomp.split("\t", 2)
|
|
112
|
+
next unless path && sum
|
|
113
|
+
result[path] = sum.match?(/\A\h{32}\z/) ? sum.downcase : nil
|
|
114
|
+
end
|
|
115
|
+
result
|
|
116
|
+
rescue Errno::ENOENT
|
|
117
|
+
nil # ssh not installed
|
|
118
|
+
end
|
|
119
|
+
|
|
120
|
+
# Which of the tools twin relies on exist on the far side? One ssh
|
|
121
|
+
# round-trip per host, for `twin doctor`. rsync carries the sync itself;
|
|
122
|
+
# stat/date feed the batched mtime round; md5/md5sum feed the content
|
|
123
|
+
# check. Same single-quote rule as the other batch scripts.
|
|
124
|
+
# Returns {tool => present?}, or nil when ssh itself failed.
|
|
125
|
+
PREFLIGHT_SCRIPT = <<~SH.freeze
|
|
126
|
+
for t in rsync stat date md5 md5sum; do
|
|
127
|
+
if command -v "$t" >/dev/null 2>&1; then
|
|
128
|
+
printf "%s\tok\n" "$t"
|
|
129
|
+
else
|
|
130
|
+
printf "%s\t-\n" "$t"
|
|
131
|
+
fi
|
|
132
|
+
done
|
|
133
|
+
SH
|
|
134
|
+
|
|
135
|
+
def preflight(host)
|
|
136
|
+
raise "PREFLIGHT_SCRIPT must not contain single quotes" if PREFLIGHT_SCRIPT.include?("'")
|
|
137
|
+
|
|
138
|
+
out, _err, status = Open3.capture3("ssh", *SSH_OPTS, host, "/bin/sh -c '#{PREFLIGHT_SCRIPT}'")
|
|
139
|
+
return nil unless status.success?
|
|
140
|
+
parse_preflight(out)
|
|
141
|
+
rescue Errno::ENOENT
|
|
142
|
+
nil # ssh not installed
|
|
143
|
+
end
|
|
144
|
+
|
|
145
|
+
def parse_preflight(out)
|
|
146
|
+
result = {}
|
|
147
|
+
out.each_line do |line|
|
|
148
|
+
tool, state = line.chomp.split("\t", 2)
|
|
149
|
+
next unless tool && state
|
|
150
|
+
result[tool] = state == "ok"
|
|
151
|
+
end
|
|
152
|
+
result
|
|
153
|
+
end
|
|
154
|
+
|
|
60
155
|
# Create a directory on the remote side (mkdir -p equivalent).
|
|
61
156
|
def mkdir_p(host, dir)
|
|
62
157
|
_out, _err, status = Open3.capture3("ssh", *SSH_OPTS, host, "mkdir", "-p", shellesc(dir))
|
data/lib/twin/scanner.rb
CHANGED
|
@@ -2,6 +2,7 @@ require "json"
|
|
|
2
2
|
require "open3"
|
|
3
3
|
|
|
4
4
|
require_relative "remote"
|
|
5
|
+
require_relative "conflict"
|
|
5
6
|
|
|
6
7
|
module Twin
|
|
7
8
|
# One YAML block from a sync-file, enriched with live filesystem state.
|
|
@@ -9,7 +10,7 @@ module Twin
|
|
|
9
10
|
:program, :path, :description, :active, :excludes, :owned, :label,
|
|
10
11
|
:source, :target, :cmd, :delete, :render, :render_outdated, :target_path_field, :sync_file,
|
|
11
12
|
:source_exists, :target_exists, :source_mtime, :target_mtime, :conflict,
|
|
12
|
-
:target_unreachable,
|
|
13
|
+
:target_unreachable, :directory, :content_equal, :drift, :verify,
|
|
13
14
|
keyword_init: true,
|
|
14
15
|
) do
|
|
15
16
|
def source_path = File.join(source, path)
|
|
@@ -29,6 +30,19 @@ module Twin
|
|
|
29
30
|
# Render jobs compare by content, not mtime — a rendered target's mtime
|
|
30
31
|
# bears no relation to the template's.
|
|
31
32
|
return render_outdated ? :source_newer : :in_sync if render
|
|
33
|
+
# Directory jobs carry no mtime verdict at all: a directory's mtime
|
|
34
|
+
# records the last entry added or removed — after a sync that is the
|
|
35
|
+
# sync itself, and even an *excluded* file moves it. `twin status`
|
|
36
|
+
# fills in `drift` by asking rsync; without it the honest answer is
|
|
37
|
+
# "not checked", not a guess.
|
|
38
|
+
if directory
|
|
39
|
+
return :unverified unless drift
|
|
40
|
+
return :target_newer if drift.conflicts.any?
|
|
41
|
+
return :source_newer if drift.pending.any?
|
|
42
|
+
return :in_sync
|
|
43
|
+
end
|
|
44
|
+
# Same bytes under a newer timestamp (a `cat >` copy) is not drift.
|
|
45
|
+
return :in_sync if content_equal
|
|
32
46
|
return :target_newer if conflict
|
|
33
47
|
return :in_sync if source_mtime.nil? || target_mtime.nil?
|
|
34
48
|
delta = source_mtime - target_mtime
|
|
@@ -50,7 +64,7 @@ module Twin
|
|
|
50
64
|
# Aggregate status across jobs — worst first.
|
|
51
65
|
def status
|
|
52
66
|
states = jobs.map(&:status)
|
|
53
|
-
%i[unreachable both_missing missing_source missing_target target_newer source_newer disabled in_sync]
|
|
67
|
+
%i[unreachable both_missing missing_source missing_target target_newer source_newer unverified disabled in_sync]
|
|
54
68
|
.find { |s| states.include?(s) } || :in_sync
|
|
55
69
|
end
|
|
56
70
|
|
|
@@ -103,9 +117,32 @@ module Twin
|
|
|
103
117
|
mtime = stats[rpath]
|
|
104
118
|
j.target_exists = !mtime.nil?
|
|
105
119
|
j.target_mtime = mtime
|
|
106
|
-
j.conflict = j.source_exists && mtime && j.source_mtime &&
|
|
120
|
+
j.conflict = !j.directory && j.source_exists && mtime && j.source_mtime &&
|
|
107
121
|
mtime - j.source_mtime >= 60
|
|
108
122
|
end
|
|
123
|
+
verify_remote_file_content(host, host_jobs)
|
|
124
|
+
end
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
# Remote counterpart of the local content check in build_job: file jobs
|
|
128
|
+
# whose mtimes drifted get one batched md5 round per host. Identical
|
|
129
|
+
# content clears the conflict — the timestamps merely disagree.
|
|
130
|
+
def verify_remote_file_content(host, host_jobs)
|
|
131
|
+
candidates = host_jobs.select do |j|
|
|
132
|
+
j.verify != false && !j.directory && !j.render &&
|
|
133
|
+
j.source_exists && j.target_exists &&
|
|
134
|
+
j.source_mtime && j.target_mtime && (j.target_mtime - j.source_mtime).abs >= 60
|
|
135
|
+
end
|
|
136
|
+
return if candidates.empty?
|
|
137
|
+
|
|
138
|
+
sums = Twin::Remote.md5_paths(host, candidates.map { |j| Twin::Remote.split(j.target_path).last })
|
|
139
|
+
return if sums.nil?
|
|
140
|
+
|
|
141
|
+
candidates.each do |j|
|
|
142
|
+
remote_sum = sums[Twin::Remote.split(j.target_path).last]
|
|
143
|
+
next if remote_sum.nil?
|
|
144
|
+
j.content_equal = remote_sum == Twin::Conflict.local_md5(j.source_path)
|
|
145
|
+
j.conflict = false if j.content_equal
|
|
109
146
|
end
|
|
110
147
|
end
|
|
111
148
|
|
|
@@ -147,6 +184,11 @@ module Twin
|
|
|
147
184
|
return nil if path.empty? || source.empty? || target.empty?
|
|
148
185
|
|
|
149
186
|
render = r["Render"] == true
|
|
187
|
+
# Verify: false — no content verification for this entry, ever: too big
|
|
188
|
+
# or too remote for md5/dry-run rounds (a node_modules tree over SMB).
|
|
189
|
+
# Status falls back to mtime for files and stays ∘ for directories; the
|
|
190
|
+
# pre-sync conflict check skips it (--update still protects the target).
|
|
191
|
+
verify = r["Verify"] != false
|
|
150
192
|
excludes = split_list(r["Exclude"])
|
|
151
193
|
# Own: paths inside the sync scope that the TARGET owns — machine-specific
|
|
152
194
|
# config the source must never clobber. Same rsync effect as Exclude, but
|
|
@@ -170,8 +212,22 @@ module Twin
|
|
|
170
212
|
# target). conflict stays false so the mtime conflict-warning skips them.
|
|
171
213
|
render_outdated = render ? render_outdated?(src_full, tgt_full, vars, path) : nil
|
|
172
214
|
|
|
215
|
+
# rsync mirrors directories, so a directory source means a directory
|
|
216
|
+
# target — also for remote jobs, whose far side can't be inspected here.
|
|
217
|
+
directory = src_exists && File.directory?(src_full)
|
|
218
|
+
|
|
219
|
+
# A file job whose mtimes drifted apart may still hold the same bytes
|
|
220
|
+
# (a `cat >` copy before the first twin run). Check before judging;
|
|
221
|
+
# a directory's own mtime is judged not at all (see Job#status).
|
|
222
|
+
content_equal = nil
|
|
223
|
+
if verify && !render && !remote && !directory && src_exists && tgt_exists &&
|
|
224
|
+
src_mtime && tgt_mtime && (tgt_mtime - src_mtime).abs >= 60
|
|
225
|
+
content_equal = Twin::Conflict.same_content?(src_full, tgt_full)
|
|
226
|
+
end
|
|
227
|
+
|
|
173
228
|
# Same 60s tolerance as Job#status, so mtime jitter never flags a conflict.
|
|
174
|
-
conflict = !render &&
|
|
229
|
+
conflict = !render && !directory && !content_equal &&
|
|
230
|
+
src_exists && tgt_exists && tgt_mtime && src_mtime &&
|
|
175
231
|
tgt_mtime - src_mtime >= 60
|
|
176
232
|
|
|
177
233
|
Job.new(
|
|
@@ -196,6 +252,9 @@ module Twin
|
|
|
196
252
|
target_mtime: tgt_mtime,
|
|
197
253
|
conflict: !!conflict,
|
|
198
254
|
target_unreachable: false,
|
|
255
|
+
directory: directory,
|
|
256
|
+
content_equal: content_equal,
|
|
257
|
+
verify: verify,
|
|
199
258
|
)
|
|
200
259
|
end
|
|
201
260
|
|
data/lib/twin/sync.rb
CHANGED
|
@@ -130,6 +130,9 @@ module Twin
|
|
|
130
130
|
|
|
131
131
|
xfr = !dry_run && transferred?(output)
|
|
132
132
|
|
|
133
|
+
# conflict is content-verified by the scanner and only ever set on file
|
|
134
|
+
# jobs — a directory's mtime says nothing, so directory jobs surface
|
|
135
|
+
# target-side changes through the pre-run prompt instead.
|
|
133
136
|
if job.conflict && !xfr && !dry_run && !force
|
|
134
137
|
output += "\nskipped: target is newer, source not synced"
|
|
135
138
|
end
|
data/lib/twin/version.rb
CHANGED