mark-twin 0.1.3 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/ARCHITECTURE.md +80 -12
- data/README.md +71 -7
- data/lib/twin/cli.rb +79 -1
- data/lib/twin/config.rb +26 -1
- data/lib/twin/scanner.rb +54 -24
- data/lib/twin/sync.rb +84 -8
- data/lib/twin/template.rb +36 -0
- data/lib/twin/version.rb +1 -1
- data/lib/twin.rb +1 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 39f52a8341611b4a17bbeaffae64e21e7ffc2123f1b766df494eb422db697cb6
|
|
4
|
+
data.tar.gz: 224e40d39832efc3542c1c3de00e4c6fe30d01c72ca4f84ff5bc9868f15cedd3
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 8f217e753b18810cad48b2e6c2f3360a3cc84e4f6449dd14f9f62aae4243b55d44a4d2b8321a8f60228c4ca696e97106f9cdca1e6a54f404436af284130609dd
|
|
7
|
+
data.tar.gz: 301088d4bd68082461e9e8d0730d1c921336256e8fe3f3a755f918c6e189fc4238fc4bd7c3f7e4e8879474f9736ef4272a5117ce8b3b97316983031e0656201b
|
data/ARCHITECTURE.md
CHANGED
|
@@ -9,14 +9,17 @@ sync-files (.md)
|
|
|
9
9
|
grubber parse Markdown, extract YAML blocks, merge frontmatter
|
|
10
10
|
│
|
|
11
11
|
▼
|
|
12
|
+
Template substitute {{tokens}} in record fields (no-op without hosts)
|
|
13
|
+
│
|
|
14
|
+
▼
|
|
12
15
|
Scanner load_jobs → list[Job] → group(jobs) → list[Program]
|
|
13
16
|
│
|
|
14
|
-
├──▶ CLI list / status / sync
|
|
17
|
+
├──▶ CLI list / status / sync / doctor
|
|
15
18
|
│
|
|
16
19
|
└──▶ Picker fzf + apex preview, returns selected Program
|
|
17
20
|
│
|
|
18
21
|
▼
|
|
19
|
-
Sync rsync per Job, mount check, Cmd hook
|
|
22
|
+
Sync rsync (or render) per Job, mount check, Cmd hook
|
|
20
23
|
```
|
|
21
24
|
|
|
22
25
|
## Package layout
|
|
@@ -24,9 +27,10 @@ sync-files (.md)
|
|
|
24
27
|
```
|
|
25
28
|
lib/twin/
|
|
26
29
|
version.rb
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
+
template.rb {{token}} substitution + render-file helper
|
|
31
|
+
config.rb ~/.config/twin/config.yaml loader; host table → var_map
|
|
32
|
+
scanner.rb Job, Program structs; grubber + template + stat → grouped Programs
|
|
33
|
+
sync.rb rsync / render execution, mount check, post-sync hook
|
|
30
34
|
picker.rb fzf wrapper with apex preview
|
|
31
35
|
cli.rb subcommand dispatcher
|
|
32
36
|
|
|
@@ -39,12 +43,15 @@ test/test_pure.rb
|
|
|
39
43
|
**Job** — one YAML block:
|
|
40
44
|
|
|
41
45
|
```
|
|
42
|
-
program, path, description, active, excludes, label, source, target, cmd,
|
|
46
|
+
program, path, description, active, excludes, label, source, target, cmd,
|
|
47
|
+
delete, render, render_outdated, target_path_field, sync_file,
|
|
43
48
|
source_exists, target_exists, source_mtime, target_mtime, conflict
|
|
44
49
|
```
|
|
45
50
|
|
|
46
51
|
`Job#status` → one of `disabled / both_missing / missing_source / missing_target /
|
|
47
|
-
target_newer / in_sync / source_newer`.
|
|
52
|
+
target_newer / in_sync / source_newer`. Render jobs derive status from content
|
|
53
|
+
(`render_outdated`), not mtime; non-render jobs ignore those fields.
|
|
54
|
+
`Job#target_path` joins `target` with `target_path_field || path`.
|
|
48
55
|
|
|
49
56
|
**Program** — group of Jobs sharing a `program` name:
|
|
50
57
|
|
|
@@ -64,9 +71,20 @@ sync_dir: /path/to/sync-files
|
|
|
64
71
|
global_excludes: [".DS_Store", ".git/"]
|
|
65
72
|
apex_theme: ralf
|
|
66
73
|
apex_width: 80
|
|
74
|
+
|
|
75
|
+
host: mini # which host twin runs as
|
|
76
|
+
target: book # default sync target
|
|
77
|
+
hosts:
|
|
78
|
+
mini: { home: /Volumes/lightning/users/extern, git: /Volumes/lightning/Git }
|
|
79
|
+
book: { home: /Users/ralf, git: /Users/ralf/git, mount: /Volumes/ralf }
|
|
67
80
|
```
|
|
68
81
|
|
|
69
|
-
|
|
82
|
+
`Config#var_map` flattens the host table for the (`host` → `target`) pair into
|
|
83
|
+
`{ "src.home" => …, "dst.home" => …, "dst.mount" => … }` — empty when no hosts
|
|
84
|
+
are configured (templating inert). See the **Templating** section.
|
|
85
|
+
|
|
86
|
+
Environment overrides: `TWIN_SYNC_DIR` (sync_dir), `TWIN_CONFIG` (config path),
|
|
87
|
+
`TWIN_HOST` (host).
|
|
70
88
|
|
|
71
89
|
## Sync-files
|
|
72
90
|
|
|
@@ -77,6 +95,32 @@ record is self-contained.
|
|
|
77
95
|
Multiple blocks may share the same `Program` value — these are treated as
|
|
78
96
|
one logical unit by twin.
|
|
79
97
|
|
|
98
|
+
## Templating
|
|
99
|
+
|
|
100
|
+
Substitution sits between grubber and `build_job`, so grubber never sees
|
|
101
|
+
`{{tokens}}` and stays untouched. `Scanner.load_jobs` calls
|
|
102
|
+
`Template.substitute_record` on each record's path-bearing fields (`Source`,
|
|
103
|
+
`Target`, `Path`, `Target-Path`, `Exclude`, `Cmd`) using `cfg.var_map`. This
|
|
104
|
+
*must* run before `build_job`, which immediately `stat`s the resolved paths.
|
|
105
|
+
Unknown `{{token}}` → hard error (never sync a half-rendered path).
|
|
106
|
+
|
|
107
|
+
Three namespaces, one fixed meaning each:
|
|
108
|
+
|
|
109
|
+
- `{{src.*}}` — the running host's own paths (read side, `Source:`).
|
|
110
|
+
- `{{dst.mount}}` — where the target is mounted here (write side, `Target:`).
|
|
111
|
+
- `{{dst.*}}` — the target's native paths, used in **rendered file content**.
|
|
112
|
+
|
|
113
|
+
The mount/native split is the crux: a file written to `/Volumes/ralf/…` but read
|
|
114
|
+
by the target machine must contain `/Users/ralf/…`. Path fields and file content
|
|
115
|
+
draw from different namespaces, so a token never means two things.
|
|
116
|
+
|
|
117
|
+
`{{` opens a YAML flow mapping, so templated values must be quoted in the
|
|
118
|
+
sync-file (`Source: "{{src.home}}"`) — as in Ansible. Without a `hosts` table
|
|
119
|
+
`var_map` is empty and substitution is a no-op.
|
|
120
|
+
|
|
121
|
+
See [docs/templating-design.md](docs/templating-design.md) for the full
|
|
122
|
+
rationale.
|
|
123
|
+
|
|
80
124
|
## Picker
|
|
81
125
|
|
|
82
126
|
Two stages:
|
|
@@ -107,6 +151,11 @@ File argument resolution (`twin <arg>` and `--file=<arg>`):
|
|
|
107
151
|
Unknown options (anything starting with `-` that isn't `--help`) print an
|
|
108
152
|
error pointing at `twin --help` and exit 1.
|
|
109
153
|
|
|
154
|
+
`twin doctor` checks required tools (grubber, rsync, fzf), optional renderers
|
|
155
|
+
(apex, glow, bat), templating (host/target resolve, every `{{token}}` resolves),
|
|
156
|
+
and whether all configured sync targets are mounted. Exits 1 if any required
|
|
157
|
+
check fails.
|
|
158
|
+
|
|
110
159
|
## Sync
|
|
111
160
|
|
|
112
161
|
Before syncing:
|
|
@@ -116,14 +165,33 @@ Before syncing:
|
|
|
116
165
|
2. **Conflict warning** — emits stderr listing jobs where the target is
|
|
117
166
|
newer than the source. Continues anyway (`rsync --update` skips them).
|
|
118
167
|
|
|
119
|
-
Then per Job:
|
|
168
|
+
Then per Job, **rsync path** (non-render):
|
|
120
169
|
|
|
121
170
|
```
|
|
122
|
-
rsync -av --update [--exclude=...]* src/ tgt/
|
|
171
|
+
rsync -av --itemize-changes --update [--delete] [--exclude=...]* src/ tgt/
|
|
123
172
|
```
|
|
124
173
|
|
|
125
|
-
|
|
126
|
-
|
|
174
|
+
`--delete` is added when the Job has `delete: true` (from `Delete: true`).
|
|
175
|
+
`--itemize-changes` makes change detection deterministic: `Sync.transferred?`
|
|
176
|
+
matches itemize lines (`/\A[<>ch*][fdLDS]/` — `>f…`, `cd…`, `*deleting`),
|
|
177
|
+
covering files, directories and deletions, with no scraping of rsync's prose.
|
|
178
|
+
|
|
179
|
+
If `Cmd` is set, it runs via `sh -c` after rsync — but only when something was
|
|
180
|
+
actually transferred. No-op syncs (target up to date, or target newer and
|
|
181
|
+
skipped by `--update`) leave the hook silent. On failure the exit code is
|
|
182
|
+
included in the output and the job is marked failed.
|
|
183
|
+
|
|
184
|
+
When a job has a known conflict (`target_newer`) and nothing transferred, the
|
|
185
|
+
output notes `"skipped: target is newer, source not synced"`.
|
|
186
|
+
|
|
187
|
+
**Render path** (`render: true`) — `Sync.render_job`. Templates can't be
|
|
188
|
+
rsync'd byte-for-byte, so instead: read source, substitute `{{dst.*}}` in the
|
|
189
|
+
content, compare against the current target bytes, write only if they differ.
|
|
190
|
+
`changed` drives the same `Cmd` gate. Content-hash comparison (not mtime)
|
|
191
|
+
sidesteps the `--update` trap — a freshly rendered temp is always "newer".
|
|
192
|
+
Render is file-only; a directory source is an error. `twin status` mirrors this:
|
|
193
|
+
render-job status comes from the same content comparison (`render_outdated`),
|
|
194
|
+
so a stale target with a matching mtime is still flagged `source_newer`.
|
|
127
195
|
|
|
128
196
|
## External dependencies
|
|
129
197
|
|
data/README.md
CHANGED
|
@@ -119,6 +119,7 @@ twin status # listing with source/target mtimes
|
|
|
119
119
|
twin sync -p grubber # sync one program by name pattern
|
|
120
120
|
twin sync --file=repos # sync all programs from a sync-file
|
|
121
121
|
twin sync --dry-run # preview without writing
|
|
122
|
+
twin doctor # check tools, renderers, and sync targets
|
|
122
123
|
twin --help # show usage
|
|
123
124
|
```
|
|
124
125
|
|
|
@@ -145,7 +146,8 @@ global_excludes:
|
|
|
145
146
|
# apex_code_highlight_theme: dark
|
|
146
147
|
```
|
|
147
148
|
|
|
148
|
-
Environment overrides: `TWIN_SYNC_DIR`, `TWIN_CONFIG
|
|
149
|
+
Environment overrides: `TWIN_SYNC_DIR`, `TWIN_CONFIG`, `TWIN_HOST` (which host
|
|
150
|
+
twin runs as — lets one config serve both machines).
|
|
149
151
|
|
|
150
152
|
## Sync-files
|
|
151
153
|
|
|
@@ -181,12 +183,74 @@ Frontmatter fields (`Active`, `Label`, `Source`, `Target`) are merged into
|
|
|
181
183
|
every block by grubber. Multiple blocks can share the same `Program` — twin
|
|
182
184
|
groups them and treats the program as the unit of selection.
|
|
183
185
|
|
|
184
|
-
The optional `Cmd` field
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
186
|
+
The optional `Cmd` field runs an arbitrary shell command after a successful
|
|
187
|
+
sync — typically a `curl` to a local automation endpoint like
|
|
188
|
+
[mi.lan](https://github.com/rhsev/mi.lan) to reload a program, restart a
|
|
189
|
+
service, or notify another machine. The command only runs when rsync actually
|
|
190
|
+
transferred bytes; no-op syncs skip it. See the Helix entry in
|
|
191
|
+
[examples/home.md](examples/home.md).
|
|
192
|
+
|
|
193
|
+
The optional `Delete: true` field adds `--delete` to the rsync invocation,
|
|
194
|
+
so files removed from the source are also removed on the target. Useful for
|
|
195
|
+
directory syncs where the target should mirror the source exactly.
|
|
196
|
+
|
|
197
|
+
## Templating
|
|
198
|
+
|
|
199
|
+
Some configs differ per machine — a LaunchAgent plist that points at
|
|
200
|
+
`/Volumes/lightning/…` on one Mac and `/Users/ralf/…` on another, a
|
|
201
|
+
`settings.json` with a device-specific id. Those used to fall out of twin and
|
|
202
|
+
get hand-maintained. Templating folds them back into one source of truth.
|
|
203
|
+
|
|
204
|
+
Define a host table in `~/.config/twin/config.yaml`:
|
|
205
|
+
|
|
206
|
+
```yaml
|
|
207
|
+
host: mini # which machine twin runs on
|
|
208
|
+
target: book # the machine being synced to
|
|
209
|
+
|
|
210
|
+
hosts:
|
|
211
|
+
mini: { home: /Volumes/lightning/users/extern, git: /Volumes/lightning/Git }
|
|
212
|
+
book: { home: /Users/ralf, git: /Users/ralf/git, mount: /Volumes/ralf }
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
That exposes three sets of `{{tokens}}`, each with one fixed meaning:
|
|
216
|
+
|
|
217
|
+
| Token | Resolves to | Use in |
|
|
218
|
+
|---|---|---|
|
|
219
|
+
| `{{src.home}}`, `{{src.git}}`, … | the running host's own paths | `Source:` (read side) |
|
|
220
|
+
| `{{dst.mount}}` | where the target is mounted here (`/Volumes/ralf`) | `Target:` (write side) |
|
|
221
|
+
| `{{dst.home}}`, `{{dst.git}}`, … | the target's *native* paths | rendered file **content** |
|
|
222
|
+
|
|
223
|
+
The distinction matters: a file *written* to the mount (`/Volumes/ralf/…`) but
|
|
224
|
+
*read* by the target machine must contain that machine's native paths
|
|
225
|
+
(`/Users/ralf/…`). `{{dst.mount}}` and `{{dst.home}}` keep the two apart.
|
|
226
|
+
|
|
227
|
+
> **Quote templated values.** `{{` at the start of a YAML value collides with
|
|
228
|
+
> YAML flow-mapping syntax, so write `Source: "{{src.home}}"`, not
|
|
229
|
+
> `Source: {{src.home}}` — exactly as in Ansible.
|
|
230
|
+
|
|
231
|
+
`Render: true` turns a block from copy into *render*: twin reads the source as a
|
|
232
|
+
template, substitutes `{{…}}` in its **content**, and writes the result only if
|
|
233
|
+
it differs from the current target (so a `Cmd` hook fires only on a real change).
|
|
234
|
+
`Target-Path:` overrides the target-side relative path when it differs from the
|
|
235
|
+
source layout:
|
|
236
|
+
|
|
237
|
+
````markdown
|
|
238
|
+
## LiveSync LaunchAgent
|
|
239
|
+
|
|
240
|
+
```yaml
|
|
241
|
+
Program: livesync-agent
|
|
242
|
+
Source: "{{src.home}}/Automation/launchd"
|
|
243
|
+
Path: com.ralf.livesync.plist
|
|
244
|
+
Target: "{{dst.mount}}"
|
|
245
|
+
Target-Path: Library/LaunchAgents/com.ralf.livesync.plist
|
|
246
|
+
Render: true
|
|
247
|
+
Cmd: curl -s http://mi.lan/livesync-reload
|
|
248
|
+
```
|
|
249
|
+
````
|
|
250
|
+
|
|
251
|
+
`twin doctor` checks that every `{{token}}` across your sync-files resolves, and
|
|
252
|
+
`twin status` compares rendered output by content (not mtime). Without a `hosts`
|
|
253
|
+
table, templating is inert and literal-path sync-files behave exactly as before.
|
|
190
254
|
|
|
191
255
|
## Design
|
|
192
256
|
|
data/lib/twin/cli.rb
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
require "optparse"
|
|
2
2
|
require "json"
|
|
3
3
|
require "set"
|
|
4
|
+
require "time" # Time#iso8601 for --json output
|
|
4
5
|
|
|
5
6
|
require_relative "config"
|
|
6
7
|
require_relative "scanner"
|
|
@@ -21,6 +22,7 @@ module Twin
|
|
|
21
22
|
twin list [--all] [--label X] [--file X] [--json]
|
|
22
23
|
twin status [--all] [--label X] [--file X] [--json]
|
|
23
24
|
twin sync [-p PATTERN] [--label X] [--file X] [--all] [--dry-run]
|
|
25
|
+
twin doctor check tools, renderers, and sync targets
|
|
24
26
|
twin --help show this message
|
|
25
27
|
|
|
26
28
|
FILE ARGUMENT:
|
|
@@ -44,6 +46,7 @@ module Twin
|
|
|
44
46
|
when "list" then cmd_list(cfg, argv.drop(1))
|
|
45
47
|
when "status" then cmd_status(cfg, argv.drop(1))
|
|
46
48
|
when "sync" then cmd_sync(cfg, argv.drop(1))
|
|
49
|
+
when "doctor" then cmd_doctor(cfg)
|
|
47
50
|
when "-h", "--help", "help"
|
|
48
51
|
puts USAGE
|
|
49
52
|
when /\A-/
|
|
@@ -188,13 +191,88 @@ module Twin
|
|
|
188
191
|
|
|
189
192
|
puts "→ #{program.name}"
|
|
190
193
|
jobs.each do |job|
|
|
191
|
-
success, output = Twin::Sync.run_job(cfg, job, dry_run: dry_run)
|
|
194
|
+
success, output, = Twin::Sync.run_job(cfg, job, dry_run: dry_run)
|
|
192
195
|
puts " • #{job.path}"
|
|
193
196
|
puts output.gsub(/^/, " ") if output && !output.strip.empty?
|
|
194
197
|
warn " error syncing #{job.path}" unless success
|
|
195
198
|
end
|
|
196
199
|
end
|
|
197
200
|
|
|
201
|
+
# ── doctor ─────────────────────────────────────────────────────────────────
|
|
202
|
+
|
|
203
|
+
def cmd_doctor(cfg)
|
|
204
|
+
ok = true
|
|
205
|
+
|
|
206
|
+
puts "Tools"
|
|
207
|
+
%w[grubber rsync fzf].each do |bin|
|
|
208
|
+
if tool_available?(bin)
|
|
209
|
+
puts " ✓ #{bin}"
|
|
210
|
+
else
|
|
211
|
+
puts " ✗ #{bin} (required — not found in PATH)"
|
|
212
|
+
ok = false
|
|
213
|
+
end
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
puts "\nRenderers (preview)"
|
|
217
|
+
found_renderer = false
|
|
218
|
+
%w[apex glow bat].each do |bin|
|
|
219
|
+
if tool_available?(bin)
|
|
220
|
+
puts " ✓ #{bin}"
|
|
221
|
+
found_renderer = true
|
|
222
|
+
else
|
|
223
|
+
puts " – #{bin} (not installed)"
|
|
224
|
+
end
|
|
225
|
+
end
|
|
226
|
+
puts " ⚠ no renderer found — file preview will fall back to cat" unless found_renderer
|
|
227
|
+
|
|
228
|
+
puts "\nTemplating"
|
|
229
|
+
if cfg.hosts.empty?
|
|
230
|
+
puts " – no hosts configured"
|
|
231
|
+
else
|
|
232
|
+
%i[host target].each do |attr|
|
|
233
|
+
name = cfg.send(attr)
|
|
234
|
+
if name.empty?
|
|
235
|
+
puts " ✗ #{attr} not set in config"
|
|
236
|
+
ok = false
|
|
237
|
+
elsif cfg.hosts.key?(name)
|
|
238
|
+
puts " ✓ #{attr}: #{name}"
|
|
239
|
+
else
|
|
240
|
+
puts " ✗ #{attr} #{name.inspect} not found in hosts"
|
|
241
|
+
ok = false
|
|
242
|
+
end
|
|
243
|
+
end
|
|
244
|
+
end
|
|
245
|
+
|
|
246
|
+
puts "\nTargets"
|
|
247
|
+
begin
|
|
248
|
+
programs = Scanner.load_programs(cfg, show_all: true)
|
|
249
|
+
puts " ✓ all template tokens resolved" unless cfg.hosts.empty?
|
|
250
|
+
targets = programs.flat_map(&:jobs).map(&:target).uniq.sort
|
|
251
|
+
if targets.empty?
|
|
252
|
+
puts " (no programs loaded)"
|
|
253
|
+
else
|
|
254
|
+
targets.each do |tgt|
|
|
255
|
+
if Twin::Sync.mounted?(tgt)
|
|
256
|
+
puts " ✓ #{tgt}"
|
|
257
|
+
else
|
|
258
|
+
puts " ✗ #{tgt} (not mounted)"
|
|
259
|
+
ok = false
|
|
260
|
+
end
|
|
261
|
+
end
|
|
262
|
+
end
|
|
263
|
+
rescue => e
|
|
264
|
+
puts " ✗ #{e.message}"
|
|
265
|
+
ok = false
|
|
266
|
+
end
|
|
267
|
+
|
|
268
|
+
puts ok ? "\nAll checks passed." : "\nSome checks failed."
|
|
269
|
+
exit 1 unless ok
|
|
270
|
+
end
|
|
271
|
+
|
|
272
|
+
def tool_available?(name)
|
|
273
|
+
system("command -v #{name} > /dev/null 2>&1")
|
|
274
|
+
end
|
|
275
|
+
|
|
198
276
|
# ── option parsing ─────────────────────────────────────────────────────────
|
|
199
277
|
|
|
200
278
|
def parse_filter_opts(args)
|
data/lib/twin/config.rb
CHANGED
|
@@ -5,7 +5,8 @@ module Twin
|
|
|
5
5
|
class Config
|
|
6
6
|
attr_accessor :sync_dir, :global_excludes,
|
|
7
7
|
:apex_theme, :apex_width,
|
|
8
|
-
:apex_code_highlight, :apex_code_highlight_theme
|
|
8
|
+
:apex_code_highlight, :apex_code_highlight_theme,
|
|
9
|
+
:hosts, :host, :target
|
|
9
10
|
|
|
10
11
|
DEFAULTS = {
|
|
11
12
|
"global_excludes" => [".DS_Store"],
|
|
@@ -13,6 +14,9 @@ module Twin
|
|
|
13
14
|
"apex_width" => nil,
|
|
14
15
|
"apex_code_highlight" => nil,
|
|
15
16
|
"apex_code_highlight_theme" => nil,
|
|
17
|
+
"hosts" => {},
|
|
18
|
+
"host" => "",
|
|
19
|
+
"target" => "",
|
|
16
20
|
}.freeze
|
|
17
21
|
|
|
18
22
|
def initialize(data = {})
|
|
@@ -23,6 +27,27 @@ module Twin
|
|
|
23
27
|
@apex_width = merged["apex_width"]
|
|
24
28
|
@apex_code_highlight = merged["apex_code_highlight"]
|
|
25
29
|
@apex_code_highlight_theme = merged["apex_code_highlight_theme"]
|
|
30
|
+
@hosts = merged["hosts"] || {}
|
|
31
|
+
@host = ENV["TWIN_HOST"] || merged["host"].to_s
|
|
32
|
+
@target = merged["target"].to_s
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# Build the flat substitution map {src.home => ..., dst.home => ..., dst.mount => ...}.
|
|
36
|
+
# Returns empty hash when no hosts are configured (substitution becomes a no-op).
|
|
37
|
+
def var_map
|
|
38
|
+
return {} if hosts.empty?
|
|
39
|
+
raise "host not set in config" if host.empty?
|
|
40
|
+
raise "target not set in config" if target.empty?
|
|
41
|
+
|
|
42
|
+
src_host = hosts[host]
|
|
43
|
+
dst_host = hosts[target]
|
|
44
|
+
raise "unknown host #{host.inspect} (not in hosts)" unless src_host
|
|
45
|
+
raise "unknown target #{target.inspect} (not in hosts)" unless dst_host
|
|
46
|
+
|
|
47
|
+
map = {}
|
|
48
|
+
src_host.each { |k, v| map["src.#{k}"] = v.to_s }
|
|
49
|
+
dst_host.each { |k, v| map["dst.#{k}"] = v.to_s }
|
|
50
|
+
map
|
|
26
51
|
end
|
|
27
52
|
|
|
28
53
|
def self.load
|
data/lib/twin/scanner.rb
CHANGED
|
@@ -5,18 +5,21 @@ module Twin
|
|
|
5
5
|
# One YAML block from a sync-file, enriched with live filesystem state.
|
|
6
6
|
Job = Struct.new(
|
|
7
7
|
:program, :path, :description, :active, :excludes, :label,
|
|
8
|
-
:source, :target, :cmd, :sync_file,
|
|
8
|
+
:source, :target, :cmd, :delete, :render, :render_outdated, :target_path_field, :sync_file,
|
|
9
9
|
:source_exists, :target_exists, :source_mtime, :target_mtime, :conflict,
|
|
10
10
|
keyword_init: true,
|
|
11
11
|
) do
|
|
12
12
|
def source_path = File.join(source, path)
|
|
13
|
-
def target_path = File.join(target, path)
|
|
13
|
+
def target_path = File.join(target, target_path_field || path)
|
|
14
14
|
|
|
15
15
|
def status
|
|
16
16
|
return :disabled if active != 1
|
|
17
17
|
return :both_missing if !source_exists && !target_exists
|
|
18
18
|
return :missing_source unless source_exists
|
|
19
19
|
return :missing_target unless target_exists
|
|
20
|
+
# Render jobs compare by content, not mtime — a rendered target's mtime
|
|
21
|
+
# bears no relation to the template's.
|
|
22
|
+
return render_outdated ? :source_newer : :in_sync if render
|
|
20
23
|
return :target_newer if conflict
|
|
21
24
|
return :in_sync if source_mtime.nil? || target_mtime.nil?
|
|
22
25
|
delta = source_mtime - target_mtime
|
|
@@ -65,7 +68,11 @@ module Twin
|
|
|
65
68
|
rescue JSON::ParserError => e
|
|
66
69
|
raise "grubber returned invalid JSON: #{e.message}"
|
|
67
70
|
end
|
|
68
|
-
|
|
71
|
+
vars = cfg.var_map
|
|
72
|
+
records.filter_map do |r|
|
|
73
|
+
context = "#{r["Program"]} in #{File.basename(r["_note_file"].to_s)}"
|
|
74
|
+
build_job(Twin::Template.substitute_record(r, vars, context: context), vars)
|
|
75
|
+
end
|
|
69
76
|
end
|
|
70
77
|
|
|
71
78
|
def load_programs(cfg, file: nil, label: nil, show_all: false)
|
|
@@ -98,38 +105,49 @@ module Twin
|
|
|
98
105
|
.map { |(name, _file), js| Program.new(name: name, jobs: js) }
|
|
99
106
|
end
|
|
100
107
|
|
|
101
|
-
def build_job(r)
|
|
102
|
-
path
|
|
103
|
-
source
|
|
104
|
-
target
|
|
108
|
+
def build_job(r, vars = {})
|
|
109
|
+
path = r["Path"].to_s
|
|
110
|
+
source = r["Source"].to_s
|
|
111
|
+
target = r["Target"].to_s
|
|
112
|
+
target_path_field = r["Target-Path"].then { |v| v.to_s.empty? ? nil : v.to_s }
|
|
105
113
|
return nil if path.empty? || source.empty? || target.empty?
|
|
106
114
|
|
|
115
|
+
render = r["Render"] == true
|
|
107
116
|
excludes = (r["Exclude"] || "").split(",").map(&:strip).reject(&:empty?)
|
|
108
117
|
|
|
109
118
|
src_full = File.join(source, path)
|
|
110
|
-
tgt_full = File.join(target, path)
|
|
119
|
+
tgt_full = File.join(target, target_path_field || path)
|
|
111
120
|
src_exists, src_mtime = stat(src_full)
|
|
112
121
|
tgt_exists, tgt_mtime = stat(tgt_full)
|
|
122
|
+
|
|
123
|
+
# Render jobs: status is content-based (mtime is meaningless for a rendered
|
|
124
|
+
# target). conflict stays false so the mtime conflict-warning skips them.
|
|
125
|
+
render_outdated = render ? render_outdated?(src_full, tgt_full, vars, path) : nil
|
|
126
|
+
|
|
113
127
|
# Same 60s tolerance as Job#status, so mtime jitter never flags a conflict.
|
|
114
|
-
conflict = src_exists && tgt_exists && tgt_mtime && src_mtime &&
|
|
128
|
+
conflict = !render && src_exists && tgt_exists && tgt_mtime && src_mtime &&
|
|
115
129
|
tgt_mtime - src_mtime >= 60
|
|
116
130
|
|
|
117
131
|
Job.new(
|
|
118
|
-
program:
|
|
119
|
-
path:
|
|
120
|
-
description:
|
|
121
|
-
active:
|
|
122
|
-
excludes:
|
|
123
|
-
label:
|
|
124
|
-
source:
|
|
125
|
-
target:
|
|
126
|
-
cmd:
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
132
|
+
program: r["Program"].to_s,
|
|
133
|
+
path: path,
|
|
134
|
+
description: r["Description"].to_s,
|
|
135
|
+
active: (r["Active"] || 0).to_i,
|
|
136
|
+
excludes: excludes,
|
|
137
|
+
label: r["Label"].to_s,
|
|
138
|
+
source: source,
|
|
139
|
+
target: target,
|
|
140
|
+
cmd: r["Cmd"].to_s,
|
|
141
|
+
delete: r["Delete"] == true,
|
|
142
|
+
render: render,
|
|
143
|
+
render_outdated: render_outdated,
|
|
144
|
+
target_path_field: target_path_field,
|
|
145
|
+
sync_file: r["_note_file"].to_s,
|
|
146
|
+
source_exists: src_exists,
|
|
147
|
+
target_exists: tgt_exists,
|
|
148
|
+
source_mtime: src_mtime,
|
|
149
|
+
target_mtime: tgt_mtime,
|
|
150
|
+
conflict: !!conflict,
|
|
133
151
|
)
|
|
134
152
|
end
|
|
135
153
|
|
|
@@ -139,5 +157,17 @@ module Twin
|
|
|
139
157
|
rescue Errno::ENOENT, Errno::EACCES
|
|
140
158
|
[false, nil]
|
|
141
159
|
end
|
|
160
|
+
|
|
161
|
+
# For a render job: is the target out of date with the rendered template?
|
|
162
|
+
# nil when source is missing/a directory (status falls through to those).
|
|
163
|
+
# True when target is absent or content differs, or the template can't be
|
|
164
|
+
# rendered (unresolved token) — i.e. needs attention.
|
|
165
|
+
def render_outdated?(src_full, tgt_full, vars, context)
|
|
166
|
+
return nil unless File.file?(src_full)
|
|
167
|
+
rendered = Twin::Template.render_file(src_full, vars, context: context)
|
|
168
|
+
!File.exist?(tgt_full) || File.binread(tgt_full) != rendered
|
|
169
|
+
rescue
|
|
170
|
+
true
|
|
171
|
+
end
|
|
142
172
|
end
|
|
143
173
|
end
|
data/lib/twin/sync.rb
CHANGED
|
@@ -4,6 +4,18 @@ module Twin
|
|
|
4
4
|
module Sync
|
|
5
5
|
module_function
|
|
6
6
|
|
|
7
|
+
# A line from `rsync --itemize-changes` describing a real change: an
|
|
8
|
+
# itemize code whose first column is the update type (< > c h *) and second
|
|
9
|
+
# the file type (f d L D S), e.g. ">f+++++++++", "cd+++++++++", "*deleting".
|
|
10
|
+
# No-op runs emit no such line; headers ("sending …", "created directory …")
|
|
11
|
+
# and the summary don't match. Deterministic — no scraping of prose.
|
|
12
|
+
ITEMIZE_CHANGE = /\A[<>ch*][fdLDS]/
|
|
13
|
+
|
|
14
|
+
# True when rsync reported at least one changed item.
|
|
15
|
+
def transferred?(output)
|
|
16
|
+
output.lines.any? { |l| ITEMIZE_CHANGE.match?(l) }
|
|
17
|
+
end
|
|
18
|
+
|
|
7
19
|
# True if the path lives on a mounted volume other than the root filesystem.
|
|
8
20
|
# Walks up parents until it finds a mount point (different device than parent)
|
|
9
21
|
# or hits "/" (path is on the root volume, not externally mounted).
|
|
@@ -20,16 +32,67 @@ module Twin
|
|
|
20
32
|
false
|
|
21
33
|
end
|
|
22
34
|
|
|
23
|
-
#
|
|
35
|
+
# Render a template Job: read source, substitute {{vars}}, write if changed.
|
|
36
|
+
# Returns [success, output, changed].
|
|
37
|
+
def render_job(cfg, job, dry_run: false)
|
|
38
|
+
src = job.source_path
|
|
39
|
+
tgt = job.target_path
|
|
40
|
+
|
|
41
|
+
return [false, "source not found: #{src}", false] unless File.exist?(src)
|
|
42
|
+
return [false, "render: source must be a file, not a directory: #{src}", false] if File.directory?(src)
|
|
43
|
+
|
|
44
|
+
begin
|
|
45
|
+
rendered = Twin::Template.render_file(src, cfg.var_map, context: job.path)
|
|
46
|
+
rescue => e
|
|
47
|
+
return [false, e.message, false]
|
|
48
|
+
end
|
|
49
|
+
|
|
50
|
+
if dry_run
|
|
51
|
+
return [true, "(dry-run) would render #{File.basename(src)} → #{tgt}", false]
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
FileUtils.mkdir_p(File.dirname(tgt))
|
|
55
|
+
|
|
56
|
+
current = File.exist?(tgt) ? File.binread(tgt) : nil
|
|
57
|
+
changed = (current != rendered)
|
|
58
|
+
|
|
59
|
+
if changed
|
|
60
|
+
File.binwrite(tgt, rendered)
|
|
61
|
+
output = "rendered #{File.basename(src)} → #{tgt}"
|
|
62
|
+
else
|
|
63
|
+
output = "#{File.basename(src)}: content unchanged"
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
if !job.cmd.empty?
|
|
67
|
+
if changed
|
|
68
|
+
cmd_out, cmd_status = run(["sh", "-c", job.cmd])
|
|
69
|
+
output += "\ncmd: #{job.cmd}\n#{cmd_out}"
|
|
70
|
+
unless cmd_status.success?
|
|
71
|
+
output += "cmd failed (exit #{cmd_status.exitstatus})"
|
|
72
|
+
return [false, output, changed]
|
|
73
|
+
end
|
|
74
|
+
else
|
|
75
|
+
output += "\ncmd skipped (content unchanged)"
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
[true, output, changed]
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
# Sync one Job. Returns [success, combined_output, transferred].
|
|
83
|
+
# transferred is true when rsync actually moved bytes (false on no-op or dry_run).
|
|
24
84
|
def run_job(cfg, job, dry_run: false)
|
|
85
|
+
return render_job(cfg, job, dry_run: dry_run) if job.render
|
|
86
|
+
|
|
25
87
|
src = job.source_path
|
|
26
88
|
tgt = job.target_path
|
|
27
89
|
|
|
28
|
-
return [false, "source not found: #{src}"] unless File.exist?(src)
|
|
90
|
+
return [false, "source not found: #{src}", false] unless File.exist?(src)
|
|
29
91
|
|
|
30
92
|
FileUtils.mkdir_p(File.dirname(tgt))
|
|
31
93
|
|
|
32
|
-
args = ["rsync", "-av", "--update"]
|
|
94
|
+
args = ["rsync", "-av", "--itemize-changes", "--update"]
|
|
95
|
+
args << "--delete" if job.delete
|
|
33
96
|
args << "--dry-run" if dry_run
|
|
34
97
|
cfg.global_excludes.each { |ex| args << "--exclude=#{ex}" }
|
|
35
98
|
job.excludes.each { |ex| args << "--exclude=#{ex}" }
|
|
@@ -41,15 +104,28 @@ module Twin
|
|
|
41
104
|
end
|
|
42
105
|
|
|
43
106
|
output, status = run(args)
|
|
44
|
-
return [false, output] unless status.success?
|
|
107
|
+
return [false, output, false] unless status.success?
|
|
108
|
+
|
|
109
|
+
xfr = !dry_run && transferred?(output)
|
|
110
|
+
|
|
111
|
+
if job.conflict && !xfr && !dry_run
|
|
112
|
+
output += "\nskipped: target is newer, source not synced"
|
|
113
|
+
end
|
|
45
114
|
|
|
46
115
|
if !job.cmd.empty? && !dry_run
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
116
|
+
if xfr
|
|
117
|
+
cmd_out, cmd_status = run(["sh", "-c", job.cmd])
|
|
118
|
+
output += "\ncmd: #{job.cmd}\n#{cmd_out}"
|
|
119
|
+
unless cmd_status.success?
|
|
120
|
+
output += "cmd failed (exit #{cmd_status.exitstatus})"
|
|
121
|
+
return [false, output, xfr]
|
|
122
|
+
end
|
|
123
|
+
else
|
|
124
|
+
output += "\ncmd skipped (nothing transferred)"
|
|
125
|
+
end
|
|
50
126
|
end
|
|
51
127
|
|
|
52
|
-
[true, output]
|
|
128
|
+
[true, output, xfr]
|
|
53
129
|
end
|
|
54
130
|
|
|
55
131
|
# Sync all jobs in a Program. Returns array of [job, success, output].
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
module Twin
|
|
2
|
+
module Template
|
|
3
|
+
PATTERN = /\{\{([^}]+)\}\}/
|
|
4
|
+
|
|
5
|
+
# Fields in a grubber record that may contain {{tokens}}.
|
|
6
|
+
FIELDS = %w[Source Target Path Target-Path Exclude Cmd].freeze
|
|
7
|
+
|
|
8
|
+
module_function
|
|
9
|
+
|
|
10
|
+
# Replace {{token}} in value using vars. Raises on unknown token.
|
|
11
|
+
def substitute(value, vars, context:)
|
|
12
|
+
return value unless value.is_a?(String)
|
|
13
|
+
value.gsub(PATTERN) do
|
|
14
|
+
token = $1
|
|
15
|
+
vars.fetch(token) { raise "unknown template token {{#{token}}} in #{context}" }
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
# Read a template file and substitute {{tokens}} in its content.
|
|
20
|
+
# Binary read preserves bytes exactly; raises on unknown token.
|
|
21
|
+
def render_file(path, vars, context:)
|
|
22
|
+
substitute(File.binread(path), vars, context: context)
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
# Return a copy of grubber record r with FIELDS substituted.
|
|
26
|
+
def substitute_record(r, vars, context:)
|
|
27
|
+
return r if vars.empty?
|
|
28
|
+
result = r.dup
|
|
29
|
+
FIELDS.each do |field|
|
|
30
|
+
next unless result.key?(field) && result[field].is_a?(String)
|
|
31
|
+
result[field] = substitute(result[field], vars, context: context)
|
|
32
|
+
end
|
|
33
|
+
result
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
data/lib/twin/version.rb
CHANGED
data/lib/twin.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: mark-twin
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.2.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Ralf Hülsmann
|
|
@@ -43,6 +43,7 @@ files:
|
|
|
43
43
|
- lib/twin/preview.rb
|
|
44
44
|
- lib/twin/scanner.rb
|
|
45
45
|
- lib/twin/sync.rb
|
|
46
|
+
- lib/twin/template.rb
|
|
46
47
|
- lib/twin/version.rb
|
|
47
48
|
homepage: https://github.com/rhsev/mark-twin
|
|
48
49
|
licenses:
|