mark-twin 0.3.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0a1efe3b9e90b401c9c6f8a25652db649ddea26054ede30f2c221d0d73246297
-  data.tar.gz: b85e21f28805ade66696c9997b986b0472f2d3028971075a30f15d35ede113b7
+  metadata.gz: 5bae405ea7916f59395530f5372cd3bde2aa4ff742ca4cf08142ca16be5c7cd9
+  data.tar.gz: b05b48d23901e7c6c2fd313ecc0f8ef2dbf172ab77ef38b5575088031dc186da
 SHA512:
-  metadata.gz: cf13dd6e51d53af11adc5b49c5e183ab6dd07ec9a49e4d83cafa60c712ca2bb492bb944f00b1afca864e051c09f7c054465d725d1da858d0960c48f6507778ff
-  data.tar.gz: d7a00b682a1e8b833c7aceda323da4c69625fa3538def0810334dd1ed757696aca7d92d34d1e914bb06b60d1539ed6b990f5b24fdb94a59847bee2404272e51e
+  metadata.gz: 3d10977e33311149b1d90486c553cf6ce773c9c2927a081f3e264dcb9a8d0c97be61bb62883b070cb739ab66aa710d2d359b95db33018f8463bd0ac9987c0a95
+  data.tar.gz: ec1eecdf09d77e1c0ef6480edeb889c9d737b87cbd9eda5a9e83e210ca5fc2eeee85d06f86ccb65e81040197a8a4402001eeb79bda60710f44c77cd7e2e16c26

data/ARCHITECTURE.md

@@ -19,6 +19,11 @@ sync-files (.md)
       └──▶  Picker    fzf + apex preview, returns selected Program
                           │
                           ▼
+                          │
+                          ▼
+                    Conflict   paired dry-runs → files --update would hold
+                          │     back → content compare → ask once, or abort
+                          ▼
                        Sync   rsync (or render) per Job, mount check, Cmd hook
 ```
 
@@ -32,6 +37,7 @@ lib/twin/
   config.rb     ~/.config/twin/config.yaml loader; host table → var_map
   scanner.rb    Job, Program structs; grubber + template + stat → grouped Programs
   sync.rb       rsync / render execution, mount check, post-sync hook
+  conflict.rb   target-side changes: detection via paired dry-runs, diffs
   journal.rb    append-only sync journal (~/.local/state/twin/log.jsonl)
   add.rb        `twin add` — interactive scaffolding of new sync entries
   picker.rb     fzf wrapper with apex preview
@@ -46,11 +52,16 @@ test/test_pure.rb
 **Job** — one YAML block:
 
 ```
-program, path, description, active, excludes, label, source, target, cmd,
+program, path, description, active, excludes, owned, label, source, target, cmd,
 delete, render, render_outdated, target_path_field, sync_file,
 source_exists, target_exists, source_mtime, target_mtime, conflict
 ```
 
+`excludes` and `owned` both become `--exclude` (via `Job#all_excludes`); they
+are kept apart so `status` can report intent. `conflict` is the coarse mtime
+flag used by `status` only — conflict *resolution* ignores it and asks rsync
+directly, because a directory's mtime says nothing about edits inside it.
+
 `Job#status` → one of `disabled / unreachable / both_missing / missing_source /
 missing_target / target_newer / in_sync / source_newer`. Render jobs derive status from content
 (`render_outdated`), not mtime; non-render jobs ignore those fields.
@@ -65,6 +76,22 @@ name, jobs
 `Program#status` aggregates jobs (worst state wins). Selection in the picker
 operates on Programs, not individual Jobs.
 
+**Job order is part of the contract.** Jobs of a Program run in the order their
+YAML blocks appear in the sync-file. Sync-files rely on this — a `Cmd` that
+restarts a service belongs in the last block, so it fires after every path is in
+place. Reorder the jobs and a deploy restarts against half-written state,
+without any error to show for it.
+
+The order is guaranteed at both ends of the pipeline, not merely observed:
+
+- grubber emits records in document order — first block, first record — for all
+  three output formats, pinned by its own `TestBlockOrderFollowsDocument`
+  across `Extract` and `StreamJSONL`.
+- twin preserves it through `filter_map` and `group_by` (insertion order per
+  key), pinned by `test_job_order_follows_document_order`.
+
+Neither side may quietly sort.
+
 ## Configuration
 
 `~/.config/twin/config.yaml`:

data/README.md

@@ -4,26 +4,56 @@
 [![Tests](https://github.com/rhsev/mark-twin/actions/workflows/test.yml/badge.svg)](https://github.com/rhsev/mark-twin/actions/workflows/test.yml)
 [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
 
-Sync configuration folders between two Macs from self-documenting Markdown files.
+Sync configuration between machines, from Markdown files that explain
+themselves.
 
-Sync entries are defined in Markdown files with YAML blocks — human-readable,
-self-documenting, and queryable via [grubber](https://github.com/rhsev/grubber).
-Selection is interactive via [fzf](https://github.com/junegunn/fzf), with a
-Markdown preview rendered by [apex](https://github.com/ttscoff/apex) and
-optional post-sync actions on the target via [mi.lan](https://github.com/rhsev/mi.lan).
+A sync-file is a normal Markdown document. The prose explains it to you — and
+to whatever assistant you point at the file later. The fenced YAML blocks are
+what twin acts on:
 
-## Why?
+````markdown
+---
+Active: 1
+Source: /Users/admin
+Target: admin@macbook:/Users/admin
+---
+
+## Fish Shell
 
-Sync-definitions in Markdown + YAML are three things at once:
+Shell config, including completions and abbreviations. `local.fish` stays
+machine-specific — the laptop keeps its own.
+
+```yaml
+Program: Fish Shell
+Path: .config/fish
+Description: Fish Shell configuration
+Own: conf.d/local.fish
+```
+````
 
-- **Human-readable.** Plain Markdown, no twin-specific syntax to learn. The
-  Markdown frame documents *why* a path is synced, not just what, so you can
-  read your own sync-files in a year and still understand them.
-- **Machine-readable.** The YAML blocks are queryable via grubber, so any tool
-  (twin, but also future ones) can act on the same source of truth.
-- **AI-writable.** LLMs handle Markdown + YAML well. You can ask an assistant to
-  add new entries or refactor existing ones, and the result stays valid for both
-  humans and grubber.
+`twin` then shows you what differs and syncs what you pick. Under the
+hood it is `rsync`; the point is the file above, which still tells you in a
+year *why* a path is synced, not just that it is.
+
+That is the whole idea, and for most entries it stays exactly that small. The
+rest of this README is long because twin does handle the awkward cases — files
+the other machine owns, both sides changed since last time, paths that differ
+per host — not because you need any of them on day one. Skip what you don't
+need.
+
+## Why Markdown
+
+- **You can read it later.** The reason a path is synced lives next to the
+  path, in prose, not in a comment you stopped writing after the third entry.
+- **Machines can read it too.** The YAML blocks are extracted by
+  [grubber](https://github.com/rhsev/grubber), so the same file can feed other
+  tools, not just twin.
+- **It stays editable by hand.** No generated state, no database. Add a block
+  in your editor and twin picks it up.
+- **And yes, ask your AI to write them.** Markdown with YAML blocks is exactly
+  what language models are good at. Show one an existing sync-file, describe
+  the next entry, and what comes back is valid for grubber and still readable
+  by you — which is the whole bargain of this format.
 
 ## Screenshots
 
@@ -37,22 +67,21 @@ compact preview of the relevant sync-file section, rendered by apex:
 
 ![Stage 2 — Fish Shell paths with apex preview](https://raw.githubusercontent.com/rhsev/mark-twin/main/docs/stage_2_fish.png)
 
-## Installation
-
-### 1. Install grubber
-
-twin parses sync-files via [grubber](https://github.com/rhsev/grubber), a
-small Go binary. Download the latest release for your platform from
-[github.com/rhsev/grubber/releases](https://github.com/rhsev/grubber/releases)
-and put it somewhere in your `PATH` (e.g. `/usr/local/bin/grubber`).
-
-### 2. Install twin
+## Install
 
 ```bash
 gem install mark-twin
+brew install fzf
 ```
 
-Or from source:
+Plus [grubber](https://github.com/rhsev/grubber/releases), a small Go binary —
+download it and put it in your `PATH`. `rsync` ships with macOS.
+
+Optional: one of `apex`, `glow` or `bat` for the preview pane (tried in that
+order, `cat` if none are present). `twin doctor` reports what it found.
+
+<details>
+<summary>From source</summary>
 
 ```bash
 git clone https://github.com/rhsev/mark-twin.git
@@ -60,242 +89,235 @@ cd mark-twin
 gem build mark-twin.gemspec
 gem install ./mark-twin-*.gem
 ```
+</details>
 
-### 3. Other tools
-
-Also required in `PATH`: `rsync` (preinstalled on macOS), `fzf`
-(`brew install fzf`). For the stage-2 preview, one of `apex`, `glow`,
-or `bat` is recommended (falls back in that order; `cat` if none are
-present).
+## Getting started
 
-## Quickstart
+**1. Decide where the other side lives.** `Target:` takes either form, and
+neither is the special case:
 
-Twin assumes the target machine is reachable as a mounted volume (typically
-via SMB or NFS). The mount check is enforced before any sync.
+```yaml
+Target: admin@macbook:/Users/admin              # over ssh
+Target: /Volumes/macbook/Users/admin            # a mounted volume (SMB, NFS, …)
+```
 
-1. **Pick a directory for sync-files** (anywhere; this example uses `~/Sync`):
+Over ssh you need key-based login — twin runs `ssh -o BatchMode=yes` and never
+prompts for a password, so set up `ssh-copy-id macbook` first. A mounted volume
+needs no keys but has to be mounted; twin checks before every sync. The
+[comparison below](#mounted-volume-or-ssh) covers the two differences that
+actually matter.
 
-   ```bash
-   mkdir -p ~/Sync
-   ```
+**2. Point twin at a folder for sync-files**, in `~/.config/twin/config.yaml`:
 
-2. **Create the config** at `~/.config/twin/config.yaml`:
+```yaml
+sync_dir: ~/Sync
 
-   ```yaml
-   sync_dir: ~/Sync
-   global_excludes:
-     - .DS_Store
-     - .git/
-   ```
+global_excludes:
+  - .DS_Store
+  - .git/
+```
 
-3. **Drop a sync-file** into `~/Sync`. The simplest starting point is to copy
-   one of the [examples](examples/) and adapt the frontmatter:
+**3. Write a sync-file**, or start from one of the [examples](examples/):
 
-   ```bash
-   cp examples/home.md ~/Sync/
-   $EDITOR ~/Sync/home.md   # edit Source: and Target:
-   ```
+```bash
+mkdir -p ~/Sync
+cp examples/home.md ~/Sync/
+$EDITOR ~/Sync/home.md      # adjust Source: and Target:
+```
 
-4. **Run twin**:
+`twin add ~/.config/fish` does the same interactively, if you prefer prompts to
+an editor — it finds the matching sync-file, derives the relative path,
+suggests excludes for what it sees in the directory, and appends a block with a
+prose stub.
 
-   ```bash
-   twin
-   ```
+**4. Look before you leap:**
 
-   Pick a program, then the paths to sync, hit Enter.
+```bash
+twin status              # what differs
+twin sync --dry-run      # what a sync would do, without doing it
+twin                     # interactive: pick a program, pick paths, Enter
+```
 
-## Usage
+That is the whole loop. Everything below is detail you can come back for.
 
-Twin has two modes: an **interactive interface** (default — see screenshots
-above) and **CLI commands** for status checks and batch sync.
+## Everyday commands
 
 ```bash
-twin                         # TUI — all programs across all sync-files
-twin home.md                 # TUI — one sync-file in sync_dir (by name)
-twin /abs/path/to/file.md    # TUI — any sync-file by absolute path
-twin ./relative/dir/         # TUI — all sync-files in a directory
+twin                         # picker — all programs across all sync-files
+twin home.md                 # picker — one sync-file in sync_dir (by name)
+twin ./some/dir/             # picker — all sync-files in a directory
 twin list                    # plain listing
 twin status                  # listing with source/target mtimes
 twin sync -p grubber         # sync one program by name pattern
 twin sync --file=repos       # sync all programs from a sync-file
 twin sync --dry-run          # preview without writing
 twin log                     # recent journal entries (-n N, --json)
-twin doctor                  # check tools, renderers, and sync targets
+twin doctor                  # check tools, renderers, and targets
 twin --help                  # show usage
 ```
 
-### Adding a sync entry
-
-`twin add <path>` scaffolds a new entry interactively, so the judgment calls
-of setting up a sync become prompts with defaults:
-
-```
-$ twin add ~/.config/fish
-sync-file: home_macbook.md  (/Users/admin → /Volumes/macbook/Users/admin)
-Program name [fish]:
-Why is this synced? (one line of prose): Shell config incl. abbreviations.
-Description (short, for listings) [fish]: Fish Shell configuration
-Exclude (comma-separated) [.git/]:
-Mirror deletions on target (Delete: true)? (y/N) [n]:
-Post-sync Cmd (empty for none):
-
-added "fish" to home_macbook.md
-Run a dry-run now? (Y/n) [y]:
-```
-
-twin matches the path against the `Source:` roots of your sync-files (asking
-which to use when several match), derives `Path:` relative to that root,
-suggests excludes for what it finds in the directory (`.git/`,
-`node_modules/`, `.venv/`, …), refuses duplicates, and appends a Markdown
-block — prose stub included. If no sync-file covers the path, it offers to
-create one (frontmatter and all), which is also the quickest way to start
-syncing to a new SSH target.
-
-Every synced job is journaled to `~/.local/state/twin/log.jsonl` (one JSON
-line per job: timestamp, program, path, outcome). `twin log` shows the recent
-history; `twin sync` exits non-zero when any job failed.
-
-### Unattended syncs
-
-For a scheduled run (launchd, cron), combine two flags:
-
-```bash
-twin sync --quiet --skip-unavailable
-```
-
-`--quiet` prints only conflicts, errors, and jobs that actually changed
-something — a no-op run is silent. `--skip-unavailable` skips targets that
-are currently unmounted or unreachable instead of aborting, so a laptop that
-isn't docked doesn't turn into an error. Combined, the run produces output
-(and a non-zero exit) only when something genuinely needs attention, which is
-exactly what launchd's stdout/stderr logging wants; the journal still records
-every job.
-
-File argument resolution:
+A file argument without `/` is matched by substring against sync-file names in
+`sync_dir`; anything containing `/` is treated as a path, file or directory.
 
-- bare name (no `/`)  → looked up by substring in `sync_dir`
-- contains `/`        → resolved as path (absolute or relative); file or directory both work
-
-## Configuration
+Every job is journaled to `~/.local/state/twin/log.jsonl` — one JSON line with
+timestamp, program, path and outcome. `twin sync` exits non-zero if any job
+failed.
 
-`~/.config/twin/config.yaml`:
-
-```yaml
-sync_dir: /path/to/sync-files
+## Sync-files
 
-global_excludes:
-  - .DS_Store
-  - .git/
+One Markdown file per relationship. Frontmatter sets it up, YAML blocks define
+the individual paths. Frontmatter fields are merged into every block, so
+`Source:`/`Target:` are usually written once at the top — but a block may
+override them, which is how one file can serve several destinations.
 
-# Optional preview rendering (apex):
-# apex_theme: default
-# apex_width: 80
-# apex_code_highlight: monokai
-# apex_code_highlight_theme: dark
-```
+Blocks sharing a `Program` are grouped: they are selected together, synced
+together, and **run in the order they appear in the file**.
 
-Environment overrides: `TWIN_SYNC_DIR`, `TWIN_CONFIG`, `TWIN_HOST` (which host
-twin runs as — lets one config serve both machines).
+### Field reference
 
-## Sync-files
+Field names are capitalised English. An unknown key is ignored silently and a
+missing `Active` counts as `0`, so a typo shows up as an entry that never syncs
+rather than as an error.
 
-Each Markdown file represents one sync relationship. Frontmatter defines the
-relationship (Source/Target); YAML blocks define individual paths.
+| Field | Where | Meaning |
+|---|---|---|
+| `Program` | block | Group name; blocks sharing it sync together |
+| `Path` | block | Path relative to `Source` (file or directory) |
+| `Source` | either | Absolute base path on this machine |
+| `Target` | either | Absolute base path, or `user@host:/path` for ssh |
+| `Target-Path` | block | Path under `Target`, when it differs from `Path` |
+| `Active` | either | `1` syncs, `0` skips. Default `0` |
+| `Description` | block | Shown in listings and the picker |
+| `Label` | either | Free-text grouping, filterable via `--label` |
+| `Exclude` | block | Comma-separated paths that are not part of the sync |
+| `Own` | block | Comma-separated paths the **target** owns |
+| `Delete` | block | `true` mirrors deletions, with backups |
+| `Cmd` | block | Shell command, run only when bytes actually moved |
+| `Render` | block | `true` substitutes `{{tokens}}` instead of copying |
+
+### Exclude or Own?
+
+Both keep rsync away from a path and both take a comma-separated list
+(`Exclude: *.log, __pycache__/`). What differs is the meaning, and `twin status`
+reports them apart:
+
+- **`Exclude:`** — not part of this sync at all. Build artefacts, logs, caches,
+  `.git/`, a test script with no business on the other machine.
+- **`Own:`** — inside the sync scope, but the **target** owns it.
+  Machine-specific configuration the source must never clobber:
+  `conf.d/local.fish`, `lazy-lock.json`, a per-host credentials file.
+
+The distinction is documentation, not mechanism. Six months on, `Own:` still
+says "deliberate, the other machine maintains this", where the same entry sitting
+in `Exclude:` between `*.dwarf` and `.DS_Store` reads like noise you once
+filtered out.
+
+### Cmd: doing something after a sync
+
+`Cmd` runs a shell command once rsync has actually transferred bytes — a no-op
+sync runs nothing. Typically a `curl` to a local automation endpoint like
+[mi.lan](https://github.com/rhsev/mi.lan), or an `ssh host '…'` for remote
+targets (`Cmd` always runs locally).
+
+It belongs to a block rather than to the program, which is what you want when
+different paths need different follow-ups — reload nginx after its config,
+re-link binaries after `bin/`. For a **restart**, put the command in the
+**last** block: jobs run in file order, so a restart placed earlier brings the
+service back before the remaining paths are written. The same command repeated
+across blocks restarts repeatedly, for the same reason. One command, last block.
+
+### Delete: mirroring removals
+
+`Delete: true` adds `--delete`, so files removed from the source disappear on
+the target too. Deleted and overwritten files are moved to
+`<target>/.twin-backup/<timestamp>/` rather than destroyed — a safety net worth
+pruning occasionally. It applies to `Delete` jobs only.
+
+## Mounted volume or ssh
+
+Both are first-class. `twin status`, the picker, `Exclude`/`Own`, `Delete` and
+`Cmd` behave identically; remote paths are stat'ed in a single ssh round-trip
+per host, and an unreachable host shows as `?` instead of failing the scan.
+
+Two differences are real:
+
+|  | mounted volume | ssh |
+|---|---|---|
+| Setup | volume must be mounted | ssh key (`ssh-copy-id`) |
+| Before syncing | mount check | reachability check |
+| `Render: true` | supported | **not** supported |
 
-See [examples/home.md](examples/home.md) and [examples/repos.md](examples/repos.md)
-for ready-to-adapt templates.
+`Render` needs to read and write file contents on the target, which twin only
+does locally. Keep rendered files on mounted targets, or render locally and sync
+the result.
 
-Minimal example:
+Syncing is push-only in both cases: `Source:` is always this machine.
 
-````markdown
+```markdown
 ---
 Active: 1
-Label: mac-mini → macbook
-Source: /Users/admin
-Target: /Volumes/macbook/Users/admin
+Label: mini → server
+Source: /Volumes/lightning/Git/Website
+Target: ralf@server:/srv/www
 ---
+```
 
-## Fish Shell
+## When the target has changed too
 
-Configuration for the fish shell, including completions and abbreviations.
+A sync has a direction: the source wins. But targets get edited — a quick fix
+made on the server at midnight, a config tweaked where it runs. Twin looks for
+that before it moves the first byte, and asks once for the whole program:
 
-```yaml
-Program: Fish Shell
-Path: .config/fish
-Description: Fish Shell configuration
-Exclude: conf.d/local.fish
 ```
-````
+target has changed since the last sync — 1 file(s) differ:
+  ! app/code.rb (target 2h newer)
+syncing would replace them with the source version.
 
-Frontmatter fields (`Active`, `Label`, `Source`, `Target`) are merged into
-every block by grubber. Multiple blocks can share the same `Program` — twin
-groups them and treats the program as the unit of selection.
-
-The optional `Cmd` field runs an arbitrary shell command after a successful
-sync — typically a `curl` to a local automation endpoint like
-[mi.lan](https://github.com/rhsev/mi.lan) to reload a program, restart a
-service, or notify another machine. The command only runs when rsync actually
-transferred bytes; no-op syncs skip it. See the Helix entry in
-[examples/home.md](examples/home.md).
+overwrite these on the target and sync? [y]es / [d]iff / [n]o (abort)
+```
 
-The optional `Delete: true` field adds `--delete` to the rsync invocation,
-so files removed from the source are also removed on the target. Useful for
-directory syncs where the target should mirror the source exactly. As a
-safety net, deleted and overwritten files are moved to a per-run backup
-directory on the target (`<target>/.twin-backup/<timestamp>/`) instead of
-being destroyed — prune it occasionally.
+`d` prints a unified diff per file, then asks again. `n` aborts the entire
+program — nothing is written, so you never end up with half a deploy applied.
 
-## SSH targets
+Two properties make this bearable day to day:
 
-`Target:` accepts remote destinations in rsync notation — `user@host:/path` or
-`host:/path`. Everything else stays the same: the YAML block, excludes,
-`Delete:`, the `Cmd` hook.
+- **Content, not timestamps.** A file that is merely newer on the target with
+  identical bytes is not a conflict and does not ask. Sync a tree in both
+  directions and you collect dozens of those; a prompt that fires on them gets
+  answered without being read.
+- **Directory mtimes are ignored.** Editing a file in place leaves its
+  directory's mtime untouched, and `rsync -a` equalises those anyway. Twin asks
+  rsync what it would actually transfer instead of guessing from a directory.
 
-````markdown
----
-Active: 1
-Label: mini → server
-Source: /Volumes/lightning/Git/Website
-Target: ralf@server:/srv/www
----
+`twin status` is still mtime-based and cannot see an in-place edit. It is the
+cheap overview; `twin sync` is what decides.
 
-## Website
+## Automation
 
-Static site, deployed straight from the build directory.
+For a scheduled run (launchd, cron):
 
-```yaml
-Program: website
-Path: public
-Description: static site
-Exclude: .git/
-Cmd: ssh ralf@server 'sudo systemctl reload caddy'
+```bash
+twin sync --quiet --skip-unavailable --skip-conflicts
 ```
-````
 
-Details:
-
-- **Push only.** `Source:` stays local; twin syncs *to* the remote host.
-- **Key-based auth required.** twin probes and stats hosts with
-  `ssh -o BatchMode=yes`, which never prompts for a password. Set up an SSH
-  key (`ssh-copy-id host`) first; `twin doctor` shows whether a host is
-  reachable.
-- **Status works remotely.** `twin status` and the picker stat all remote
-  paths of a host in a single ssh round-trip (macOS and Linux targets both
-  supported). An unreachable host shows as `?` instead of failing the scan.
-- **The mount check becomes a reachability check** — sync aborts if the host
-  doesn't answer.
-- **`Cmd` runs locally**, exactly as for mounted targets. To act on the
-  server, make the command an `ssh host '…'` call (see example above).
-- **`Render: true` is not supported** for remote targets (twin would have to
-  read and write remote file contents). Render locally or keep rendered files
-  on mounted targets.
+- `--quiet` — only conflicts, errors and jobs that changed something. A no-op
+  run is silent.
+- `--skip-unavailable` — a laptop that isn't docked is skipped, not an error.
+- `--skip-conflicts` — leave target-side changes alone and sync the rest.
+  Use `--force` instead to overwrite them.
+
+Without a terminal and without one of those two flags, a real conflict aborts
+the run with exit code 1 rather than picking an answer for you. Output and a
+non-zero exit therefore mean something genuinely needs attention — which is
+what launchd's logging wants. The journal records every job regardless.
 
 ## Templating
 
-Some configs differ per machine — a LaunchAgent plist that points at
-`/Volumes/lightning/…` on one Mac and `/Users/ralf/…` on another, a
-`settings.json` with a device-specific id. Those used to fall out of twin and
-get hand-maintained. Templating folds them back into one source of truth.
+*Skip this until you hit the problem it solves.*
+
+Some configs differ per machine — a LaunchAgent plist pointing at
+`/Volumes/lightning/…` on one Mac and `/Users/ralf/…` on another. Those used to
+fall out of twin and get hand-maintained.
 
 Define a host table in `~/.config/twin/config.yaml`:
 
@@ -326,9 +348,8 @@ The distinction matters: a file *written* to the mount (`/Volumes/ralf/…`) but
 
 `Render: true` turns a block from copy into *render*: twin reads the source as a
 template, substitutes `{{…}}` in its **content**, and writes the result only if
-it differs from the current target (so a `Cmd` hook fires only on a real change).
-`Target-Path:` overrides the target-side relative path when it differs from the
-source layout:
+it differs from the current target, so a `Cmd` hook fires only on a real change.
+`Target-Path:` overrides the target-side relative path:
 
 ````markdown
 ## LiveSync LaunchAgent
@@ -344,15 +365,35 @@ Cmd: curl -s http://mi.lan/livesync-reload
 ```
 ````
 
-`twin doctor` checks that every `{{token}}` across your sync-files resolves, and
-`twin status` compares rendered output by content (not mtime). Without a `hosts`
-table, templating is inert and literal-path sync-files behave exactly as before.
+`twin doctor` checks that every `{{token}}` resolves, and `twin status` compares
+rendered output by content rather than mtime. Without a `hosts` table,
+templating is inert and literal-path sync-files behave exactly as before.
+
+## Configuration
+
+`~/.config/twin/config.yaml`:
+
+```yaml
+sync_dir: /path/to/sync-files
+
+global_excludes:
+  - .DS_Store
+  - .git/
+
+# Optional preview rendering (apex):
+# apex_theme: default
+# apex_width: 80
+# apex_code_highlight: monokai
+# apex_code_highlight_theme: dark
+```
+
+Environment overrides: `TWIN_SYNC_DIR`, `TWIN_CONFIG`, `TWIN_HOST` (which host
+twin runs as — lets one config serve both machines).
 
 ## Design
 
-Sync instructions and context in one place — the same Markdown file holds
-both the `Path:` directives and the prose explaining them. No TUI framework:
-`fzf` does the interactive part, `apex` the rendering.
+Sync instructions and their context in one place. No TUI framework: `fzf` does
+the interactive part, `apex` the rendering, `rsync` the work.
 
 See [ARCHITECTURE.md](ARCHITECTURE.md) for the data model and internals.
 

data/lib/twin/cli.rb

@@ -25,6 +25,7 @@ module Twin
         twin status [--all] [--label X] [--file X] [--json]
         twin sync   [-p PATTERN] [--label X] [--file X] [--all] [--dry-run]
                     [--quiet] [--skip-unavailable]
+                    [--force] [--skip-conflicts]
         twin add <path>            scaffold a new sync entry for a local path
         twin log    [-n N] [--json]  recent journal entries (default 20)
         twin doctor                check tools, renderers, and sync targets
@@ -34,6 +35,12 @@ module Twin
         bare name (no /)  → matched by substring against sync-file names
         contains /        → resolved as path; file or directory both work
 
+      TARGET-SIDE CHANGES:
+        Before syncing, twin looks for files the target changed more recently
+        AND whose content differs, then asks once for the whole program.
+        --force           overwrite them without asking (for automation)
+        --skip-conflicts  leave them alone, sync everything else
+
       CONFIG:
         ~/.config/twin/config.yaml
         TWIN_SYNC_DIR  overrides sync_dir
@@ -149,6 +156,8 @@ module Twin
           puts "    #{j.path}#{conflict}"
           puts "      src #{src}"
           puts "      dst #{tgt}"
+          # Named, not hidden: these belong to the target on purpose.
+          puts "      own #{j.owned.join(', ')}" unless j.owned.nil? || j.owned.empty?
         end
       end
     end
@@ -171,7 +180,8 @@ module Twin
       results = programs.map do |p|
         sync_jobs(cfg, p, p.active_jobs,
                   dry_run: opts[:dry_run], quiet: opts[:quiet],
-                  skip_unavailable: opts[:skip_unavailable])
+                  skip_unavailable: opts[:skip_unavailable],
+                  force: opts[:force], skip_conflicts: opts[:skip_conflicts])
       end
       exit 1 unless results.all?
     end
@@ -179,7 +189,8 @@ module Twin
     # Sync the given jobs. Returns true when every attempted job succeeded.
     # quiet:            print only conflicts, errors, and jobs that changed something
     # skip_unavailable: skip jobs whose target is unmounted/unreachable instead of aborting
-    def sync_jobs(cfg, program, jobs, dry_run: false, quiet: false, skip_unavailable: false)
+    def sync_jobs(cfg, program, jobs, dry_run: false, quiet: false, skip_unavailable: false,
+                  force: false, skip_conflicts: false)
       jobs = jobs.select { |j| j.active == 1 }
       return true if jobs.empty?
 
@@ -199,17 +210,17 @@ module Twin
       end
       return true if jobs.empty?
 
-      conflicts = jobs.select(&:conflict)
-      unless conflicts.empty?
-        warn "warning: target is newer than source:"
-        conflicts.each { |j| warn "  ! #{j.path}" }
-        warn "continuing sync (--update skips newer files on target)."
-      end
+      # Decide about target-side changes BEFORE the first byte moves: a partly
+      # applied program is worse than none at all. resolve_conflicts returns
+      # false when the run must not happen.
+      force = resolve_conflicts(cfg, jobs, dry_run: dry_run, quiet: quiet,
+                                force: force, skip_conflicts: skip_conflicts)
+      return false if force == :abort
 
       header_printed = false
       all_ok = true
       jobs.each do |job|
-        success, output, transferred = Twin::Sync.run_job(cfg, job, dry_run: dry_run)
+        success, output, transferred = Twin::Sync.run_job(cfg, job, dry_run: dry_run, force: force)
         Twin::Journal.record(job, success: success, transferred: transferred, output: output) unless dry_run
         all_ok &&= success
         next if quiet && success && !transferred
@@ -225,6 +236,67 @@ module Twin
       all_ok
     end
 
+    # Settle what happens to files the target changed more recently, before any
+    # job runs. Returns true (overwrite them), false (leave them, --update keeps
+    # them) or :abort (sync nothing at all).
+    #
+    # The mtime pre-filter on each Job is coarse and fires often; only files
+    # whose content really differs reach the prompt. A prompt that cries wolf
+    # gets answered without reading it.
+    def resolve_conflicts(cfg, jobs, dry_run:, quiet:, force:, skip_conflicts:)
+      return true  if force
+      return false if skip_conflicts || dry_run
+
+      conflicts = jobs.flat_map { |j| Twin::Conflict.detect(cfg, j) }
+      return false if conflicts.empty?
+
+      report_conflicts(conflicts)
+
+      unless $stdin.tty? && $stdout.tty?
+        warn ""
+        warn "abort: target has changes of its own and there is no terminal to ask."
+        warn "       re-run with --force to overwrite them, or --skip-conflicts to keep them."
+        return :abort
+      end
+
+      loop do
+        print "\noverwrite these on the target and sync? [y]es / [d]iff / [n]o (abort) "
+        $stdout.flush
+        case $stdin.gets&.strip&.downcase
+        when "y", "yes"  then return true
+        when "n", "no", "", nil then puts "aborted — nothing was synced."; return :abort
+        when "d", "diff" then show_diffs(conflicts)
+        else puts "please answer y, d or n."
+        end
+      end
+    end
+
+    def report_conflicts(conflicts)
+      warn "target has changed since the last sync — #{conflicts.size} file(s) differ:"
+      conflicts.each do |c|
+        delta = c.age_delta
+        age   = delta ? " (target #{format_age(delta)} newer)" : ""
+        warn "  ! #{c.job.path == c.rel ? c.rel : File.join(c.job.path, c.rel)}#{age}"
+      end
+      warn "syncing would replace them with the source version."
+    end
+
+    def show_diffs(conflicts)
+      conflicts.each do |c|
+        puts
+        puts "── #{c.rel} " + "─" * [0, 60 - c.rel.length].max
+        puts Twin::Conflict.diff(c)
+      end
+    end
+
+    def format_age(seconds)
+      s = seconds.to_i.abs
+      return "#{s}s"           if s < 90
+      return "#{s / 60}m"      if s < 5400
+      return "#{s / 3600}h"    if s < 172_800
+      "#{s / 86_400}d"
+    end
+
     # :ok, or a human-readable reason the target can't be synced right now.
     def target_availability(job)
       if job.remote?
@@ -374,11 +446,13 @@ module Twin
 
     def parse_sync_opts(args)
       opts = { show_all: false, label: nil, file: nil, pattern: nil, dry_run: false,
-               quiet: false, skip_unavailable: false }
+               quiet: false, skip_unavailable: false, force: false, skip_conflicts: false }
       OptionParser.new do |o|
         o.on("--all")          { opts[:show_all] = true }
         o.on("--label=L")      { |v| opts[:label] = v }
         o.on("--file=F")       { |v| opts[:file] = v }
+        o.on("--force")           { opts[:force] = true }
+        o.on("--skip-conflicts")  { opts[:skip_conflicts] = true }
         o.on("-p", "--pattern=P") { |v| opts[:pattern] = v }
         o.on("--dry-run")      { opts[:dry_run] = true }
         o.on("-q", "--quiet")  { opts[:quiet] = true }

data/lib/twin/conflict.rb

@@ -0,0 +1,143 @@
+require "digest"
+
+require_relative "remote"
+
+module Twin
+  # Finding out which files on the target would be silently skipped by rsync's
+  # --update, and whether that actually matters.
+  #
+  # `Job#conflict` is no help here. It compares the mtime of the job's own path,
+  # and for a directory job that is the directory's mtime — which says nothing
+  # about the files inside it. Worse, it is wrong in exactly the case that
+  # matters: editing a file in place leaves its directory's mtime untouched, and
+  # `rsync -a` equalises directory mtimes on every run anyway. A hand-edit on
+  # the target is therefore invisible to it.
+  #
+  # So we ask rsync, which has the answer already and knows its own matching
+  # rules better than any reimplementation would:
+  #
+  #   1. Which files does --update hold back?
+  #      Dry-run twice, once with --update and once without. Everything the
+  #      second run would transfer but the first would not is exactly the set
+  #      --update protects.
+  #
+  #   2. Of those, which differ in content?
+  #      Only these are worth asking about. A file that is merely newer — same
+  #      bytes, later timestamp — is noise, and noise is what turns a prompt
+  #      into a reflex. Sync both sides of a tree in either order and you get
+  #      dozens of them.
+  #
+  # The first dry-run is also the cheap exit: when a forced run would move
+  # nothing, the job is fully in sync and the second run is skipped. That is the
+  # common case, so a quiet sync costs one extra stat-walk per job and no more.
+  module Conflict
+    # One file the target owns more recently than the source, with content that
+    # actually differs. `same_content` is nil when it could not be determined
+    # (remote targets) — treated as a conflict, because guessing in the other
+    # direction would overwrite work.
+    Entry = Struct.new(:job, :rel, :source_path, :target_path,
+                       :source_mtime, :target_mtime, keyword_init: true) do
+      def age_delta
+        return nil unless source_mtime && target_mtime
+        target_mtime - source_mtime
+      end
+    end
+
+    # rsync --itemize-changes line → relative path. Change lines start with an
+    # update type and a file type (">f.st...... lib/foo.rb"); "*deleting" and
+    # the surrounding prose do not match.
+    ITEMIZE_LINE = /\A[<>ch][fdLDS]\S*\s+(.+?)\s*\z/
+
+    module_function
+
+    # Real conflicts for one job, in the order rsync reports them.
+    # Empty when --update holds nothing back, or holds back only identical files.
+    def detect(cfg, job)
+      # Render jobs compare by content already and never use --update.
+      return [] if job.render
+      return [] unless job.source_exists && job.target_exists
+
+      held_back_paths(cfg, job).filter_map { |rel| entry_for(job, rel) }
+    end
+
+    # Relative paths that --update would skip: (would transfer forced) minus
+    # (would transfer normally).
+    def held_back_paths(cfg, job)
+      forced = itemized_paths(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: true))
+      return [] if forced.empty?   # nothing to move at all — no need to ask rsync twice
+
+      normal = itemized_paths(Twin::Sync.rsync_args(cfg, job, dry_run: true, force: false))
+      forced - normal
+    end
+
+    def itemized_paths(args)
+      output, status = Twin::Sync.run(args)
+      return [] unless status.success?
+      output.lines.filter_map do |line|
+        m = ITEMIZE_LINE.match(line)
+        next unless m
+        rel = m[1]
+        next if rel == "./" || rel.end_with?("/")   # directories carry no content
+        rel
+      end
+    end
+
+    # Build an Entry unless source and target hold the same bytes.
+    def entry_for(job, rel)
+      src = resolve(job.source_path, rel)
+      tgt = resolve(job.target_path, rel)
+
+      # Remote targets can't be read here; report them rather than assume.
+      unless job.remote?
+        return nil if same_content?(src, tgt)
+      end
+
+      Entry.new(
+        job: job, rel: rel, source_path: src, target_path: tgt,
+        source_mtime: mtime(src), target_mtime: job.remote? ? nil : mtime(tgt),
+      )
+    end
+
+    # A job path may be a single file — rsync then itemizes its basename, and
+    # the job path is already the full path.
+    def resolve(base, rel)
+      File.directory?(base) ? File.join(base, rel) : base
+    end
+
+    def same_content?(a, b)
+      return false unless File.file?(a) && File.file?(b)
+      return false unless File.size(a) == File.size(b)
+      digest(a) == digest(b)
+    rescue Errno::ENOENT, Errno::EACCES
+      false
+    end
+
+    def digest(path) = Digest::SHA256.file(path).hexdigest
+
+    def mtime(path)
+      File.mtime(path)
+    rescue Errno::ENOENT, Errno::EACCES
+      nil
+    end
+
+    # Unified diff for one entry, or a short note when it can't be produced.
+    def diff(entry)
+      return "  (remote target — no diff available)" if entry.job.remote?
+      return "  (binary or unreadable)" unless text?(entry.source_path) && text?(entry.target_path)
+
+      out, _status = Twin::Sync.run([
+        "diff", "-u",
+        "--label", "target (#{entry.target_path})", entry.target_path,
+        "--label", "source (#{entry.source_path})", entry.source_path,
+      ])
+      out.empty? ? "  (no textual difference)" : out
+    end
+
+    # Cheap heuristic: a NUL byte in the first 8 KiB means binary.
+    def text?(path)
+      File.open(path, "rb") { |f| !f.read(8192).to_s.include?("\0") }
+    rescue Errno::ENOENT, Errno::EACCES
+      false
+    end
+  end
+end

data/lib/twin/scanner.rb

@@ -6,7 +6,7 @@ require_relative "remote"
 module Twin
   # One YAML block from a sync-file, enriched with live filesystem state.
   Job = Struct.new(
-    :program, :path, :description, :active, :excludes, :label,
+    :program, :path, :description, :active, :excludes, :owned, :label,
     :source, :target, :cmd, :delete, :render, :render_outdated, :target_path_field, :sync_file,
     :source_exists, :target_exists, :source_mtime, :target_mtime, :conflict,
     :target_unreachable,
@@ -16,6 +16,10 @@ module Twin
     def target_path = File.join(target, target_path_field || path)
     def remote?     = Twin::Remote.remote?(target)
 
+    # Everything rsync must not touch: Exclude (not part of the sync at all)
+    # plus Own (part of the scope, but the target owns it).
+    def all_excludes = excludes + (owned || [])
+
     def status
       return :disabled if active != 1
       return :unreachable if target_unreachable
@@ -143,7 +147,12 @@ module Twin
       return nil if path.empty? || source.empty? || target.empty?
 
       render   = r["Render"] == true
-      excludes = (r["Exclude"] || "").split(",").map(&:strip).reject(&:empty?)
+      excludes = split_list(r["Exclude"])
+      # Own: paths inside the sync scope that the TARGET owns — machine-specific
+      # config the source must never clobber. Same rsync effect as Exclude, but
+      # kept apart so `status` can name the intent instead of hiding it among
+      # build artefacts and .DS_Store.
+      owned    = split_list(r["Own"])
       remote   = Twin::Remote.remote?(target)
 
       if render && remote
@@ -171,6 +180,7 @@ module Twin
         description:      r["Description"].to_s,
         active:           (r["Active"] || 0).to_i,
         excludes:         excludes,
+        owned:            owned,
         label:            r["Label"].to_s,
         source:           source,
         target:           target,
@@ -196,6 +206,11 @@ module Twin
       [false, nil]
     end
 
+    # Comma-separated block field → array of trimmed, non-empty entries.
+    def split_list(value)
+      (value || "").split(",").map(&:strip).reject(&:empty?)
+    end
+
     # For a render job: is the target out of date with the rendered template?
     # nil when source is missing/a directory (status falls through to those).
     # True when target is absent or content differs, or the template can't be

data/lib/twin/sync.rb

@@ -85,7 +85,7 @@ module Twin
 
     # Sync one Job. Returns [success, combined_output, transferred].
     # transferred is true when rsync actually moved bytes (false on no-op or dry_run).
-    def run_job(cfg, job, dry_run: false)
+    def run_job(cfg, job, dry_run: false, force: false)
       return render_job(cfg, job, dry_run: dry_run) if job.render
 
       src = job.source_path
@@ -102,12 +102,12 @@ module Twin
         FileUtils.mkdir_p(File.dirname(tgt))
       end
 
-      output, status = run(rsync_args(cfg, job, dry_run: dry_run))
+      output, status = run(rsync_args(cfg, job, dry_run: dry_run, force: force))
       return [false, output, false] unless status.success?
 
       xfr = !dry_run && transferred?(output)
 
-      if job.conflict && !xfr && !dry_run
+      if job.conflict && !xfr && !dry_run && !force
         output += "\nskipped: target is newer, source not synced"
       end
 
@@ -128,18 +128,23 @@ module Twin
     end
 
     # Full rsync argument vector for a job.
-    def rsync_args(cfg, job, dry_run: false)
+    #
+    # force: drop --update, so a file that is newer on the target is overwritten
+    # anyway. Only ever set after the user agreed to it (see Twin::Conflict), or
+    # via `twin sync --force`.
+    def rsync_args(cfg, job, dry_run: false, force: false)
       src = job.source_path
       tgt = job.target_path
 
-      args = ["rsync", "-av", "--itemize-changes", "--update"]
+      args = ["rsync", "-av", "--itemize-changes"]
+      args << "--update" unless force
       if job.delete
         args << "--delete"
         args.concat(backup_args(job))
       end
       args << "--dry-run" if dry_run
       cfg.global_excludes.each { |ex| args << "--exclude=#{ex}" }
-      job.excludes.each       { |ex| args << "--exclude=#{ex}" }
+      job.all_excludes.each   { |ex| args << "--exclude=#{ex}" }
 
       if File.directory?(src)
         args << "#{src}/" << "#{tgt}/"
@@ -166,8 +171,8 @@ module Twin
     end
 
     # Sync all jobs in a Program. Returns array of [job, success, output].
-    def run_program(cfg, program, dry_run: false)
-      program.active_jobs.map { |job| [job, *run_job(cfg, job, dry_run: dry_run)] }
+    def run_program(cfg, program, dry_run: false, force: false)
+      program.active_jobs.map { |job| [job, *run_job(cfg, job, dry_run: dry_run, force: force)] }
     end
 
     def run(args)

data/lib/twin/version.rb

@@ -1,3 +1,3 @@
 module Twin
-  VERSION = "0.3.0"
+  VERSION = "0.4.0"
 end

data/lib/twin.rb

@@ -4,6 +4,7 @@ require_relative "twin/template"
 require_relative "twin/config"
 require_relative "twin/scanner"
 require_relative "twin/sync"
+require_relative "twin/conflict"
 require_relative "twin/journal"
 require_relative "twin/add"
 require_relative "twin/preview"

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: mark-twin
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.4.0
 platform: ruby
 authors:
 - Ralf Hülsmann
@@ -40,6 +40,7 @@ files:
 - lib/twin/add.rb
 - lib/twin/cli.rb
 - lib/twin/config.rb
+- lib/twin/conflict.rb
 - lib/twin/journal.rb
 - lib/twin/picker.rb
 - lib/twin/preview.rb
@@ -81,7 +82,6 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 4.0.4
 specification_version: 4
-summary: Sync configuration folders between two Macs from self-documenting Markdown
-  files
+summary: Sync configuration between machines, from self-documenting Markdown files
 test_files: []
 ...