@kova1/pullr 0.7.0 → 0.9.0

package/README.md

@@ -13,13 +13,13 @@ everything current means `cd`-ing into each one.
 ```console
 $ pullr ~/work
 + api (main, 3 new commits)
-= web (main)
-~ docs (main, 2 commits held back by local changes)
++ web (main, 1 new commit, local changes reapplied)
+~ docs (main, 2 commits held back because local changes conflict with them)
 - scratch (main has no upstream)
 x infra (main)
     diverged from upstream (behind 2, ahead 1): cannot fast-forward
 
-Repos: 5  updated: 1  up to date: 1  dirty: 1  skipped: 1  failed: 1
+Repos: 5  updated: 2  up to date: 0  dirty: 1  skipped: 1  failed: 1
 Dirty: docs
 Failed: infra
 ```
@@ -55,10 +55,13 @@ first. It will never:
   failed and left exactly as it was. Rebasing it is opt-in with `--rebase`,
   and a rebase that hits a conflict is aborted, leaving the repository as it
   was.
-- **Stash, overwrite or discard your changes.** There is no auto-stash. If
-  local edits or untracked files are in the way of incoming changes, the
-  repository is reported as dirty and left as it was; edits that don't touch
-  incoming files don't stop the update.
+- **Leave your changes half-merged, or lose them.** When local edits or
+  untracked files are in the way of an update, pullr stashes them, updates,
+  and re-applies them on top. If they conflict with the incoming commits, it
+  puts the repository back exactly as it was - same commit, same files, same
+  staged and unstaged changes, no stash left behind - and reports it as
+  dirty. Edits that don't touch incoming files are never stashed at all.
+  `--no-autostash` skips the stash and just reports the repository as dirty.
 - **Touch a repository on a detached `HEAD`**, or a branch with no upstream.
   Both are skipped.
 - **Enter a nested repository.** Repositories inside another repository are
@@ -75,6 +78,9 @@ first. It will never:
 pullr [options] [DIR]
 ```
 
+Short options can be combined: `-rs` is `-r -s`, and `-j8` or `-rj 8` set
+the number of jobs.
+
 | Option                       | Default | Meaning |
 | :--- | :--- | :--- |
 | `DIR` | current directory | Where to look for repositories |
@@ -82,7 +88,8 @@ pullr [options] [DIR]
 | `--max-depth N` | `2` | How many directory levels below `DIR` to search. `0` = only `DIR` itself, `1` = `DIR` and its direct children, ... |
 | `-n`, `--dry-run` | | Show what a run would do without pulling, see below |
 | `-r`, `--rebase` | off | Rebase local commits onto the upstream instead of refusing a diverged branch (`git pull --rebase`). A rebase that hits a conflict is aborted and the repository is left as it was. |
-| `--submodules` | off | Also fast-forward each submodule that is checked out on a branch, see below |
+| `--no-autostash` | | Don't stash local changes that are in the way of an update; report the repository as dirty instead. Autostash is on by default, see "What it will never do". |
+| `-s`, `--submodules` | off | Also fast-forward each submodule that is checked out on a branch, see below |
 | `--no-color` | | Disable colored output. Also disabled when the [`NO_COLOR`](https://no-color.org) environment variable is set, or when output is not a terminal. |
 | `-V`, `--version` | | Print the version |
 | `-h`, `--help` | | Show help |
@@ -90,7 +97,7 @@ pullr [options] [DIR]
 ### Output
 
 Each repository gets one line: `+` updated, `=` already up to date, `~`
-dirty (local changes are in the way of the update, which is held back), `-`
+dirty (local changes conflict with the update, which is held back), `-`
 skipped, `x` failed (with the reason indented below it). Exits `1` if any
 pull failed, `2` on invalid arguments, `0` otherwise; a dirty repository is
 not a failure.
@@ -113,9 +120,11 @@ Failed: infra
 ```
 
 It runs `git fetch` in each repository first, so the answer reflects the
-remote as it is now, then prints the same lines a real run would (with
-`--rebase`, a diverged repository shows as `+ ... , 2 to rebase` instead of
-`x`; whether the rebase would conflict is only known by running it). Fetching
+remote as it is now, then prints the same lines a real run would. An update
+that would need to stash local changes shows as `+ ... , local changes to
+autostash`, and with `--rebase` a diverged repository shows as
+`+ ... , 2 to rebase` instead of `x`; whether re-applying or rebasing would
+conflict is only known by running it. Fetching
 only updates remote-tracking refs: the working tree and current branch are
 never modified. Exit codes match a real run.
 
@@ -148,7 +157,7 @@ skipped, and uninitialized ones are left out.
 | Pulls in parallel | 8 at a time by default | all at once, no limit | no | opt-in | opt-in |
 | Update | fetch + fast‑forward | `git pull` | fetch + fast‑forward | `git pull` | no built-in pull |
 | Merge commits | never | depends on your git config | never | depends on your git config | depends on your command |
-| Local changes in the way | reported as dirty | git's error | skipped | git's error | - |
+| Local changes in the way | stashed and reapplied; rolled back on conflict | git's error | skipped | git's error | - |
 | Dry run | yes | no | no | no | prints commands |
 
 **Why "8 at a time" and not "all at once".** Many self-hosted git servers

package/bin/pullr

@@ -1,9 +1,9 @@
 #!/usr/bin/env bash
 # Recursively find git repos under a directory and fast-forward pull each one.
-# Usage: pullr [--jobs N] [--max-depth N] [--dry-run] [--rebase] [--submodules] [--no-color] [DIR]
+# Usage: pullr [--jobs N] [--max-depth N] [--dry-run] [--rebase] [--no-autostash] [--submodules] [--no-color] [DIR]
 set -euo pipefail
 
-VERSION="0.7.0" # set by the release workflow
+VERSION="0.9.0" # set by the release workflow
 
 usage() {
   cat <<'EOF'
@@ -24,19 +24,27 @@ Options:
                      refusing a branch that has diverged (git pull --rebase).
                      A rebase that hits a conflict is aborted, leaving the
                      repository as it was.
-      --submodules   Also fast-forward each submodule that is checked out on a
+      --no-autostash Don't stash local changes that are in the way of an
+                     update. By default they are stashed, the update is
+                     made and they are re-applied; if re-applying conflicts,
+                     the repository is put back exactly as it was.
+      --autostash    The default; accepted for scripts that want to be explicit.
+  -s, --submodules   Also fast-forward each submodule that is checked out on a
                      branch, with the same rules as any other repository.
                      Submodules on a detached HEAD are skipped.
       --no-color     Disable colored output. Also disabled when NO_COLOR is
                      set (https://no-color.org) or output is not a terminal.
   -V, --version      Print the version.
   -h, --help         Show this help.
+
+Short options can be combined: -rs is -r -s, and -j8 or -rj 8 set the jobs.
 EOF
 }
 
 mode=pull
 jobs=8
 rebase=0
+autostash=1
 submodules=0
 color=1
 max_depth=2
@@ -44,13 +52,25 @@ root=""
 
 while (($#)); do
   case "$1" in
+    # bundled short options: split -rs into -r -s, and -j8 into -j 8
+    -[!-]?*)
+      opt="${1:1:1}" rest="${1:2}"
+      shift
+      if [[ "$opt" == j ]]; then
+        set -- -j "$rest" "$@"
+      else
+        set -- "-$opt" "-$rest" "$@"
+      fi
+      ;;
     -j|--jobs) jobs="${2:-}"; shift 2 || { usage >&2; exit 2; } ;;
     --jobs=*) jobs="${1#*=}"; shift ;;
     --max-depth) max_depth="${2:-}"; shift 2 || { usage >&2; exit 2; } ;;
     --max-depth=*) max_depth="${1#*=}"; shift ;;
     -n|--dry-run) mode=dryrun; shift ;;
     -r|--rebase) rebase=1; shift ;;
-    --submodules) submodules=1; shift ;;
+    --autostash) autostash=1; shift ;;
+    --no-autostash) autostash=0; shift ;;
+    -s|--submodules) submodules=1; shift ;;
     --no-color) color=0; shift ;;
     -h|--help) usage; exit 0 ;;
     -V|--version) echo "pullr $VERSION"; exit 0 ;;
@@ -125,8 +145,8 @@ report_diverged() {
 
 report_dirty() {
   printf 'dirty\t%s\n' "$1"
-  printf '%s~%s %s %s(%s, %s held back by local changes)%s\n' \
-    "$yellow" "$reset" "$1" "$dim" "$2" "$(commits "$3")" "$reset"
+  printf '%s~%s %s %s(%s, %s held back %s)%s\n' \
+    "$yellow" "$reset" "$1" "$dim" "$2" "$(commits "$3")" "${4:-by local changes}" "$reset"
 }
 
 report_current() {
@@ -138,6 +158,7 @@ report_updated() {
   local note
   note="$3 new commit$([[ "$3" == 1 ]] || printf s)"
   [[ "$4" == 0 ]] || note+=", $4 $5"
+  [[ -z "${6:-}" ]] || note+=", $6"
   echo updated
   printf '%s+%s %s %s(%s, %s)%s\n' "$green" "$reset" "$1" "$dim" "$2" "$note" "$reset"
 }
@@ -178,6 +199,8 @@ pull_repo() {
     # git refuses, touching nothing, when local changes are in the way
     if out="$(git -C "$repo" merge --ff-only --quiet '@{upstream}' 2>&1)"; then
       report_synced "$repo" "$rel" "$branch" "$behind" 0 ""
+    elif has_local_changes "$repo" && [[ "$autostash" == 1 ]]; then
+      autostash_update "$repo" "$rel" "$branch" "$behind" 0 ff
     elif has_local_changes "$repo"; then
       report_dirty "$rel" "$branch" "$behind"
     else
@@ -185,6 +208,8 @@ pull_repo() {
     fi
   elif [[ "$rebase" == 0 ]]; then
     report_diverged "$rel" "$branch" "$behind" "$ahead"
+  elif has_tracked_changes "$repo" && [[ "$autostash" == 1 ]]; then
+    autostash_update "$repo" "$rel" "$branch" "$behind" "$ahead" rebase
   elif has_tracked_changes "$repo"; then
     report_dirty "$rel" "$branch" "$behind"
   elif out="$(git -C "$repo" rebase --quiet '@{upstream}' 2>&1)"; then
@@ -202,11 +227,87 @@ pull_repo() {
   fi
 }
 
+# Local changes are in the way of the update ($6: ff or rebase). Stash them,
+# update, and re-apply them. If re-applying is anything but clean, put the
+# repository back exactly as it was and report it as dirty. The stash is only
+# dropped once its changes are back in the working tree.
+autostash_update() {
+  local repo="$1" rel="$2" branch="$3" behind="$4" ahead="$5" how="$6"
+  local before stash out
+  before="$(git -C "$repo" rev-parse HEAD)"
+  if ! out="$(git -C "$repo" stash push --include-untracked --quiet -m "pullr autostash" 2>&1)" ||
+    ! stash="$(git -C "$repo" rev-parse --verify --quiet refs/stash)"; then
+    report_failed "$rel" "$branch" "could not stash local changes:"$'\n'"$out"
+    return
+  fi
+
+  if [[ "$how" == ff ]]; then
+    out="$(git -C "$repo" merge --ff-only --quiet '@{upstream}' 2>&1)"
+  else
+    out="$(git -C "$repo" rebase --quiet '@{upstream}' 2>&1)"
+  fi || {
+    # Not caused by local changes - they were stashed. Undo, then restore
+    # them onto the commit they came from, where they apply cleanly.
+    git -C "$repo" rebase --abort >/dev/null 2>&1 || true
+    if restore_stash "$repo" "$stash"; then
+      report_failed "$rel" "$branch" "$out"
+    else
+      report_failed "$rel" "$branch" "$out"$'\n'"$(stash_kept "$repo")"
+    fi
+    return
+  }
+
+  if git -C "$repo" stash apply --index --quiet "$stash" >/dev/null 2>&1; then
+    drop_stash "$repo" "$stash"
+    local verb=""
+    [[ "$how" == ff ]] || verb=rebased
+    report_synced "$repo" "$rel" "$branch" "$behind" "$([[ "$how" == ff ]] && echo 0 || echo "$ahead")" "$verb" "local changes reapplied"
+    return
+  fi
+
+  # Conflict: go back to where we started.
+  git -C "$repo" reset --hard --quiet "$before"
+  clear_restored_untracked "$repo" "$stash"
+  if restore_stash "$repo" "$stash"; then
+    report_dirty "$rel" "$branch" "$behind" "because local changes conflict with them"
+  else
+    report_failed "$rel" "$branch" "could not re-apply local changes after an update conflict."$'\n'"$(stash_kept "$repo")"
+  fi
+}
+
+# Re-apply a stash onto the commit it was made from, then drop it.
+restore_stash() {
+  git -C "$1" stash apply --index --quiet "$2" >/dev/null 2>&1 || return 1
+  drop_stash "$1" "$2"
+}
+
+# `git stash drop` takes stash@{n}, not a commit: find the entry that is ours.
+drop_stash() {
+  local ref
+  ref="$(git -C "$1" stash list --format='%gd %H' | awk -v c="$2" '$2 == c { print $1; exit }')"
+  [[ -z "$ref" ]] || git -C "$1" stash drop --quiet "$ref" >/dev/null 2>&1 || true
+}
+
+# A failed `stash apply` may already have written the stash's untracked files
+# back. They were moved into the stash, so anything at those paths now came
+# from that apply; remove them so the stash can be applied again.
+clear_restored_untracked() {
+  local path
+  git -C "$1" rev-parse --verify --quiet "$2^3" >/dev/null || return 0
+  while IFS= read -r path; do
+    [[ -z "$path" ]] || rm -f -- "$1/$path"
+  done < <(git -C "$1" ls-tree -r --name-only "$2^3")
+}
+
+stash_kept() {
+  printf 'Your local changes are safe in the stash (git -C %q stash list).' "$1"
+}
+
 # The repository was updated; report it, unless syncing submodules failed.
 report_synced() {
   local out
   if out="$(sync_submodules "$1")"; then
-    report_updated "$2" "$3" "$4" "$5" "$6"
+    report_updated "$2" "$3" "$4" "$5" "$6" "${7:-}"
   else
     report_failed "$2" "$3" "updated, but updating submodules failed:"$'\n'"$out"
   fi
@@ -230,13 +331,17 @@ dryrun_repo() {
   if [[ "$behind" == 0 ]]; then
     report_current "$rel" "$branch"
   elif [[ "$ahead" == 0 ]]; then
-    if changes_block_update "$repo"; then
+    if changes_block_update "$repo" && [[ "$autostash" == 1 ]]; then
+      report_updated "$rel" "$branch" "$behind" 0 "" "local changes to autostash"
+    elif changes_block_update "$repo"; then
       report_dirty "$rel" "$branch" "$behind"
     else
       report_updated "$rel" "$branch" "$behind" 0 ""
     fi
   elif [[ "$rebase" == 0 ]]; then
     report_diverged "$rel" "$branch" "$behind" "$ahead"
+  elif has_tracked_changes "$repo" && [[ "$autostash" == 1 ]]; then
+    report_updated "$rel" "$branch" "$behind" "$ahead" "to rebase" "local changes to autostash"
   elif has_tracked_changes "$repo"; then
     report_dirty "$rel" "$branch" "$behind"
   else
@@ -316,8 +421,8 @@ if ((jobs > 1 && count > 1)); then
   scratch="$(mktemp -d)"
   trap 'rm -rf "$scratch"' EXIT
   export GIT_TERMINAL_PROMPT=0
-  export mode rebase submodules root green yellow red dim reset
-  export -f worker pull_repo dryrun_repo repo_label indented prepare report_failed report_diverged report_dirty report_current report_updated report_synced commits has_tracked_changes has_local_changes sync_submodules changes_block_update
+  export mode rebase autostash submodules root green yellow red dim reset
+  export -f worker pull_repo dryrun_repo repo_label indented prepare report_failed report_diverged report_dirty report_current report_updated report_synced autostash_update restore_stash drop_stash clear_restored_untracked stash_kept commits has_tracked_changes has_local_changes sync_submodules changes_block_update
   # shellcheck disable=SC2016  # the single-quoted body runs inside the child shell
   for ((i = 0; i < count; i++)); do printf '%s\0%s\0' "$i" "${repos[i]}"; done |
     xargs -0 -n 2 -P "$jobs" "$BASH" -c 'worker "$@"' _ "$scratch" &

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@kova1/pullr",
-  "version": "0.7.0",
+  "version": "0.9.0",
   "description": "Safe, fast-forward pull every git repository under a directory",
   "keywords": [
     "git",