claude-multiacc 1.0.5 → 1.0.7

package/README.md

@@ -9,7 +9,7 @@ no binary edits, no install-dir writes, survives `claude update` — the real bi
 resolved dynamically at exec time.
 
 Tested on: macOS (bash 3.2, zsh, Claude Code 2.1.207) and Ubuntu 24.04
-(bash 5.2, Claude Code 2.1.201). Test suite: `tests/run-tests.sh` (35 sandboxed tests,
+(bash 5.2, Claude Code 2.1.201). Test suite: `tests/run-tests.sh` (221 sandboxed tests,
 no network / no quota — passes on both platforms).
 
 ---
@@ -31,12 +31,18 @@ repo bin/ first on PATH (rc-file block)    /root/claude-multiacc/  (addon repo)
 
 1. `CLAUDE_CONFIG_DIR` or `CLAUDE_CODE_OAUTH_TOKEN` already set → pass straight through.
 2. `CLAUDE_ACCOUNT=acct-NN` → use that account (even if limit-marked — explicit wins).
-3. Among accounts valid **on this machine** (`.credentials.json` or `server.token`
-   present) and not excluded, pick the one with the **most remaining headroom** (see
-   below). Ties break randomly to spread load. `CLAUDE_SHIM_SELECT=random` restores plain
-   uniform-random selection.
-4. If every account is excluded → the highest-headroom valid account anyway + a warning
-   in `selection.log` (degraded beats down: the 100% rule).
+3. Drop every account that cannot serve a request — a **dead login** (expired refresh
+   token, revoked grant, a run that failed to authenticate) or an **org-blocked** one
+   (Claude Code disabled for that account). Those can only fail, so they never enter
+   selection.
+4. Among the accounts that remain valid **on this machine** (`.credentials.json` or
+   `server.token` present) and not limit-excluded, pick the one with the **most remaining
+   headroom** (see below). Ties break randomly to spread load. `CLAUDE_SHIM_SELECT=random`
+   restores plain uniform-random selection.
+5. If every account is limit-excluded → the highest-headroom *authenticable* account
+   anyway + a warning in `selection.log` (degraded beats down: the 100% rule).
+6. If nothing is usable at all → stock passthrough, with the reason in `selection.log`
+   (and, on a terminal, one line naming the accounts to fix).
 
 **Which account has "the most headroom"?** A Max subscription has three limits that a
 Fable request consumes at once: the 5-hour **session** bucket, the **weekly all-models**
@@ -51,6 +57,35 @@ lowest wins.) **Exclusion** still fires on *any* bucket ≥ 90% — a full sessi
 really does block right now — but that marker expires when the session resets, not days
 later.
 
+**Unusable accounts are excluded, not rotated into.** An account that cannot serve a
+request is dropped from selection before anything else is considered — it is not eligible
+as the all-limited fallback either, because a limit-marked account is *degraded* while
+this one is *down*, and handing work to it guarantees a hard failure. Two kinds:
+
+- **Dead login** (`EXPIRED`) — refresh token expired, grant revoked, or a real call that
+  came back *"OAuth session expired and could not be refreshed"*. Fixed by
+  `claude-accounts relogin`.
+- **Org-blocked** (`BLOCKED`) — the account authenticates fine, but the run comes back
+  *"Your organization has disabled Claude subscription access for Claude Code"*. Handled
+  like any other dead login (`claude-accounts relogin` re-issues the grant and normally
+  clears it); if it comes back BLOCKED after a re-login, an admin has to enable Claude
+  Code for that account. Telemetry cannot see this — those accounts' usage endpoint
+  answers normally — so only a real call (`verify`, or a `-p` run) detects it, and the
+  park deliberately survives credential refreshes (a new access token says nothing about
+  an org policy).
+
+`claude-accounts expired` lists exactly what is excluded, why, and the fix for each.
+Exclusion self-heals, but only against the evidence it was based on: a **credential**
+park (dead login) clears as soon as a newer credential lands — a re-login, or a refresh
+by another process — or when a usage fetch authenticates; a **policy** park (org block)
+ignores both, because a fresh access token says nothing about whether the org re-enabled
+Claude Code, and only a passing real call or a re-login lifts it. Parks the **shim**
+writes from a single failed run additionally carry a `soft_until` stamp and expire on
+their own — the shim's guess never outlives the evidence for it — while parks
+`claude-accounts` writes from proof (an `invalid_grant` refresh response, a failed real
+call) stay until the account works again. If *every* account is unusable the shim still
+fails open into stock `claude`, and logs why to `selection.log`.
+
 The shim prints nothing, logs `timestamp account cwd` (never prompt text) to
 `selection.log`, and `exec`s the real binary — stdin/stdout/exit codes pass through
 byte-identically. If anything is missing (no manifest, no accounts, unreadable state,
@@ -86,8 +121,12 @@ then says re-login is needed). Overrides: `CLAUDE_MULTIACC_TOKEN_URL`,
 `CLAUDE_MULTIACC_CLIENT_ID` (used by the sandboxed tests; defaults are correct for real use).
 
 **Auto-retry** (`-p`/`--print` only, default on, `CLAUDE_SHIM_RETRY=0` disables): on an
-auth/rate-limit-looking failure the shim marks the account with a 10-minute error
-cooldown and retries once on a different account. Output is buffered so a retried call
+auth- or rate-limit-looking failure the shim retries once on a different account and
+marks the failed one — a **10-minute cooldown** for a rate limit (it heals on its own),
+or an **`.expired`** park for an authentication failure / org block (they do not). The
+park patterns are deliberately narrow and carry a `soft_until` stamp, because this grep
+also sees the model's own answer: a `-p` run that merely *mentions* a 403 must never cost
+an account, and if one slips through it returns to the pool by itself. Output is buffered so a retried call
 never double-emits. Only engages when stdin is finite (tty / regular file / `/dev/null`)
 and ≥2 accounts are eligible; service-spawned pipes take the plain exec path untouched.
 
@@ -151,10 +190,42 @@ What install does (all reversible, nothing else):
 ```bash
 claude-accounts list      # pool at a glance
 claude-accounts status    # per-account auth health, per-bucket limits incl. Fable, markers, last pick
+claude-accounts expired   # ONLY the accounts that cannot authenticate, and why (exit 1 if any)
+claude-accounts relogin   # sign those back in, one at a time (ids or --all to pick your own)
 ```
 
 Every account in the pool is authenticated — `add` cannot register one otherwise.
 
+### When an account stops working
+
+Subscription logins expire (the refresh token has a finite life), can be revoked from
+claude.ai, and an organization can disable Claude Code for one of its accounts entirely.
+Such an account is **never selected** — `claude` keeps working on the healthy ones — and
+shows up here:
+
+```bash
+claude-accounts expired
+#   acct-01   support@example.com   EXPIRED   refresh token expired 2026-08-05 (2d ago)
+#                                             fix: claude-accounts relogin acct-01
+#   acct-08   dev@example.com       BLOCKED   this account's organization has disabled
+#                                             Claude Code subscription access
+#                                             fix: claude-accounts relogin acct-08
+
+claude-accounts relogin              # re-authenticate everything a sign-in can fix
+claude-accounts relogin acct-01      # or just one
+claude-accounts relogin acct-01 --token   # portable token instead (Mac + server)
+```
+
+`relogin` covers `BLOCKED` accounts too — a fresh sign-in re-issues the grant and usually
+clears the block. Use `claude-accounts verify` (a real `-p` call per account) to re-test
+one at any time; a PASS clears the park automatically.
+
+`relogin` runs the same verified sign-in as `login` (it refuses to save a credential for
+the wrong email), one account at a time, and syncs to the server once at the end.
+`expired` exits **1** when anything needs a human, so it drops straight into a cron or
+monitoring check. `list` tags such accounts `EXPIRED-LOGIN`, and `status` says
+`selectable: NO` with the reason.
+
 ### Adopt this machine's existing login (no browser needed)
 
 ```bash
@@ -240,7 +311,7 @@ failures are loud and non-zero.
 ## Verification
 
 ```bash
-tests/run-tests.sh            # 35 sandboxed tests, no quota
+tests/run-tests.sh            # 221 sandboxed tests, no quota
 claude-accounts verify        # real matrix: `claude -p "reply OK"` per authed account
 claude-accounts verify --quick# auth presence/expiry only, no inference
 claude-accounts limits        # live per-bucket usage incl. the Fable bucket
@@ -259,7 +330,14 @@ repointed via `CLAUDE_BIN=/usr/local/bin/claude` and restarted healthy.
   that the marked block is the LAST PATH manipulation in your rc file
   (`grep -A2 'claude-multiacc >>>' ~/.zshrc`).
 - **An account never gets picked** — `claude-accounts status`: no auth on this machine,
-  or an active `.limited` marker (shows bucket + minutes to reset).
+  a dead login (`selectable: NO`, see `claude-accounts expired`), or an active `.limited`
+  marker (shows bucket + minutes to reset).
+- **"Failed to authenticate: OAuth session expired and could not be refreshed"** — the
+  login behind that run is dead. Run `claude-accounts expired` to see which, then
+  `claude-accounts relogin`. Accounts in that state are excluded from selection
+  automatically, so this only surfaces when *every* account needs a re-login (the shim
+  then falls back to the machine's own `~/.claude` login) or when it was pinned with
+  `CLAUDE_ACCOUNT`.
 - **Everything marked limited** — the shim still runs (least-utilized fallback);
   check `selection.log` for `all-limited` lines.
 - **Sync fails** — `tail ~/.claude-accounts/sync.log`; it's ssh/rsync to the manifest's

package/bin/claude

@@ -4,6 +4,9 @@
 # Self-contained on purpose: no sourcing, so a broken repo file can never break `claude`.
 # Selection: CLAUDE_CONFIG_DIR/CLAUDE_CODE_OAUTH_TOKEN passthrough > CLAUDE_ACCOUNT pin >
 # random among limit-eligible accounts > least-utilized fallback (degraded beats down).
+# Accounts whose login is DEAD (expired refresh token, or a `.expired` marker from a
+# failed auth) are never selected — not even as the all-limited fallback — because they
+# fail every call outright; `claude-accounts expired` / `relogin` fix them.
 
 set -u
 
@@ -157,6 +160,77 @@ over_threshold() { # $1 = acct dir
 # dead account selectable and turn a working stock run into an auth failure).
 has_auth() { [ -s "$1/.credentials.json" ] || [ -s "$1/server.token" ]; }
 
+cred_num() { # cred_num <file> <json key> -> integer (epoch ms) or empty
+  # The leading quote makes "expiresAt" unambiguous: "refreshTokenExpiresAt" cannot match it.
+  LC_ALL=C sed -n "s/.*\"$2\"[[:space:]]*:[[:space:]]*\([0-9][0-9]*\).*/\1/p" "$1" 2>/dev/null | head -1
+}
+
+# TRUE when <dir>/.credentials.json exists but can no longer authenticate: the access
+# token is expired AND nothing can renew it (no refresh token, or the refresh token is
+# itself expired). Such an account fails EVERY run with "OAuth session expired and
+# could not be refreshed", so selecting it is strictly worse than not having it.
+# Anything unparseable fails OPEN (treated as alive) — telemetry-free guesswork must
+# never take a working account out of the pool.
+creds_dead() { # $1 = acct dir
+  local f="$1/.credentials.json" exp rexp
+  [ -s "$f" ] || return 1
+  exp="$(cred_num "$f" expiresAt)"
+  case "$exp" in ''|*[!0-9]*) exp=0 ;; esac
+  [ $((exp / 1000)) -gt "$now" ] && return 1        # access token still valid
+  # Access token expired: only a live refresh token can save it.
+  grep -q '"refreshToken"[[:space:]]*:[[:space:]]*"[^"]' "$f" 2>/dev/null || return 0
+  rexp="$(cred_num "$f" refreshTokenExpiresAt)"
+  case "$rexp" in ''|*[!0-9]*) return 1 ;; esac     # unknown expiry => fail open
+  [ $((rexp / 1000)) -le "$now" ]
+}
+
+# `.expired` is the persistent "this account needs a re-login" marker: written by
+# claude-accounts (refresh grant expired/revoked) and by the retry path below when a
+# real call fails with an auth error. It SELF-HEALS: any credential written after the
+# marker (successful re-login, or a refresh by another process) clears it, and a
+# successful `claude-accounts limits` fetch removes it outright.
+# A marker written by the SHIM (a guess from one failed run) also carries
+# `soft_until=<epoch>`: once that passes the account returns to the pool by itself, so
+# a misread never costs an account permanently. Markers written by claude-accounts —
+# a refresh grant that answered "revoked", a real verify call — carry no soft_until and
+# stay until the account provably works again.
+expired_marked() { # $1 = acct dir
+  local m="$1/.expired" mt f soft reason
+  [ -f "$m" ] || return 1
+  reason="$(LC_ALL=C sed -n 's/.*reason=\([A-Za-z0-9._-][A-Za-z0-9._-]*\).*/\1/p' "$m" 2>/dev/null | head -1)"
+  # CREDENTIAL-scoped parks clear the moment a newer credential lands — that is the
+  # evidence they were about. A POLICY park (org-blocked) is about the account, not the
+  # credential: refreshing its token does not re-enable Claude Code for it. Letting a
+  # credential rewrite clear it silently returned blocked accounts to the pool every
+  # few hours (the limits refresher rewrites credentials), so runs kept failing with
+  # "Your organization has disabled Claude subscription access".
+  if [ "$reason" != "org-blocked" ]; then
+    mt="$(file_mtime "$m")"
+    for f in "$1/.credentials.json" "$1/server.token"; do
+      if [ -f "$f" ] && [ "$(file_mtime "$f")" -gt "$mt" ]; then
+        rm -f "$m" 2>/dev/null
+        return 1
+      fi
+    done
+  fi
+  soft="$(LC_ALL=C sed -n 's/.*soft_until=\([0-9][0-9]*\).*/\1/p' "$m" 2>/dev/null | head -1)"
+  case "$soft" in
+    ''|*[!0-9]*) return 0 ;;                       # no soft stamp => proven dead, keep
+    *) [ "$now" -lt "$soft" ] && return 0
+       rm -f "$m" 2>/dev/null                      # soft window elapsed: give it another go
+       return 1 ;;
+  esac
+}
+
+# Auth that cannot work right now. A non-empty server.token is an independent
+# credential (no expiry we can read), so it keeps an account alive even when the
+# OAuth credential beside it is dead — the token is exported instead.
+auth_dead() { # $1 = acct dir
+  expired_marked "$1" && return 0
+  [ -s "$1/server.token" ] && return 1
+  creds_dead "$1"
+}
+
 # Explicit pin wins over everything — markers, and even missing auth: the
 # add/login ceremony pins to a dir that has no credentials yet, and the login
 # must land exactly there, never in a randomly selected account's dir.
@@ -177,17 +251,54 @@ fi
 
 valid=()
 eligible=()
+expired=()
 for d in "$ACC_ROOT"/acct-*; do
   [ -d "$d" ] || continue
   has_auth "$d" || continue
+  # Expired logins are excluded BEFORE anything else: unlike a limit marker (degraded
+  # but working), dead auth guarantees a hard "OAuth session expired" failure, so it
+  # can never be the "degraded beats down" fallback either.
+  if auth_dead "$d"; then
+    expired+=("$d")
+    continue
+  fi
   valid+=("$d")
   marker_active "$d" && continue
   over_threshold "$d" && continue
   eligible+=("$d")
 done
 
-# No usable accounts => stock behavior (fail open, never block work).
-[ "${#valid[@]}" -gt 0 ] || exec "$REAL" "$@"
+if [ "${#expired[@]}" -gt 0 ]; then
+  ids=""
+  for d in "${expired[@]}"; do ids="$ids $(basename "$d")"; done
+  sel_log "skipped-expired:$ids (unusable — see: claude-accounts expired)"
+  # One actionable line, at most hourly, and only on a terminal — a service-spawned
+  # `claude -p` must keep its stderr byte-clean.
+  if [ -t 2 ]; then
+    n="$ACC_ROOT/.expired-notice"
+    last=0
+    [ -f "$n" ] && last="$(file_mtime "$n")"
+    if [ $((now - last)) -gt 3600 ]; then
+      : > "$n" 2>/dev/null || true
+      printf 'claude-multiacc: %s account(s) unusable (%s) — see: claude-accounts expired\n' \
+        "${#expired[@]}" "${ids# }" >&2
+    fi
+  fi
+fi
+
+# No usable accounts => stock behavior (fail open, never block work), but say WHY when
+# the pool is merely un-authenticated: this is the one case the user can actually fix.
+if [ "${#valid[@]}" -eq 0 ]; then
+  if [ "${#expired[@]}" -gt 0 ]; then
+    sel_log "all-expired: falling back to the default login (see: claude-accounts expired)"
+    # Terminal only: a service-spawned `claude -p` must keep its stderr byte-clean, and
+    # the fallback may well succeed on the machine's own login. The reason is always in
+    # selection.log, and `claude-accounts expired` spells it out.
+    [ -t 2 ] && printf 'claude-multiacc: no pool account is usable (%s) — see: claude-accounts expired, then: claude-accounts relogin\n' \
+      "${ids# }" >&2
+  fi
+  exec "$REAL" "$@"
+fi
 
 # Pick the account with the MOST headroom (lowest ranking score = most weekly headroom,
 # session as tiebreaker). Ties break randomly so equally-idle accounts still spread load.
@@ -242,8 +353,10 @@ fi
 acct="$(basename "$pick")"
 sel_log "$acct weekly=$(fresh_field "$pick" weekly_percent || echo '?')% session=$(fresh_field "$pick" session_percent || echo '?')% pwd=$PWD"
 
-acct_token() { # $1 = acct dir; prints token if dir is token-auth (no local creds)
-  if [ ! -f "$1/.credentials.json" ] && [ -s "$1/server.token" ]; then
+acct_token() { # $1 = acct dir; prints token if the dir must authenticate by token
+  # Token-auth dirs (no local creds), and dirs whose OAuth credential is dead but which
+  # still carry a portable setup-token — the token is the only thing that can work there.
+  if [ -s "$1/server.token" ] && { [ ! -f "$1/.credentials.json" ] || creds_dead "$1"; }; then
     tr -d '[:space:]' < "$1/server.token"
   fi
 }
@@ -310,7 +423,22 @@ if [ ! -t 0 ]; then
   cat > "$stdin_file" 2>/dev/null || [ -s "$stdin_file" ] || stdin_file=""
 fi
 
-ERRPAT='rate[ _-]?limit|usage limit|limit (reached|exceeded)|overloaded|"?529"?|401|403|unauthorized|authentication[_ ]error|invalid[_ ](bearer|token|api key)|token (expired|revoked|invalid)|oauth.*(error|expired|invalid)|credit balance'
+# ERRPAT decides whether to RETRY at all (deliberately broad, unchanged behavior).
+# The two PARK patterns below decide whether the failed account is also taken out of
+# the pool, and they are deliberately NARROW: this grep also sees the model's own
+# answer on stdout (a `-p` run that merely *discusses* a 403 must not cost an account).
+#   PARK_AUTH  — the credential is dead: a cooldown would just re-fail, so the account
+#                is parked until a re-login / a fetch that authenticates clears it.
+#   PARK_ORG   — the account's organization turned Claude Code access off; no re-login
+#                fixes that.
+# Both shim-written parks carry a soft_until stamp, so even a false positive returns to
+# the pool on its own — the shim's guess must never outlive the evidence for it.
+PARK_AUTH='failed to authenticate|oauth (session|token)[a-z ]{0,20}(expired|invalid|revoked)|could not be refreshed|invalid api key|authentication_error|invalid bearer token|please run /login|run /login to'
+PARK_ORG='organization has disabled|subscription access[a-z ]{0,20}disabl|disabled claude subscription|ask your admin to enable|not authorized to use claude code'
+LIMITPAT='rate[ _-]?limit|usage limit|limit (reached|exceeded)|overloaded|"?529"?|credit balance'
+ERRPAT="$LIMITPAT|$PARK_AUTH|$PARK_ORG"'|401|403|unauthorized|authentication[_ ]error|invalid[_ ](bearer|token|api key)|token (expired|revoked|invalid)|oauth.*(error|expired|invalid)'
+PARK_SOFT_AUTH=3600      # 1h: a mis-parked healthy account is back within the hour
+PARK_SOFT_ORG=21600      # 6h: an org policy will not change in minutes
 
 attempt=1
 cur="$pick"
@@ -327,14 +455,39 @@ while :; do
   exec 3<&-
   if [ "$rc" -ne 0 ] && [ "$attempt" -eq 1 ] \
     && grep -qiE "$ERRPAT" "$tmpd/out" "$tmpd/err" 2>/dev/null; then
-    # Atomic marker write: a reader must never observe a half-written .limited
+    # Atomic marker writes: a reader must never observe a half-written marker
     # (it would parse as garbage and, before, could be deleted as "expired").
-    {
-      echo $((now + 600))
-      echo "bucket=error-cooldown percent=? marked_at=$(date -u +%Y-%m-%dT%H:%M:%SZ) reason=error-cooldown"
-    } > "$cur/.limited.$$" 2>/dev/null \
-      && mv -f "$cur/.limited.$$" "$cur/.limited" 2>/dev/null \
-      || rm -f "$cur/.limited.$$" 2>/dev/null || true
+    # A rate limit wins the classification: a limit message that happens to mention an
+    # auth word must get the self-expiring cooldown, never a park.
+    park_reason=""
+    park_soft=0
+    if grep -qiE "$LIMITPAT" "$tmpd/out" "$tmpd/err" 2>/dev/null; then
+      :
+    elif grep -qiE "$PARK_ORG" "$tmpd/out" "$tmpd/err" 2>/dev/null; then
+      park_reason="org-blocked"
+      park_detail="the account's organization has disabled Claude Code subscription access"
+      park_soft=$((now + PARK_SOFT_ORG))
+    elif grep -qiE "$PARK_AUTH" "$tmpd/out" "$tmpd/err" 2>/dev/null; then
+      park_reason="auth-error"
+      park_detail="run failed to authenticate"
+      park_soft=$((now + PARK_SOFT_AUTH))
+    fi
+    if [ -n "$park_reason" ]; then
+      {
+        echo "$now"
+        echo "reason=$park_reason soft_until=$park_soft marked_at=$(date -u +%Y-%m-%dT%H:%M:%SZ) detail=$park_detail"
+      } > "$cur/.expired.$$" 2>/dev/null \
+        && mv -f "$cur/.expired.$$" "$cur/.expired" 2>/dev/null \
+        || rm -f "$cur/.expired.$$" 2>/dev/null || true
+      sel_log "$(basename "$cur") parked ($park_reason until $park_soft) — see: claude-accounts expired"
+    else
+      {
+        echo $((now + 600))
+        echo "bucket=error-cooldown percent=? marked_at=$(date -u +%Y-%m-%dT%H:%M:%SZ) reason=error-cooldown"
+      } > "$cur/.limited.$$" 2>/dev/null \
+        && mv -f "$cur/.limited.$$" "$cur/.limited" 2>/dev/null \
+        || rm -f "$cur/.limited.$$" 2>/dev/null || true
+    fi
     next=""
     n="${#eligible[@]}"
     start=$((RANDOM % n))