@drakon-systems/multi-clawd 1.9.8 → 1.10.1

package/README.md

@@ -223,6 +223,12 @@ claude-cli/claude-fable-5        # main login
   stream-json and records per-window health to
   `~/.openclaw/state/multi-clawd/<account>.json`. Passthrough-first: a state
   write can never break a live turn.
+- 📡 **Live usage (v1.10)** — the pool also *asks* the provider what is left,
+  on a timer, using the login each account already holds. Both windows come
+  back as real percentages with reset times — including the 5-hour window the
+  stream never puts a number on — so hand-over happens at 85% of the session
+  window, not at the error. Passing 95% raises an operator alert through the
+  heartbeat; `multi-clawd usage` shows it on demand.
 - 🧰 **Full harness on every hop** — each backend is a genuine Claude Code
   subprocess: native tools, skills, MCP bridge, and native compaction all
   stay intact when failover steps across accounts.
@@ -376,7 +382,7 @@ openclaw plugins install (Get-Location).Path
 **Or let your agent install it.** Running an OpenClaw assistant or Claude
 Code on the target machine already? Paste it this and go make coffee:
 
-> Read https://raw.githubusercontent.com/Drakon-Systems-Ltd/multi-clawd/v1.9.8/SETUP-AGENT.md
+> Read https://raw.githubusercontent.com/Drakon-Systems-Ltd/multi-clawd/v1.10.1/SETUP-AGENT.md
 > and follow it to set up multi-clawd on this machine. I own a second
 > Claude account — ask me when you need me to log in.
 
@@ -574,9 +580,11 @@ How it decides, per launch (all data from each account's live
 | no data / stale data | used — never rotate on missing evidence |
 | whole pool exhausted | home account anyway → real limit error → your chain drops provider |
 
-**Why the short-window rule exists (v1.7.2).** The threshold rule needs a
-utilization number, and Anthropic does not send one for the 5-hour session
-window — across every observation we hold from two accounts it arrives as a
+**Why the short-window rule exists (v1.7.2).** (v1.10's live usage poll now
+supplies the missing number — see below — so this rule is the fallback for
+hosts where polling is off or the account cannot be polled.) The threshold
+rule needs a utilization number, and Anthropic does not send one for the
+5-hour session window — across every observation we hold from two accounts it arrives as a
 bare status plus a reset time, while the weekly windows carry percentages. A
 rule that waits for a number therefore could never pre-empt the session
 limit: the pool would take the hit and rotate afterwards. So on hour-scoped
@@ -596,6 +604,77 @@ Notes:
   which runs on every launch on every turn path. Details in
   [`DESIGN.md`](./DESIGN.md).
 
+## Live usage: ask, don't wait to be told (v1.10)
+
+Everything above learns about quota from the stream — the `rate_limit_event`
+records the CLI emits at the top of a turn. That is reactive by construction:
+nothing is observed between turns, and the 5-hour session window arrives with
+a status but no percentage, so the pool could see "warning" but never "how
+close". v1.10 closes both gaps by reading each pooled account's **live usage
+from the provider** on a timer — the same figures the Claude CLI's own
+`/usage` command shows — with the OAuth session the CLI already holds.
+
+```jsonc
+"pool": {
+  "id": "clawd",
+  "accounts": ["claw1", "claw2"],
+  "utilizationThreshold": 0.85,     // hand over here (unchanged)
+  "usagePoll": {                    // v1.10 — on by default
+    "intervalMs": 120000,           // every 2 min (minimum 60000)
+    "warnThreshold": 0.95           // operator alert from here
+  }
+}
+```
+
+What a poll does, per account:
+
+| Provider says | Health file gets | Pool does |
+|---|---|---|
+| 5-hour at 40%, weekly at 20% | `usage:five_hour` / `usage:seven_day`, `allowed` with the number and reset | nothing — home account serves |
+| 5-hour at 86% | the same, `utilization 0.86` | **next launch rotates** to the spare (threshold rule, now with a real number on the session window) |
+| 5-hour at 96% | the same | rotates, **and raises an alert**: account, window, reset time, where launches now go |
+| 5-hour at 100% | `rejected` with the provider's reset | `exhausted` until that reset — for every model, before any turn is refused |
+| every account past 95% | — | **pool-wide alert**: at 100% the ladder or the host's chain takes over; soonest reset named |
+| model-scoped limit (e.g. one family at 97%) | — | alert naming the model; the shim's reactive capture still handles the refusal |
+
+Polled records live under their own keys (`usage:<window>`) so they never race
+the shim's `five_hour` / `seven_day` for the same slot — the shim's next bare
+`allowed` cannot erase the number the poll just wrote, and `explain` shows
+both ("5-hour (live) 40%"). The health rules are the existing ones; the poll
+only improves what they can see.
+
+Boundaries:
+
+- **Read-only on credentials.** The poll reads the access token from the
+  account's `.credentials.json` and sends it to the usage endpoint — nowhere
+  else. It never refreshes, rewrites, or copies a token; an expired one is a
+  skipped tick (the next CLI launch refreshes it, as it always has).
+- **Best-effort.** A failed poll changes nothing: the health file keeps what
+  the shim wrote, selection keeps working from it, and the failure is logged
+  once per transition, not per tick.
+- **Native and `configDir` accounts only.** A token-based account
+  (`oauthTokenFile` / `oauthTokenRef`) runs in the default login dir on a
+  token of its own, so the credentials file there belongs to a different
+  account; those accounts keep stream telemetry only and `explain` says so.
+  macOS native logins kept in the keychain (no `.credentials.json`) are
+  likewise not polled.
+- **Off switch:** `"usagePoll": { "enabled": false }`.
+
+On demand:
+
+```
+multi-clawd usage           # live figures per account + what the next launch would do
+multi-clawd usage --json
+```
+
+**What "seamless" means here.** Rotation is decided per launch, so a turn that
+starts after the poll has seen 85% simply runs on the other account — the user
+sees nothing. A turn already in flight when its account runs out is covered by
+the shim's in-turn retry (v1.9): the refusal is held back, the sibling is
+spawned, the conversation is handed over, and the turn completes there. The
+poll's job is to make that second path rare by moving work off an account
+before it reaches the cliff, and to tell you when every account is nearing one.
+
 ## Direct route: the pool for `anthropic/*` too (v1.9)
 
 OpenClaw reaches Claude two ways. The pool above drives **Claude Code CLI**

package/dist/credential-state.js

@@ -5,6 +5,9 @@ import { parseStoredState, mergeHealthStates, clearCredentialFailure, } from "./
 export function healthStateFile(accountId) {
     return join(homedir(), ".openclaw", "state", "multi-clawd", `${accountId}.json`);
 }
+export function usageStateFile(accountId) {
+    return join(homedir(), ".openclaw", "state", "multi-clawd", `${accountId}.usage.json`);
+}
 export function clearAccountCredentialFailure(accountId) {
     const file = healthStateFile(accountId);
     let state;

package/dist/explain-core.js

@@ -2,6 +2,8 @@ const WINDOW_LABELS = {
     five_hour: "5-hour",
     seven_day: "weekly",
     seven_day_overage_included: "weekly incl. overage",
+    "usage:five_hour": "5-hour (live)",
+    "usage:seven_day": "weekly (live)",
 };
 export function relativeUntil(ms, nowMs) {
     const mins = Math.max(0, Math.round((ms - nowMs) / 60000));

package/dist/index.js

@@ -16,8 +16,9 @@ import { resolvePoolExecutionArgs } from "./tool-cap.js";
 import { RETRY_ROSTER_ENV } from "./retry-plan.js";
 import { SESSION_DIRS_ENV } from "./session-handover.js";
 import { addAlert, alertKeysWithPrefix, clearAlert, pendingAlertText, } from "./alerts.js";
-import { healthStateFile, clearAccountCredentialFailure } from "./credential-state.js";
-export { healthStateFile, clearAccountCredentialFailure };
+import { healthStateFile, usageStateFile, clearAccountCredentialFailure } from "./credential-state.js";
+import { createUsagePollController, effectiveUsagePollInterval, effectiveUsageWarnThreshold, usageAlertPrefix, usagePoolAlertKey, } from "./usage-poll.js";
+export { healthStateFile, usageStateFile, clearAccountCredentialFailure };
 import { accountConfigDir, buildAccountChildEnv, tokenFileModeWarning, validateAccountTokenSources, } from "./account-env.js";
 import { diffCatalogModels, formatNewModelNotice, } from "./model-currency.js";
 import { checkAccountCredential, keychainServiceForConfigDir, createRefProbeTracker, } from "./login-health.js";
@@ -582,18 +583,34 @@ export default definePluginEntry({
                 },
                 logger,
             });
+            startUsagePoll({
+                accounts: accounts.filter((a) => seen.has(a.id.trim())),
+                pool: cfg.pool,
+                registrationMode: api.registrationMode,
+                logger,
+            });
         }
         api.logger.info(`[multi-clawd] registered ${seen.size} backend(s)+provider(s): ${[...seen].join(", ")}`);
     },
 });
 function readHealthState(accountId) {
+    return mergeStoredHealth(readStateFileLenient(healthStateFile(accountId)), readStateFileLenient(usageStateFile(accountId)));
+}
+function readStateFileLenient(file) {
     try {
-        return JSON.parse(readFileSync(healthStateFile(accountId), "utf8"));
+        return JSON.parse(readFileSync(file, "utf8"));
     }
     catch {
         return undefined;
     }
 }
+function mergeStoredHealth(shim, usage) {
+    if (!shim)
+        return usage;
+    if (!usage)
+        return shim;
+    return mergeHealthStates(shim, usage);
+}
 function readStickyEntry(file) {
     try {
         const parsed = JSON.parse(readFileSync(file, "utf8"));
@@ -898,3 +915,131 @@ export function startDirectOrderSync(params) {
 export async function runDirectOrderTickNow() {
     return directController?.tick();
 }
+let usageTimer;
+let usageInitial;
+let usageController;
+export function stopUsagePoll() {
+    if (usageTimer)
+        clearInterval(usageTimer);
+    if (usageInitial)
+        clearTimeout(usageInitial);
+    usageTimer = undefined;
+    usageInitial = undefined;
+    usageController = undefined;
+}
+export function usagePollCredentialsFile(account) {
+    if (account.oauthTokenFile || account.oauthTokenRef) {
+        return { reason: "token-based login — usage is read from its own stream telemetry only" };
+    }
+    if (!account.native && !account.configDir) {
+        return { reason: "no login dir to read credentials from" };
+    }
+    return { file: join(accountConfigDir(account), ".credentials.json") };
+}
+export function startUsagePoll(params) {
+    const { logger, pool } = params;
+    const mode = params.registrationMode;
+    if (mode !== undefined && mode !== "full")
+        return { active: false, members: [] };
+    if (!pool || pool.usagePoll?.enabled === false) {
+        stopUsagePoll();
+        return { active: false, members: [] };
+    }
+    const poolId = pool.id?.trim() || "clawd";
+    const members = [];
+    const skipped = [];
+    for (const id of pool.accounts ?? []) {
+        const account = params.accounts.find((a) => a.id.trim() === id);
+        if (!account)
+            continue;
+        const source = usagePollCredentialsFile(account);
+        if ("file" in source)
+            members.push({ id: account.id.trim(), credentialsFile: source.file });
+        else
+            skipped.push(`${account.id.trim()} (${source.reason})`);
+    }
+    if (members.length === 0) {
+        stopUsagePoll();
+        if (skipped.length > 0)
+            logger.info(`[multi-clawd] usage poll: no pollable account — ${skipped.join("; ")}`);
+        return { active: false, members: [] };
+    }
+    const intervalMs = effectiveUsagePollInterval(pool.usagePoll);
+    const warnThreshold = effectiveUsageWarnThreshold(pool.usagePoll);
+    const healthOptions = {
+        utilizationThreshold: pool.utilizationThreshold,
+        staleAfterMs: pool.staleAfterMs,
+        rotateOnOverage: pool.rotateOnOverage,
+    };
+    const signature = JSON.stringify({ poolId, members, intervalMs, warnThreshold, healthOptions });
+    if (usageController?.signature === signature && usageController.fetchImpl === params.fetchImpl && usageTimer) {
+        return { active: true, members: members.map((m) => m.id) };
+    }
+    stopUsagePoll();
+    const fetchImpl = params.fetchImpl ?? ((url, init) => fetch(url, init));
+    const controller = createUsagePollController({
+        poolId,
+        members,
+        healthOptions,
+        warnThreshold,
+        io: {
+            readFile: (path) => {
+                try {
+                    return readFileSync(path, "utf8");
+                }
+                catch (err) {
+                    if (err.code === "ENOENT")
+                        return undefined;
+                    throw err;
+                }
+            },
+            readHealth: (id) => {
+                let shim;
+                try {
+                    shim = parseStoredState(readFileSync(healthStateFile(id), "utf8"));
+                    if (!shim)
+                        throw new Error("not valid health-state JSON");
+                }
+                catch (err) {
+                    if (err.code !== "ENOENT")
+                        throw err;
+                    shim = undefined;
+                }
+                let usage;
+                try {
+                    usage = parseStoredState(readFileSync(usageStateFile(id), "utf8"));
+                }
+                catch {
+                    usage = undefined;
+                }
+                return mergeStoredHealth(shim, usage);
+            },
+            writeUsage: (id, state) => {
+                const file = usageStateFile(id);
+                mkdirSync(dirname(file), { recursive: true });
+                const tmp = `${file}.tmp-${process.pid}`;
+                writeFileSync(tmp, JSON.stringify(state, null, 2), { mode: 0o600 });
+                renameSync(tmp, file);
+            },
+            fetchImpl,
+            raiseAlert: (alert) => raiseAlert(alert),
+            clearAlert: (key) => {
+                alertState = clearAlert(alertState, key);
+            },
+            alertKeysWithPrefix: (prefix) => alertKeysWithPrefix(alertState, prefix),
+            logger,
+        },
+    });
+    usageController = { signature, fetchImpl: params.fetchImpl, tick: controller.tick };
+    const tick = () => void controller.tick().catch((err) => logger.warn(`[multi-clawd] usage poll tick failed: ${String(err)}`));
+    usageInitial = setTimeout(tick, Math.min(20_000, intervalMs));
+    usageInitial.unref?.();
+    usageTimer = setInterval(tick, intervalMs);
+    usageTimer.unref?.();
+    logger.info(`[multi-clawd] usage poll: reading live usage for ${members.map((m) => m.id).join(", ")} every ${Math.round(intervalMs / 1000)}s — rotate at ${Math.round((pool.utilizationThreshold ?? 0.85) * 100)}%, warn at ${Math.round(warnThreshold * 100)}%${skipped.length > 0 ? `; not polled: ${skipped.join("; ")}` : ""}`);
+    return { active: true, members: members.map((m) => m.id) };
+}
+export async function runUsagePollTickNow() {
+    return usageController?.tick();
+}
+export { usageAlertPrefix, usagePoolAlertKey };

package/dist/usage-poll.js

@@ -0,0 +1,372 @@
+import { classifyAccountHealth } from "./health.js";
+import { mergeHealthStates } from "./shim-core.js";
+export const USAGE_ENDPOINT = "https://api.anthropic.com/api/oauth/usage";
+export const USAGE_WINDOW_PREFIX = "usage:";
+export const DEFAULT_USAGE_POLL_INTERVAL_MS = 2 * 60 * 1000;
+export const MIN_USAGE_POLL_INTERVAL_MS = 60 * 1000;
+export const MAX_USAGE_POLL_INTERVAL_MS = 30 * 60 * 1000;
+export const DEFAULT_USAGE_WARN_THRESHOLD = 0.95;
+export const USAGE_FETCH_TIMEOUT_MS = 15_000;
+export function effectiveUsagePollInterval(cfg) {
+    const requested = cfg?.intervalMs ?? DEFAULT_USAGE_POLL_INTERVAL_MS;
+    if (!Number.isFinite(requested))
+        return DEFAULT_USAGE_POLL_INTERVAL_MS;
+    return Math.min(MAX_USAGE_POLL_INTERVAL_MS, Math.max(MIN_USAGE_POLL_INTERVAL_MS, requested));
+}
+export function effectiveUsageWarnThreshold(cfg) {
+    const t = cfg?.warnThreshold;
+    return typeof t === "number" && t > 0 && t <= 1 ? t : DEFAULT_USAGE_WARN_THRESHOLD;
+}
+function parseResetsAt(value) {
+    if (typeof value !== "string")
+        return undefined;
+    const ms = Date.parse(value);
+    return Number.isFinite(ms) ? Math.floor(ms / 1000) : undefined;
+}
+function parsePercent(value) {
+    if (typeof value !== "number" || !Number.isFinite(value) || value < 0)
+        return undefined;
+    return value / 100;
+}
+const WINDOW_KEY_RE = /^(five_hour|seven_day(_[a-z0-9_]+)?)$/;
+export function parseUsageResponse(body) {
+    if (typeof body !== "object" || body === null)
+        return undefined;
+    const b = body;
+    const windows = {};
+    for (const [key, value] of Object.entries(b)) {
+        if (!WINDOW_KEY_RE.test(key))
+            continue;
+        if (typeof value !== "object" || value === null)
+            continue;
+        const w = value;
+        const utilization = parsePercent(w.utilization);
+        if (utilization === undefined)
+            continue;
+        windows[key] = { utilization, resetsAt: parseResetsAt(w.resets_at) };
+    }
+    const scoped = [];
+    if (Array.isArray(b.limits)) {
+        for (const entry of b.limits) {
+            if (typeof entry !== "object" || entry === null)
+                continue;
+            const e = entry;
+            const scope = e.scope;
+            const model = scope?.model;
+            const label = model?.display_name;
+            const utilization = parsePercent(e.percent);
+            if (typeof label !== "string" || !label.trim() || utilization === undefined)
+                continue;
+            scoped.push({ label: label.trim(), utilization, resetsAt: parseResetsAt(e.resets_at) });
+        }
+    }
+    if (Object.keys(windows).length === 0)
+        return undefined;
+    return { windows, scoped };
+}
+export const ACCOUNT_USAGE_WINDOWS = ["five_hour", "seven_day"];
+export function usageHealthWindows(snapshot, nowMs) {
+    const out = {};
+    for (const [key, w] of Object.entries(snapshot.windows)) {
+        if (!ACCOUNT_USAGE_WINDOWS.includes(key))
+            continue;
+        out[`${USAGE_WINDOW_PREFIX}${key}`] = {
+            status: w.utilization >= 1 ? "rejected" : "allowed",
+            utilization: w.utilization,
+            resetsAt: w.resetsAt,
+            seenAt: nowMs,
+        };
+    }
+    return out;
+}
+export function readOAuthAccessToken(raw, nowMs) {
+    if (raw === undefined)
+        return { skip: "no .credentials.json (not signed in, or credentials held elsewhere)" };
+    let parsed;
+    try {
+        parsed = JSON.parse(raw);
+    }
+    catch {
+        return { skip: ".credentials.json is not valid JSON" };
+    }
+    const oauth = parsed?.claudeAiOauth;
+    if (typeof oauth !== "object" || oauth === null)
+        return { skip: ".credentials.json has no OAuth session" };
+    const o = oauth;
+    if (typeof o.accessToken !== "string" || o.accessToken.trim() === "") {
+        return { skip: ".credentials.json has no access token" };
+    }
+    if (typeof o.expiresAt === "number" && o.expiresAt <= nowMs) {
+        return { skip: "access token expired — the next CLI launch refreshes it" };
+    }
+    return { token: o.accessToken };
+}
+export async function fetchUsage(token, fetchImpl, timeoutMs = USAGE_FETCH_TIMEOUT_MS) {
+    const controller = typeof AbortController === "function" ? new AbortController() : undefined;
+    const timer = controller ? setTimeout(() => controller.abort(), timeoutMs) : undefined;
+    try {
+        const res = await fetchImpl(USAGE_ENDPOINT, {
+            headers: {
+                Authorization: `Bearer ${token}`,
+                "anthropic-beta": "oauth-2025-04-20",
+                accept: "application/json",
+            },
+            signal: controller?.signal,
+        });
+        if (res.status === 401 || res.status === 403) {
+            return { ok: false, kind: "auth", reason: `usage endpoint refused the login (HTTP ${res.status})` };
+        }
+        if (res.status === 429)
+            return { ok: false, kind: "throttled", reason: "usage endpoint rate-limited the poll" };
+        if (res.status < 200 || res.status >= 300) {
+            return { ok: false, kind: "transient", reason: `usage endpoint returned HTTP ${res.status}` };
+        }
+        let body;
+        try {
+            body = await res.json();
+        }
+        catch {
+            return { ok: false, kind: "parse", reason: "usage endpoint returned a non-JSON body" };
+        }
+        const snapshot = parseUsageResponse(body);
+        if (!snapshot)
+            return { ok: false, kind: "parse", reason: "usage endpoint body had no readable window" };
+        return { ok: true, snapshot };
+    }
+    catch (err) {
+        const name = err instanceof Error ? err.name : typeof err;
+        return { ok: false, kind: "transient", reason: `usage request failed (${name})` };
+    }
+    finally {
+        if (timer)
+            clearTimeout(timer);
+    }
+}
+export function usageWindowLabel(window) {
+    const bare = window.startsWith(USAGE_WINDOW_PREFIX) ? window.slice(USAGE_WINDOW_PREFIX.length) : window;
+    if (bare === "five_hour")
+        return "5-hour";
+    if (bare === "seven_day")
+        return "weekly";
+    if (bare.startsWith("seven_day_"))
+        return `weekly ${bare.slice("seven_day_".length).replace(/_/g, " ")}`;
+    return bare;
+}
+function relative(resetsAt, nowMs) {
+    if (resetsAt === undefined)
+        return "";
+    const mins = Math.max(0, Math.round((resetsAt * 1000 - nowMs) / 60000));
+    if (mins < 90)
+        return ` (resets in ~${mins}m)`;
+    const hours = Math.round(mins / 60);
+    if (hours < 36)
+        return ` (resets in ~${hours}h)`;
+    return ` (resets in ~${Math.round(hours / 24)}d)`;
+}
+export function usageAlertPrefix(poolId) {
+    return `usage:${poolId}:`;
+}
+export function usagePoolAlertKey(poolId) {
+    return `usage-pool:${poolId}`;
+}
+export function decideUsageAlerts(params) {
+    const { poolId, accounts, warnThreshold, nowMs } = params;
+    const raise = [];
+    const keep = new Set();
+    const prefix = usageAlertPrefix(poolId);
+    const answered = accounts.filter((a) => a.snapshot !== undefined);
+    const hot = new Set();
+    let soonestReset;
+    for (const account of answered) {
+        const snapshot = account.snapshot;
+        const others = accounts.filter((a) => a.id !== account.id);
+        const sibling = others.find((a) => a.verdict === "ok" || a.verdict === "no_data" || a.verdict === undefined);
+        for (const [window, w] of Object.entries(snapshot.windows)) {
+            if (w.utilization < warnThreshold)
+                continue;
+            if (ACCOUNT_USAGE_WINDOWS.includes(window)) {
+                hot.add(account.id);
+                if (w.resetsAt !== undefined && (soonestReset === undefined || w.resetsAt < soonestReset)) {
+                    soonestReset = w.resetsAt;
+                }
+            }
+            const key = `${prefix}${account.id}:${window}`;
+            keep.add(key);
+            const pct = Math.round(w.utilization * 100);
+            const where = !ACCOUNT_USAGE_WINDOWS.includes(window)
+                ? w.utilization >= 1
+                    ? "that model is refused on this account until reset"
+                    : "that model will be refused on this account at 100%"
+                : w.utilization >= 1
+                    ? "this account is exhausted on that window"
+                    : others.length === 0
+                        ? "no sibling account to hand over to"
+                        : sibling
+                            ? `new launches route to ${sibling.id}`
+                            : "no healthy sibling to hand over to";
+            raise.push({
+                key,
+                severity: "error",
+                text: `pool ${poolId}: account "${account.id}" ${usageWindowLabel(window)} usage at ${pct}%${relative(w.resetsAt, nowMs)} — ${where}`,
+            });
+        }
+        for (const limit of snapshot.scoped) {
+            if (limit.utilization < warnThreshold)
+                continue;
+            const key = `${prefix}${account.id}:model:${limit.label.toLowerCase().replace(/\s+/g, "-")}`;
+            keep.add(key);
+            const pct = Math.round(limit.utilization * 100);
+            raise.push({
+                key,
+                severity: "error",
+                text: `pool ${poolId}: account "${account.id}" ${limit.label} model limit at ${pct}%${relative(limit.resetsAt, nowMs)} — ${limit.utilization >= 1 ? "that model is refused on this account until reset" : "that model will be refused on this account at 100%"}`,
+            });
+        }
+    }
+    if (accounts.length > 0 && hot.size === accounts.length) {
+        const key = usagePoolAlertKey(poolId);
+        keep.add(key);
+        const pct = Math.round(warnThreshold * 100);
+        raise.push({
+            key,
+            severity: "error",
+            text: `pool ${poolId}: EVERY account is above ${pct}% on at least one usage window — ` +
+                `at 100% turns degrade down the ladder or fall through to the next provider` +
+                `${soonestReset !== undefined ? `; soonest reset${relative(soonestReset, nowMs)}` : ""}`,
+        });
+    }
+    return { raise, keep };
+}
+export function createUsagePollController(params) {
+    const { poolId, members, healthOptions, warnThreshold, io } = params;
+    const now = io.now ?? (() => Date.now());
+    const lastProblem = new Map();
+    const lastVerdict = new Map();
+    const noteProblem = (id, problem) => {
+        const previous = lastProblem.get(id);
+        if (problem === previous)
+            return;
+        if (problem)
+            io.logger.warn(`[multi-clawd] usage poll: account "${id}" — ${problem}`);
+        else if (previous)
+            io.logger.info(`[multi-clawd] usage poll: account "${id}" — reading usage again`);
+        if (problem)
+            lastProblem.set(id, problem);
+        else
+            lastProblem.delete(id);
+    };
+    async function tick() {
+        const at = now();
+        const reports = [];
+        for (const member of members) {
+            let raw;
+            try {
+                raw = io.readFile(member.credentialsFile);
+            }
+            catch (err) {
+                raw = undefined;
+                noteProblem(member.id, `credentials unreadable (${err instanceof Error ? err.name : String(err)})`);
+                reports.push({ id: member.id, skipped: "credentials unreadable", verdict: verdictFor(member.id) });
+                continue;
+            }
+            const token = readOAuthAccessToken(raw, at);
+            if ("skip" in token) {
+                noteProblem(member.id, token.skip);
+                reports.push({ id: member.id, skipped: token.skip, verdict: verdictFor(member.id) });
+                continue;
+            }
+            const result = await fetchUsage(token.token, io.fetchImpl);
+            if (!result.ok) {
+                noteProblem(member.id, result.reason);
+                reports.push({ id: member.id, failure: { kind: result.kind, reason: result.reason }, verdict: verdictFor(member.id) });
+                continue;
+            }
+            const live = {
+                accountId: member.id,
+                updatedAt: at,
+                windows: usageHealthWindows(result.snapshot, at),
+            };
+            try {
+                io.writeUsage(member.id, live);
+            }
+            catch (err) {
+                io.logger.warn(`[multi-clawd] usage poll: usage write failed for "${member.id}": ${String(err)}`);
+            }
+            let disk;
+            try {
+                disk = io.readHealth(member.id) ?? { accountId: member.id, windows: {} };
+            }
+            catch (err) {
+                noteProblem(member.id, `health file unreadable (${err instanceof Error ? err.message : String(err)})`);
+                reports.push({ id: member.id, snapshot: result.snapshot, skipped: "health file unreadable", verdict: "no_data" });
+                continue;
+            }
+            noteProblem(member.id, undefined);
+            const merged = mergeHealthStates(disk, live, at);
+            const verdict = classifyAccountHealth(merged, healthOptions, at).verdict;
+            const previous = lastVerdict.get(member.id);
+            if (previous !== undefined && previous !== verdict) {
+                io.logger.info(`[multi-clawd] usage poll: account "${member.id}" ${previous} → ${verdict} (${describeSnapshot(result.snapshot)})`);
+            }
+            else if (previous === undefined) {
+                io.logger.info(`[multi-clawd] usage poll: account "${member.id}" ${describeSnapshot(result.snapshot)} → ${verdict}`);
+            }
+            lastVerdict.set(member.id, verdict);
+            reports.push({ id: member.id, snapshot: result.snapshot, verdict });
+        }
+        const known = reports.filter((r) => r.snapshot !== undefined && r.skipped === undefined);
+        const unknown = reports.filter((r) => !known.includes(r));
+        const decision = decideUsageAlerts({
+            poolId,
+            accounts: reports.map((r) => ({ id: r.id, snapshot: known.includes(r) ? r.snapshot : undefined, verdict: r.verdict })),
+            warnThreshold,
+            nowMs: at,
+        });
+        for (const r of unknown) {
+            for (const key of io.alertKeysWithPrefix(`${usageAlertPrefix(poolId)}${r.id}:`))
+                decision.keep.add(key);
+        }
+        if (unknown.length > 0) {
+            for (const key of io.alertKeysWithPrefix(usagePoolAlertKey(poolId)))
+                decision.keep.add(key);
+        }
+        const raised = [];
+        for (const alert of decision.raise) {
+            io.raiseAlert(alert);
+            raised.push(alert.key);
+        }
+        const cleared = [];
+        const family = [...io.alertKeysWithPrefix(usageAlertPrefix(poolId)), ...io.alertKeysWithPrefix(usagePoolAlertKey(poolId))];
+        for (const key of family) {
+            if (decision.keep.has(key))
+                continue;
+            io.clearAlert(key);
+            cleared.push(key);
+        }
+        return { at, accounts: reports, raised, cleared };
+    }
+    const lastReadProblem = new Map();
+    function verdictFor(id) {
+        let state;
+        try {
+            state = io.readHealth(id);
+        }
+        catch (err) {
+            const problem = `health file unreadable (${err instanceof Error ? err.message : String(err)})`;
+            if (lastReadProblem.get(id) !== problem) {
+                io.logger.warn(`[multi-clawd] usage poll: account "${id}" — ${problem}`);
+                lastReadProblem.set(id, problem);
+            }
+            return "no_data";
+        }
+        lastReadProblem.delete(id);
+        return classifyAccountHealth(state, healthOptions, now()).verdict;
+    }
+    return { tick };
+}
+export function describeSnapshot(snapshot) {
+    const parts = Object.entries(snapshot.windows).map(([k, w]) => `${usageWindowLabel(k)} ${Math.round(w.utilization * 100)}%`);
+    for (const s of snapshot.scoped)
+        parts.push(`${s.label} ${Math.round(s.utilization * 100)}%`);
+    return parts.join(" · ");
+}

package/openclaw.plugin.json

@@ -1,7 +1,7 @@
 {
   "id": "multi-clawd",
   "name": "multi-clawd",
-  "version": "1.9.8",
+  "version": "1.10.1",
   "description": "Register additional Claude Code logins (Max/Pro accounts) as first-class OpenClaw CLI backends for cross-account failover, keeping the full skills/MCP harness on every account.",
   "cliBackends": [
     "claw1",
@@ -59,6 +59,25 @@
             "type": "number",
             "description": "After rotating away from home, stay on the rotated-to account at least this long before returning home (anti-flap hysteresis; health always overrides). Default 600000 (10min)."
           },
+          "usagePoll": {
+            "type": "object",
+            "additionalProperties": false,
+            "description": "Live usage polling (v1.10): read each pooled account's current usage from the provider on a timer (the same figures the Claude CLI's /usage shows) and fold them into the pool's health state, so near-limit rotation fires on the 5-hour window with a real percentage behind it and a window at 100% is exhausted before a turn is refused. Raises an operator alert (via the heartbeat) when any window passes warnThreshold. Read-only on credentials: uses the login the CLI already holds, never refreshes or copies it. Applies to native and configDir accounts; token-based accounts keep stream telemetry only.",
+            "properties": {
+              "enabled": {
+                "type": "boolean",
+                "description": "Default true. Set false to turn polling off entirely."
+              },
+              "intervalMs": {
+                "type": "number",
+                "description": "Poll interval. Default 120000 (2min); minimum 60000."
+              },
+              "warnThreshold": {
+                "type": "number",
+                "description": "Raise an operator alert when any usage window reaches this fraction. Default 0.95."
+              }
+            }
+          },
           "degrade": {
             "type": "object",
             "additionalProperties": false,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@drakon-systems/multi-clawd",
-  "version": "1.9.8",
+  "version": "1.10.1",
   "description": "Multi-account Claude Code failover for OpenClaw — register additional Claude (Max/Pro) logins as first-class CLI backends and keep the full skills/MCP harness across every account. Also imports those accounts' setup tokens into Hermes Agent's Anthropic credential pool.",
   "type": "module",
   "license": "MIT",

package/scripts/cli.mjs

@@ -39,6 +39,7 @@ ${BOLD}🦞 multi-clawd${RESET} — multi-account Claude failover for OpenClaw
   ${BOLD}explain${RESET}   your setup in plain English — accounts, pool, fallback chain
   ${BOLD}chain${RESET}     audit your model routing — what actually serves each turn
   ${BOLD}direct${RESET}    the direct anthropic/* route — status, or \`direct sync\` to store profiles
+  ${BOLD}usage${RESET}     live usage per account from the provider — what is left, and what the pool will do
   ${BOLD}update${RESET}    update the plugin to the latest version
   ${BOLD}doctor${RESET}    health check (add --probe for a live turn)
   ${BOLD}hermes${RESET}    sync or diagnose Hermes Agent's Anthropic credential pool
@@ -711,6 +712,115 @@ async function explain() {
   console.log(`\n${DIM}(health checks: multi-clawd doctor · change things: multi-clawd setup)${RESET}`);
 }
 
+/**
+ * `usage` — ask the provider for each account's live usage (the figures the
+ * Claude CLI's own /usage shows), print them next to the pool's verdict, and
+ * say which account the next pooled launch would run on. Read-only: uses the
+ * login the CLI already holds and never refreshes or copies it. `--json` for
+ * machines.
+ */
+async function usageCommand(args) {
+  const { readFileSync: rf } = await import("node:fs");
+  const { homedir } = await import("node:os");
+  let up, health, shim, env;
+  try {
+    up = await import(resolve(__dirname, "..", "dist", "usage-poll.js"));
+    health = await import(resolve(__dirname, "..", "dist", "health.js"));
+    shim = await import(resolve(__dirname, "..", "dist", "shim-core.js"));
+    env = await import(resolve(__dirname, "..", "dist", "account-env.js"));
+  } catch (err) {
+    console.error(distFailure("usage", "usage-poll.js", err));
+    process.exit(1);
+  }
+  const config = readOpenclawConfigSync();
+  if (!config) {
+    console.error("usage: could not read ~/.openclaw/openclaw.json");
+    process.exit(1);
+  }
+  const pc = config?.plugins?.entries?.["multi-clawd"]?.config ?? {};
+  const accounts = Array.isArray(pc.accounts) ? pc.accounts : [];
+  const pool = pc.pool ? { ...pc.pool, id: pc.pool.id?.trim() || "clawd", accounts: pc.pool.accounts ?? [] } : undefined;
+  const healthOptions = {
+    utilizationThreshold: pool?.utilizationThreshold,
+    staleAfterMs: pool?.staleAfterMs,
+    rotateOnOverage: pool?.rotateOnOverage,
+  };
+  const warnThreshold = up.effectiveUsageWarnThreshold(pool?.usagePoll);
+  const rotateAt = pool?.utilizationThreshold ?? 0.85;
+  const stateDir = join(homedir(), ".openclaw", "state", "multi-clawd");
+  const now = Date.now();
+  const rows = [];
+  for (const a of accounts) {
+    const row = { id: a.id, label: a.label };
+    if (a.oauthTokenFile || a.oauthTokenRef) {
+      row.skipped = "token-based login — not polled (stream telemetry only)";
+    } else if (!a.native && !a.configDir) {
+      row.skipped = "no login dir";
+    } else {
+      let raw;
+      try {
+        raw = rf(join(env.accountConfigDir(a), ".credentials.json"), "utf8");
+      } catch {
+        raw = undefined;
+      }
+      const token = up.readOAuthAccessToken(raw, now);
+      if ("skip" in token) row.skipped = token.skip;
+      else {
+        const r = await up.fetchUsage(token.token, (url, init) => fetch(url, init));
+        if (r.ok) row.snapshot = r.snapshot;
+        else row.failure = r.reason;
+      }
+    }
+    let state;
+    try {
+      state = shim.parseStoredState(rf(join(stateDir, `${a.id}.json`), "utf8"));
+    } catch {
+      /* no telemetry yet */
+    }
+    const h = health.classifyAccountHealth(state, healthOptions, now);
+    row.verdict = h.verdict;
+    row.reason = h.reason;
+    rows.push(row);
+  }
+  let next;
+  if (pool) {
+    const verdicts = pool.accounts.map((id) => ({ id, verdict: rows.find((r) => r.id === id)?.verdict ?? "no_data" }));
+    next = health.choosePoolAccount(verdicts);
+  }
+  if (args.includes("--json")) {
+    console.log(JSON.stringify({ at: new Date(now).toISOString(), rotateAt, warnThreshold, pool: pool?.id, nextLaunch: next, accounts: rows }, null, 2));
+    return;
+  }
+  const pct = (u) => `${Math.round(u * 100)}%`;
+  const until = (resetsAt) => {
+    if (resetsAt === undefined) return "";
+    const mins = Math.max(0, Math.round((resetsAt * 1000 - now) / 60000));
+    return mins < 90 ? ` (resets in ~${mins}m)` : mins < 36 * 60 ? ` (resets in ~${Math.round(mins / 60)}h)` : ` (resets in ~${Math.round(mins / 1440)}d)`;
+  };
+  const mark = (u) => (u >= 1 ? "⛔" : u >= warnThreshold ? "🔴" : u >= rotateAt ? "🟠" : "🟢");
+  console.log(`\n${BOLD}🦞 multi-clawd — live usage${RESET}  ${DIM}(rotate at ${pct(rotateAt)}, warn at ${pct(warnThreshold)})${RESET}\n`);
+  for (const r of rows) {
+    console.log(`  ${BOLD}${r.id}${RESET}${r.label ? ` ${DIM}— ${r.label}${RESET}` : ""}   pool verdict: ${r.verdict}${r.reason ? ` ${DIM}(${r.reason})${RESET}` : ""}`);
+    if (r.snapshot) {
+      for (const [w, u] of Object.entries(r.snapshot.windows)) {
+        console.log(`     ${mark(u.utilization)} ${up.usageWindowLabel(w).padEnd(12)} ${pct(u.utilization).padStart(4)}${until(u.resetsAt)}`);
+      }
+      for (const s of r.snapshot.scoped) {
+        console.log(`     ${mark(s.utilization)} ${`${s.label} model`.padEnd(12)} ${pct(s.utilization).padStart(4)}${until(s.resetsAt)}`);
+      }
+    } else if (r.failure) {
+      console.log(`     ⚠️  ${r.failure}`);
+    } else {
+      console.log(`     ${DIM}not polled: ${r.skipped}${RESET}`);
+    }
+    console.log("");
+  }
+  if (pool) {
+    console.log(next ? `  next pooled launch (${pool.id}/…) runs on ${BOLD}${next}${RESET}` : `  ${BOLD}every pooled account is exhausted${RESET} — turns degrade or fall through the chain`);
+  }
+  console.log(`\n${DIM}(the gateway polls this every ${Math.round(up.effectiveUsagePollInterval(pool?.usagePoll) / 1000)}s and alerts via the heartbeat; see "Live usage" in the README)${RESET}`);
+}
+
 /**
  * `login <account>` — launch the RIGHT Claude login flow for a configured
  * account: correct config-dir environment, dir created if missing, verified
@@ -831,6 +941,9 @@ switch (cmd) {
   case "direct":
     await direct(rest);
     break;
+  case "usage":
+    await usageCommand(rest);
+    break;
   case "doctor":
     runSibling("doctor.mjs", rest);
     break;