@trillioncore/cli 1.0.0-next.3 → 1.0.0-next.5

package/README.md

@@ -2,7 +2,7 @@
 
 The `tc` CLI signs you into your own Trillioncore organization and reads the integration accounts you authorize. No repository checkout, provider secrets, `.env` file, or manually entered organization ID is required.
 
-**Requirements:** Node.js 22+, a browser that can reach the CLI computer's loopback listener, and a Trillioncore deployment with CLI login enabled. This is a fresh v1 preview, not v0 command parity; v0 configuration is not imported.
+**Requirements:** Node.js 22+, a browser on any device, and a Trillioncore deployment with CLI device login enabled. This is a fresh v1 preview, not v0 command parity; v0 configuration is not imported.
 
 ## Get started
 
@@ -18,6 +18,8 @@ npm exec --yes --package=@trillioncore/cli@next -- tc sql '<PostgreSQL account I
 npm exec --yes --package=@trillioncore/cli@next -- tc logout
 ```
 
+`tc login` registers a five-minute request and prints a browser URL plus a separate eight-character code. Open the URL on any device and enter the code shown in your terminal. Keep the code private and approve only a login you started. The CLI polls without a callback listener, tunnel, or copied token. After approval it verifies your identity, saves the private session, and acknowledges delivery.
+
 Browser sign-in lets you choose an organization and explicitly select its enabled accounts. An organization administrator must approve access. For a brand-new account, finish organization setup and connect your integrations in the app, then rerun `tc login` if prompted. No provider reconnection is required just to authorize the CLI.
 
 Prefer `npm exec` during evaluation. `npm install -g @trillioncore/cli@next` replaces your local `tc` executable, even though publishing `next` does not change the npm `latest` tag.
@@ -53,6 +55,8 @@ The catalog describes stored content kinds or approved **active synchronized** P
 
 For native content, `recordSchemas` describes the allowed scalar fields in `get`'s `attributes`, grouped by recognized observed record kind. These field names/types come from the same definitions used to validate stored records. Fields may be absent on individual records; a schema does not prove populated values, complete history, relationship IDs, or SQL support. Empty connections return no observed schemas; unknown kinds remain listed without invented definitions. Table resources retain their separate `relation.columns` metadata.
 
+`lifecycle` reports the persisted `activeRun` (queued/running) and `lastFinishedRun` (completed/failed), independently of the last successful sync timestamp. Null means no matching run was recorded. The stored `startedAt` can be enqueue time; a running state does not prove worker health. Progress counts are processed pages/records, not percentages or newly created records. A completed run does not prove exhaustive source coverage. Native `page-upserts` may remain visible even if a later page fails; `active-revision` describes snapshot publication, not whether production execution is configured. `sourceRemovals: not-propagated` means native source removal events do not remove stored records; `revision-scoped` describes active-snapshot visibility, not a guarantee of continuous deletion tracking. Lifecycle/counter and freshness changes alone do not invalidate resource cursors; schema/revision changes still require restarting discovery. No cursors, raw failure messages or credentials appear in lifecycle metadata.
+
 Respect `operations[].available` and `warnings`. Production synced SQL is currently not wired, so its schemas can be discovered but SQL is explicitly unavailable with `synced_sql_not_configured`. Live PostgreSQL resources are outside this catalog's coverage. No semantic search or actions are advertised. Native connectors retain search/get capabilities when empty; `recordKinds` lists only observed kinds (capped at 100 with an explicit warning). Synced manifests use existing limits of 25 tables/1,000 columns. A successful sync timestamp does not establish complete history or current provider state. Business definitions, currency and attribution are not inferred.
 
 All discovery uses the same organization, live token scope, connection permission and enabled-state checks as existing agent reads. Multiple internal adapters may eventually fulfill a connection's capabilities; this catalog does not introduce a separate user-visible integration for each adapter.
@@ -64,7 +68,9 @@ Sessions refresh automatically. File-based credentials live in `~/.trillioncore/
 One profile is active at a time. A new successful login replaces it and attempts to revoke the prior connection. Change account scopes or revoke access from the app's **Tokens** page. Membership changes and server revocation are enforced on subsequent requests.
 
 - Default API: `https://api.trillioncore.com`. A saved session remains pinned to its issuing server. Custom origins must use HTTPS, except literal loopback development endpoints.
-- `--no-browser` prints the sign-in URL; it is **not** a remote device-code flow. SSH users need appropriate loopback forwarding or a browser on the same computer. Do not share the sign-in URL.
+- `--no-browser` prints the device sign-in URL and separate code without launching a browser. Use it over SSH or on a headless machine. Default login uses the same remote-capable flow; there is no loopback fallback flag.
+- Pending and transient polling failures retry within a fixed five-minute deadline. Rate-limit retries respect `Retry-After`. Ctrl-C or timeout makes one best-effort cancellation attempt; unreachable requests expire on the server. If registration itself is lost, rerun login and let the unknown request expire.
+- A lost completed poll can be retrieved again before expiry. Acknowledgement occurs only after private persistence. If acknowledgement fails, the saved session remains active and the temporary delivery expires automatically.
 - A disabled/not-yet-deployed CLI login returns a clear deployment-unavailable error. Signing in repeatedly cannot enable the server.
 - Advanced/manual use: `TRILLIONCORE_TOKEN` and `TRILLIONCORE_ORG_ID` must be supplied together and override the saved session. Remove both to use browser login. API overrides prefer `TRILLIONCORE_CLI_API_URL`, then `TRILLIONCORE_API_URL`; a mismatch with a saved session is refused, not silently redirected. Application API configuration is never rewritten.
 - `TRILLIONCORE_CONFIG_DIR` selects a private CLI-only profile directory (also useful for isolated testing). Legacy v0 configuration is not migrated.

package/dist/index.js

@@ -3051,8 +3051,8 @@ import { fileURLToPath } from "node:url";
 import { realpathSync } from "node:fs";
 
 // src/auth.ts
-import { createServer } from "node:http";
-import { createHash, randomBytes as randomBytes2 } from "node:crypto";
+import { setTimeout as delay } from "node:timers/promises";
+import { createHash, randomBytes as randomBytes2, randomInt } from "node:crypto";
 import { spawn } from "node:child_process";
 
 // src/session-store.ts
@@ -3243,6 +3243,14 @@ function sessionStore(environment = process.env) {
 
 // src/auth.ts
 var DEFAULT_API = "https://api.trillioncore.com";
+var deviceSecret = /^[A-Za-z0-9_-]{43}$/;
+var RetryableAuthError = class extends Error {
+  constructor(message, retryAfterMs = 0) {
+    super(message);
+    this.retryAfterMs = retryAfterMs;
+  }
+  retryAfterMs;
+};
 async function jsonRequest(issuer, path, init = {}) {
   let response;
   try {
@@ -3253,14 +3261,27 @@ async function jsonRequest(issuer, path, init = {}) {
     });
   } catch {
     if (init.signal?.aborted) throw init.signal.reason;
-    throw new Error(
+    throw new RetryableAuthError(
       "Cannot reach the CLI authentication service. Check connectivity and retry."
     );
   }
-  if (!response.ok)
+  if (response.status === 429 || response.status >= 500) {
+    const header = response.headers.get("retry-after")?.trim();
+    const seconds = header && /^\d+$/.test(header) ? Number(header) : NaN;
+    const until = header ? Date.parse(header) : NaN;
+    const retryAfterMs = Number.isFinite(seconds) ? seconds * 1e3 : Number.isFinite(until) ? Math.max(0, until - Date.now()) : 0;
+    await response.body?.cancel();
+    throw new RetryableAuthError(
+      "CLI authentication is temporarily unavailable.",
+      retryAfterMs
+    );
+  }
+  if (!response.ok) {
+    await response.body?.cancel();
     throw new Error(
       response.status === 404 ? "CLI login is not enabled on this deployment yet." : "CLI authentication was refused or expired. Run tc login again."
     );
+  }
   if (!response.body)
     throw new Error(
       "The CLI authentication service returned an invalid response."
@@ -3270,7 +3291,11 @@ async function jsonRequest(issuer, path, init = {}) {
   const parts = [];
   try {
     while (true) {
-      const part = await reader.read();
+      const part = await reader.read().catch(() => {
+        throw new RetryableAuthError(
+          "CLI authentication response was interrupted."
+        );
+      });
       if (part.done) break;
       size += part.value.length;
       if (size > 32768) {
@@ -3283,7 +3308,9 @@ async function jsonRequest(issuer, path, init = {}) {
     if (!value || typeof value !== "object" || Array.isArray(value))
       throw new Error();
     return value;
-  } catch {
+  } catch (error) {
+    if (init.signal?.aborted) throw init.signal.reason;
+    if (error instanceof RetryableAuthError) throw error;
     throw new Error(
       "The CLI authentication service returned an invalid response."
     );
@@ -3298,13 +3325,20 @@ function form(values) {
     body: new URLSearchParams(values)
   };
 }
-function tokens(value) {
+function tokens(value, device = false) {
   if (typeof value.access_token !== "string" || !/^tc_cli_[A-Za-z0-9_-]{43}$/.test(value.access_token) || typeof value.refresh_token !== "string" || !/^tc_refresh_[A-Za-z0-9_-]{43}$/.test(value.refresh_token) || value.token_type !== "Bearer" || typeof value.expires_in !== "number" || !Number.isFinite(value.expires_in) || value.expires_in <= 0 || value.expires_in > 86400)
     throw new Error("The service did not return a valid CLI session.");
+  const now = Date.now();
+  let expiresAt = now + value.expires_in * 1e3;
+  if (device) {
+    if (typeof value.access_expires_at !== "number" || !Number.isFinite(value.access_expires_at) || value.access_expires_at <= now || value.access_expires_at > now + 86400 * 1e3)
+      throw new Error("The service did not return a valid CLI session expiry.");
+    expiresAt = value.access_expires_at;
+  }
   return {
     accessToken: value.access_token,
     refreshToken: value.refresh_token,
-    expiresAt: Date.now() + value.expires_in * 1e3
+    expiresAt
   };
 }
 async function identity(issuer, accessToken, signal) {
@@ -3346,125 +3380,147 @@ async function revoke(session) {
       "Could not revoke this connection. Retry logout or revoke it on the Tokens page."
     );
 }
+function loginOrigin(value) {
+  const origin = apiOrigin(value);
+  if (origin.startsWith("http:") && !/^http:\/\/(?:localhost|127\.0\.0\.1|\[::1\])(?::\d+)?\/?$/i.test(value))
+    throw new Error(
+      "API URL must be an HTTPS origin or literal loopback development origin."
+    );
+  return origin;
+}
+function verificationUrl(value, requestId) {
+  try {
+    if (typeof value !== "string") throw new Error();
+    const url = new URL(value);
+    loginOrigin(value.slice(0, value.indexOf("/connect/cli")));
+    if (url.username || url.password || url.hash || url.pathname !== "/connect/cli" || url.search !== `?request=${requestId}`)
+      throw new Error();
+    return url.toString();
+  } catch {
+    throw new Error(
+      "The service returned an invalid browser verification URL."
+    );
+  }
+}
 async function login(options = {}, environment = process.env) {
-  const issuer = apiOrigin(
+  const issuer = loginOrigin(
     options.apiUrl ?? environment.TRILLIONCORE_CLI_API_URL ?? environment.TRILLIONCORE_API_URL ?? DEFAULT_API
   );
   const abort = new AbortController();
-  const state = randomBytes2(32).toString("base64url");
   const verifier = randomBytes2(32).toString("base64url");
-  let receive;
-  let rejectCallback;
-  const callback = new Promise((resolve, reject) => {
-    receive = resolve;
-    rejectCallback = reject;
-  });
-  void callback.catch(() => {
-  });
-  let expectedHost = "";
-  const server = createServer((request2, response) => {
-    response.setHeader("Cache-Control", "no-store");
-    response.setHeader("Referrer-Policy", "no-referrer");
-    let url;
+  const alphabet = "23456789ABCDEFGHJKLMNPQRSTUVWXYZ";
+  const userCode = Array.from(
+    { length: 8 },
+    () => alphabet[randomInt(alphabet.length)]
+  ).join("");
+  const started = Date.now();
+  const timeout = options.timeoutMs ?? 3e5;
+  if (!Number.isFinite(timeout) || timeout <= 0)
+    throw new Error("Invalid sign-in timeout.");
+  let deadline = started + Math.min(timeout, 3e5);
+  let requestId;
+  let persisted = false;
+  const stop = (message) => abort.abort(new Error(message));
+  const cancel = () => {
+    if (!persisted) stop("Sign-in cancelled.");
+  };
+  process.on("SIGINT", cancel);
+  let timer = setTimeout(
+    () => stop("Sign-in timed out. Run tc login again."),
+    deadline - Date.now()
+  );
+  const check = () => {
+    if (Date.now() >= deadline) stop("Sign-in timed out. Run tc login again.");
+    abort.signal.throwIfAborted();
+  };
+  const wait = async (milliseconds) => {
+    check();
     try {
-      url = new URL(request2.url ?? "/", `http://${expectedHost}`);
+      await delay(Math.min(milliseconds, deadline - Date.now()), void 0, {
+        signal: abort.signal
+      });
     } catch {
-      response.writeHead(400).end("Invalid sign-in response.");
-      return;
+      abort.signal.throwIfAborted();
     }
-    if (request2.method !== "GET" || request2.headers.host !== expectedHost || url.pathname !== "/callback") {
-      response.writeHead(404).end();
-      return;
-    }
-    if ([...url.searchParams.keys()].some(
-      (key) => url.searchParams.getAll(key).length !== 1
-    ) || url.searchParams.get("state") !== state || url.searchParams.get("iss") !== issuer) {
-      response.writeHead(400).end("Invalid sign-in response.");
-      return;
-    }
-    if (url.searchParams.has("error")) {
-      response.end("Sign-in cancelled. Return to your terminal.");
-      rejectCallback(new Error("Sign-in was not approved."));
-      return;
-    }
-    if (!/^[A-Za-z0-9_-]{43}$/.test(url.searchParams.get("code") ?? "")) {
-      response.writeHead(400).end("Invalid sign-in response.");
-      return;
-    }
-    response.end(
-      "Sign-in response received. Return to your terminal to confirm completion."
-    );
-    receive(url);
-  });
-  const stop = (message) => {
-    const error = new Error(message);
-    abort.abort(error);
-    rejectCallback(error);
+    check();
   };
-  const cancel = () => stop("Sign-in cancelled.");
-  process.once("SIGINT", cancel);
-  const timer = setTimeout(
-    () => stop("Sign-in timed out. Run tc login again."),
-    options.timeoutMs ?? 3e5
-  );
+  const post = (operation, body, signal) => jsonRequest(issuer, `/cli/oauth/device/${operation}`, {
+    method: "POST",
+    headers: { "content-type": "application/json" },
+    body: JSON.stringify(body),
+    ...signal ? { signal } : {}
+  });
   try {
-    await new Promise((resolve, reject) => {
-      server.once("error", reject);
-      server.listen(0, "127.0.0.1", resolve);
-    });
-    const address = server.address();
-    if (!address || typeof address === "string")
-      throw new Error("Unable to start local sign-in listener.");
-    expectedHost = `127.0.0.1:${address.port}`;
-    const redirectUri = `http://${expectedHost}/callback`;
-    const client = await jsonRequest(issuer, "/cli/oauth/register", {
-      signal: abort.signal,
-      method: "POST",
-      headers: { "content-type": "application/json" },
-      body: JSON.stringify({
-        client_name: "Trillioncore CLI",
-        redirect_uris: [redirectUri],
-        token_endpoint_auth_method: "none"
-      })
-    });
-    if (typeof client.client_id !== "string" || !/^cli_[a-f0-9]{32}$/.test(client.client_id))
-      throw new Error("The service returned an invalid CLI client.");
-    const authorization = new URL(`${issuer}/cli/oauth/authorize`);
-    authorization.search = new URLSearchParams({
-      client_id: client.client_id,
-      redirect_uri: redirectUri,
-      response_type: "code",
-      code_challenge_method: "S256",
-      code_challenge: createHash("sha256").update(verifier).digest("base64url"),
-      resource: `${issuer}/cli`,
-      scope: "cli:access offline_access",
-      state
-    }).toString();
+    const registration = await post(
+      "register",
+      {
+        verifier_digest: createHash("sha256").update(verifier).digest("hex"),
+        user_code_digest: createHash("sha256").update(userCode).digest("hex")
+      },
+      abort.signal
+    );
+    if (typeof registration.request_id !== "string" || !deviceSecret.test(registration.request_id))
+      throw new Error("The service returned an invalid device request.");
+    requestId = registration.request_id;
+    if (registration.client_id !== "cli_device" || typeof registration.expires_in !== "number" || !Number.isInteger(registration.expires_in) || registration.expires_in <= 0 || registration.expires_in > 300 || typeof registration.interval !== "number" || !Number.isInteger(registration.interval) || registration.interval <= 0 || registration.interval > 30)
+      throw new Error("The service returned an invalid device request.");
+    deadline = Math.min(deadline, started + registration.expires_in * 1e3);
+    clearTimeout(timer);
+    check();
+    timer = setTimeout(
+      () => stop("Sign-in timed out. Run tc login again."),
+      deadline - Date.now()
+    );
+    const authorization = verificationUrl(
+      registration.verification_uri,
+      requestId
+    );
     if (options.onAuthorizationUrl)
-      await options.onAuthorizationUrl(authorization.toString());
+      await options.onAuthorizationUrl(authorization, userCode);
     else {
       console.log(
-        `Sign in and approve the organization/accounts in your browser:
+        `Open this URL in a browser on any device:
 ${authorization}
-Do not share this sign-in link. The browser must reach this computer's loopback listener.`
+Enter this separate code: ${userCode}
+Only approve a login you started. Keep the code private.`
       );
-      if (!options.noBrowser) openBrowser(authorization.toString());
+      if (!options.noBrowser) openBrowser(authorization);
     }
-    const approved = await callback;
-    const fresh = tokens(
-      await jsonRequest(issuer, "/cli/oauth/token", {
-        ...form({
-          grant_type: "authorization_code",
-          client_id: client.client_id,
-          redirect_uri: redirectUri,
-          code: approved.searchParams.get("code"),
-          code_verifier: verifier,
-          resource: `${issuer}/cli`
-        }),
-        signal: abort.signal
-      })
-    );
-    const credentials = { issuer, clientId: client.client_id, ...fresh };
+    let approved;
+    let attempts = 0;
+    const interval = registration.interval * 1e3;
+    while (true) {
+      check();
+      try {
+        approved = await post(
+          "poll",
+          { request_id: requestId, verifier },
+          abort.signal
+        );
+        check();
+        if (approved.status !== "pending") break;
+        attempts = 0;
+        await wait(interval);
+      } catch (error) {
+        check();
+        if (!(error instanceof RetryableAuthError)) throw error;
+        const backoff = Math.min(
+          1e4,
+          interval * 2 ** Math.min(attempts++, 4)
+        );
+        await wait(
+          Math.max(
+            interval,
+            backoff * (1 + Math.random() * 0.2),
+            error.retryAfterMs
+          )
+        );
+      }
+    }
+    if (approved.client_id !== "cli_device" || typeof approved.delivery_receipt !== "string" || !deviceSecret.test(approved.delivery_receipt))
+      throw new Error("The service returned an invalid device delivery.");
+    const fresh = tokens(approved, true);
+    const credentials = { issuer, clientId: "cli_device", ...fresh };
     try {
       const who = await identity(issuer, fresh.accessToken, abort.signal);
       const value = {
@@ -3476,8 +3532,10 @@ Do not share this sign-in link. The browser must reach this computer's loopback
       const store = sessionStore(environment);
       await store.withLock(async () => {
         const previous = await store.read().catch(() => null);
-        abort.signal.throwIfAborted();
+        check();
         await store.write(value);
+        persisted = true;
+        clearTimeout(timer);
         if (previous)
           await revoke(previous).catch(
             () => console.error(
@@ -3485,20 +3543,31 @@ Do not share this sign-in link. The browser must reach this computer's loopback
             )
           );
       });
-      return { issuer, ...who };
-    } catch (error) {
-      await revoke(credentials).catch(
+      await post("acknowledge", {
+        request_id: requestId,
+        verifier,
+        delivery_receipt: approved.delivery_receipt
+      }).catch(
         () => console.error(
-          "Incomplete sign-in could not be revoked. Revoke the unused connection on the Tokens page."
+          "Signed in, but delivery acknowledgement failed. The temporary delivery expires automatically."
         )
       );
+      return { issuer, ...who };
+    } catch (error) {
+      if (!persisted)
+        await revoke(credentials).catch(
+          () => console.error(
+            "Incomplete sign-in could not be revoked. Revoke the unused connection on the Tokens page."
+          )
+        );
       throw error;
     }
   } finally {
     clearTimeout(timer);
+    if (requestId && !persisted)
+      await post("cancel", { request_id: requestId, verifier }).catch(() => {
+      });
     process.removeListener("SIGINT", cancel);
-    server.closeAllConnections();
-    await new Promise((resolve) => server.close(() => resolve()));
   }
 }
 async function authenticatedEnvironment(environment = process.env) {
@@ -3853,6 +3922,11 @@ function renderIndex(result) {
     return [
       `Resources: ${catalog.connectionRef}`,
       `Last successful sync: ${catalog.freshness.lastSuccessfulSyncAt ?? "unknown"}; source completeness: ${catalog.freshness.sourceCompleteness}`,
+      ...catalog.lifecycle ? [
+        `Publication: ${catalog.lifecycle.publication}; source removals: ${catalog.lifecycle.sourceRemovals}`,
+        ...[["Active run", catalog.lifecycle.activeRun], ["Last finished run", catalog.lifecycle.lastFinishedRun]].map(([label, run]) => run ? `${label}: ${run.status}; recorded at: ${run.startedAt}; finished: ${run.completedAt ?? "not recorded"}; attempts: ${run.attemptCount}; pages: ${run.progress.pagesProcessed}; seen: ${run.progress.recordsSeen}; upserted: ${run.progress.recordsUpserted}; skipped: ${run.progress.recordsSkipped}` : `${label}: none recorded`),
+        "Run state is recorded metadata, not worker health. Completed does not mean full source coverage."
+      ] : [],
       ...catalog.resources.flatMap((resource) => [
         `${resource.name} (${resource.kind})`,
         ...resource.recordKinds ? [`  Observed record kinds: ${resource.recordKinds.join(", ") || "none"}`] : [],

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@trillioncore/cli",
-  "version": "1.0.0-next.3",
+  "version": "1.0.0-next.5",
   "description": "Trillioncore v1 CLI: sign in and read your authorized integration records",
   "type": "module",
   "license": "MIT",