google-colab-cli 0.7.2__tar.gz → 0.7.3__tar.gz

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/AGENTS.md

@@ -6,10 +6,9 @@
 - **Client**: `ColabClient` handles API interactions (assignment, unassignment).
 - **Auth**: `auth.py` exposes a single `get_credentials(config_path, provider)` facade that dispatches on the `AuthProvider` enum. Two providers are supported, selected via the global `--auth=oauth2|adc` flag (default `oauth2`):
   - `oauth2`: public `google-auth-oauthlib` `InstalledAppFlow`, token cached at `~/.config/colab-cli/token.json`. Reads the client OAuth config from `-c/--client-oauth-config` (default `~/.colab-cli-oauth-config.json`), falling back to the **bundled** `src/colab_cli/oauth_config.json` resource (re-added in PR #41 / `9f44fe2`, 2026-05-29 — the earlier "removed in `20eb88e`" note was stale/incorrect; the file exists and `auth.py:_get_google_auth_credentials` loads it via `importlib.resources`). As of 2026-06-11 the flow is a **remote copy-paste flow**, not a localhost server: `_run_remote_flow` sets `redirect_uri=https://sdk.cloud.google.com/applicationdefaultauthcode.html` + `token_usage=remote`, prints the URL, and reads the pasted code via `input()`. NEVER revert to OOB (`urn:ietf:wg:oauth:2.0:oob`) — Google blocked it in 2022 ("OOB flow has been blocked"); the `sdk.cloud.google.com` redirect is registered only to the bundled cloud-SDK client (`764086051850-...`), so any other client id gets `redirect_uri_mismatch`. Server-side acceptance/rejection of these variants is verifiable GET-only by building the authorization URL and inspecting whether Google reaches sign-in vs. an OAuth error page (no resources allocated).
-  - `adc`: Google Application Default Credentials via `google.auth.default()`. The CLI passes `scopes=PUBLIC_SCOPES` (which includes `colaboratory`) and re-applies via `creds.with_scopes()` for credential types that support it. **User credentials minted by `gcloud auth application-default login` ignore the `scopes=` kwarg AND raise `NotImplementedError` on `with_scopes`**: ADC users must explicitly re-authenticate with `gcloud auth application-default login --scopes=openid,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/colaboratory`. `userinfo.email` is required by the session backend at `colab.research.google.com` (assign/unassign/sessions/keep-alive return 401 without it); `colaboratory` is retained for forward compatibility and other Colab features (keep-alive no longer uses `colab.pa.googleapis.com` — see the Keep-alive note below); `openid` and `cloud-platform` are mandated by `gcloud` itself, which rejects scope lists that omit `cloud-platform` with `Invalid value for [--scopes]`. Service-account / GCE / GKE / impersonated creds get the right scopes transparently via `with_scopes`.
+  - `adc`: Google Application Default Credentials via `google.auth.default()`. The CLI passes `scopes=PUBLIC_SCOPES` (which includes `colaboratory`) and re-applies via `creds.with_scopes()` for credential types that support it. **User credentials minted by `gcloud auth application-default login` ignore the `scopes=` kwarg AND raise `NotImplementedError` on `with_scopes`**: ADC users must explicitly re-authenticate with `gcloud auth application-default login --scopes=openid,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/colaboratory`. `userinfo.email` is required by the session backend at `colab.research.google.com` (assign/unassign/sessions return 401 without it); `colaboratory` is retained for forward compatibility and other Colab features; `openid` and `cloud-platform` are mandated by `gcloud` itself, which rejects scope lists that omit `cloud-platform` with `Invalid value for [--scopes]`. Service-account / GCE / GKE / impersonated creds get the right scopes transparently via `with_scopes`.
 - **Backend Hosts**: Two distinct backends with different requirements:
-  - `colab.research.google.com` (session backend / `tun/m/...`): accepts the `userinfo.email` scope. Handles assign, unassign, the contents API, **and keep-alive** (see below).
-  - **Keep-alive (2026-06-15, issue #14)**: keep-alive is a **Tunnel Frontend (TFE) HTTP ping** — `GET https://colab.research.google.com/tun/m/<endpoint>/keep-alive/` with header `X-Colab-Tunnel: Google`, authenticated by the user's own Gaia bearer token (same host/credential as `assign`). TFE records `LastActiveTime` before forwarding, refreshing the idle timer. The VM usually doesn't answer on this path, so the request commonly **read-times-out even on success** — `client.keep_alive_assignment` therefore catches `requests.exceptions.ReadTimeout` and treats it as success, while genuine HTTP errors (e.g. 404 for a deleted assignment) propagate. This mirrors the official `colab-vscode` extension's `sendKeepAlive` (`src/colab/client.ts`). **DO NOT revert to the `colab.pa.googleapis.com` `RuntimeService/KeepAliveAssignment` RPC**: that RPC requires the caller to be a `serviceusage` consumer of Colab's internal project `1014160490159`, which no ordinary user account is, so it returned HTTP 403 `USER_PROJECT_DENIED` for every external user (issue #14) and silently idle-pruned their sessions within minutes. The browser only made that RPC work by riding the user's `google.com` cookie through an internal cookie-proxy (`colab.clients6.google.com`), which a headless bearer-token CLI cannot use. Verified live 2026-06-15 with a third-party account (the RPC 403'd; the tunnel ping succeeded and kept the VM alive).
+  - `colab.research.google.com` (session backend / `tun/m/...`): accepts the `userinfo.email` scope. Handles assign, unassign, CCU info, and the contents API.
 - **Runtime**: `ColabRuntime` wraps `jupyter-kernel-client` for execution.
 - **State**:
   - `StateStore` persists session metadata in `~/.config/colab-cli/sessions.json`.
@@ -43,7 +42,7 @@
 - **Automation**: `auth`, `drivemount`, `install`, `log`, `pay`, `version`, `update`.
 
 ## Release Tagging Workflow
-The package version is derived from the git tag via `hatch-vcs` (see `pyproject.toml`), so a release is just (1) a `CHANGELOG.md` entry and (2) a `vX.Y.Z` tag on the merged commit. Follow this workflow ONLY when the user explicitly requests a release (e.g. "cut a release", "tag v0.7.0", "release the keep-alive fix"). NEVER propose or perform a release proactively.
+The package version is derived from the git tag via `hatch-vcs` (see `pyproject.toml`), so a release is just (1) a `CHANGELOG.md` entry and (2) a `vX.Y.Z` tag on the merged commit. Follow this workflow ONLY when the user explicitly requests a release (e.g. "cut a release", "tag v0.7.0"). NEVER propose or perform a release proactively.
 
 1. **Verify a clean, current `main`**: `git fetch origin && git checkout main && git pull --ff-only origin main`. Confirm `git status` is clean. If `main` has diverged or there are unmerged feature branches the user expects in the release, stop and ask.
 2. **Identify unreleased commits**: Find the last release tag with `git describe --tags --abbrev=0`, then list commits since then with `git log <last-tag>..HEAD --oneline`. These are what the new changelog section must cover. Cross-reference each commit's PR number (the `(#NN)` suffix in the merge commit subject) — every entry in the changelog should cite at least one PR.
@@ -64,23 +63,20 @@ The package version is derived from the git tag via `hatch-vcs` (see `pyproject.
 4.  **No netrc**: Avoid `netrc` for token persistence in this project.
 5.  **Mocking Interactivity**: When testing commands that branch on `stdin.isatty()`, use the `is_stdin_tty` helper in `execution.py` and mock it via `mocker.patch("colab_cli.commands.execution.is_stdin_tty", return_value=...)`. This ensures tests don't hang in CI/agent environments.
 6.  **State Isolation**: Always patch the `colab_cli.common.state` singleton in tests to control session persistence and client behavior. Refer to `tests/conftest.py` for the standard global fixture.
-7.  **Fire-and-Forget Architecture**: The Colab CLI is a "fire-and-forget" tool. Avoid using background threads for long-running tasks within the main command flows. For persistent needs such as keep-alive, utilize detached background daemon processes (with PID tracking in the session state).
+7.  **Fire-and-Forget Architecture**: The Colab CLI is a "fire-and-forget" tool. Avoid using background threads or persistent daemon processes within command flows. Commands authenticate, perform their task, and exit cleanly.
 8.  **Verify the Local Install**: A globally-installed `colab` may exist on `PATH` (e.g. at `~/.local/bin/colab`) and can shadow the project's editable install when `uv run` is invoked from outside the repo. ALWAYS run shell commands with the repo as the working directory (e.g. via the `workdir` parameter, never `cd && cmd`) so `uv run colab ...` resolves to `.venv/bin/colab`. Confirm with `which colab` and `uv run which colab` if a CLI test produces unexpected results (e.g. flag-not-recognized errors for flags you just added). **Shebang invocations always resolve via `$PATH`**, so a script like `#!/usr/bin/env -S colab run ...` will pick up the stale global tool even when the editable install is current — when testing shebang-based behavior after a code change, always run `uv tool install --reinstall --force --from . colab` first, then verify with `colab version` (the version string includes the git short SHA). Encoded 2026-05-12 after the SystemExit-suppression fix appeared not to work in `examples/hello_colab.py` because the shebang resolved to a uv-tool install pinned to the prior commit.
 9.  **Isolate the Regression First**: When a user reports an error in code you just touched, do NOT assume your change caused it. First, reproduce the failure on `main` (or the branch point) to determine whether the bug is pre-existing. Only after confirming the regression is yours should you start debugging the new code. Encoded after spending a turn debugging "ADC broke `colab new`" only to discover `colab new --gpu A100` was already failing on `main` due to an A100-quota-vs-default issue unrelated to ADC.
 10. **Live Probes Allocate Real Resources**: Probing the Colab API to debug an issue creates real, billable assignments — every successful POST `/tun/m/assign` reserves a VM. Prefer GET-only (read) probes whenever possible. For any state-mutating call, (a) record every endpoint you create as you go, and (b) clean up via `client.unassign(endpoint)` (or `colab stop`) before declaring the investigation done. Then verify with `colab sessions` that nothing was orphaned.
 11. **Push Freshness**: The remote may have advanced during a session (other contributors land commits while you work). ALWAYS `git fetch <remote>` immediately before pushing or merging. If `git log main..<remote>/main` is non-empty, reset local `main` to the remote, rebase feature branches onto it, retest, then push. NEVER force-push `main` to recover from divergence.
 12. **Amend Safety**: Before `git commit --amend`, explicitly verify all three preconditions: (a) the user requested amend OR a pre-commit hook auto-modified files for an otherwise-successful commit, (b) HEAD was created by you in this conversation (`git log -1 --format='%an %ae'`), (c) the commit has not been pushed to a remote. If any precondition fails, create a new commit instead. NEVER amend a failed/rejected commit — fix the issue and create a new one.
-13. **Run Integration Tests Yourself**: Re-read AGENTS.md "Agent Execution Limitations" before claiming you can't run a test. The CANNOT-run list is exclusively interactive commands (`colab auth`, `colab drivemount`, `colab repl`, `colab console`). Tests built on `colab new` / `colab stop` / `colab log` are non-interactive and you MUST run them yourself before declaring a fix complete. Encoded after running through a full implement-and-document cycle for a fix that the integration test would have falsified in 30 seconds — the cure was `uv run bash integration/repro_keep_alive/test.sh`.
+13. **Run Integration Tests Yourself**: Re-read AGENTS.md "Agent Execution Limitations" before claiming you can't run a test. The CANNOT-run list is exclusively interactive commands (`colab auth`, `colab drivemount`, `colab repl`, `colab console`). Tests built on `colab new` / `colab stop` / `colab log` are non-interactive and you MUST run them yourself before declaring a fix complete. Encoded after running through a full implement-and-document cycle for a fix that the integration test would have falsified in 30 seconds — the cure was `uv run bash integration/repro_run_command/test.sh`.
 14. **Heed Research Caveats**: When a research subagent surfaces a caveat ("the proto allows it but the policy may reject"), treat it as a TODO to verify, not as a footnote. The validation pattern: shell out to the actual service with the proposed inputs and confirm the response matches expectations BEFORE writing the code that depends on it. For policy-gated paths, run a one-shot probe before committing to a design.
 15. **Two distinct auths — never confuse them**: This codebase has two unrelated authentication concerns: (a) **CLI-to-Colab-control-plane**: how `Client` authenticates HTTP requests to `colab.research.google.com` and `colab.pa.googleapis.com`. Driven by the global `--auth={oauth2,adc}` flag and `auth.py:get_credentials`. The OAuth2 path is bootstrapped automatically by `google_auth_oauthlib`'s `InstalledAppFlow` on first invocation when `~/.config/colab-cli/token.json` doesn't exist; **no separate command is needed** — any `--auth=oauth2` invocation (e.g. `colab --auth=oauth2 sessions`) triggers the browser consent flow. (b) **VM-side credentials**: how the `colab auth` subcommand injects user GCP credentials *into* the running Colab kernel so notebook code (e.g. `gcloud`, BigQuery client) can make authenticated calls from inside the VM. This uses the `USE_AUTH_EPHEM='0'` gcloud-fallback path executed on the kernel via `ColabRuntime`. **NEVER tell a user to "run `colab auth`" as a prerequisite for fixing CLI-side auth issues** — they're orthogonal layers and the suggestion misleads.
-16. **Detached daemons inherit nothing useful**: When spawning a detached child process via `subprocess.Popen` (e.g. `spawn_keep_alive`), the child does NOT inherit the parent's parsed Typer flags. It re-parses argv from scratch, so any global flag the parent saw via `--auth=adc` or `--config /tmp/foo.json` MUST be re-emitted as part of the child's command line. Forgetting this causes silent fallback to Typer defaults: in 2026-04-30 this manifested as the keep-alive daemon (a) using OAUTH2 instead of the parent's ADC, and (b) reading from `~/.config/colab-cli/sessions.json` instead of the parent's `--config` path. Always propagate every relevant global flag in `cmd = [sys.executable, "-m", ...]` BEFORE the subcommand name (Typer requires global flags before the subcommand).
-17. **Persist-before-spawn for daemons that read shared state**: When a detached child reads from a state file the parent owns (e.g. `state.store.get(session_name)`), the parent MUST persist BEFORE spawning. Otherwise the child can race ahead of the parent's `add()` call and observe an empty store. Symptom in 2026-04-30: keep-alive daemon exits immediately with `keep_alive_stopped reason=session_not_found iters=1 duration=0.0s`. Fix: call `state.store.add(s)` once before `spawn_keep_alive`, and again after (to capture the PID).
-18. **[SUPERSEDED 2026-06-15 — keep-alive no longer calls `colab.pa.googleapis.com`; see issue #14 / the Keep-alive note above] `colab.pa.googleapis.com` rejects ADC user creds without `X-Goog-User-Project`**: When calling `colab.pa.googleapis.com` from ADC user credentials minted by `gcloud auth application-default login`, the bearer token carries the user's gcloud quota project. The colab API key sent alongside it belongs to a different project (Colab, `1014160490159`). The backend enforces project-match between the two and returns HTTP 400 `CONSUMER_INVALID` ("The API Key and the authentication credential are from different projects."). The old fix was to send `X-Goog-User-Project: 1014160490159` to pin the consumer project — but that in turn required `serviceusage.serviceUsageConsumer` on project `1014160490159`, which ordinary users lack, yielding HTTP 403 `USER_PROJECT_DENIED` (the issue #14 root cause). Both failure modes are now moot because keep-alive uses the TFE tunnel ping on `colab.research.google.com` instead. Retained here as institutional knowledge: if any future code path must call `colab.pa.googleapis.com` with a bearer token, it will face this same project-entitlement wall for non-internal accounts.
-19. **Pydantic validation requires a schema**: `_issue_request` accepts an optional `schema=` and historically called `TypeAdapter(schema).validate_python(...)` unconditionally. When `schema=None` (a caller that doesn't care about the response body — e.g. fire-and-forget RPCs like `KeepAliveAssignment` that return `[]`), this raises `pydantic.ValidationError: Input should be None`. Always guard with `if schema is None: return` after the empty-body short-circuit.
-20. **Suggest the branch-diff review command after committing**: The user reviews changes with `git diff main..<branch-name>` (full cumulative diff against `main`, not just the latest commit). After landing one or more commits on a feature branch, ALWAYS suggest the exact command — e.g. "Review with `git diff main..sort-help-commands`" — instead of `git show <sha>` (which only shows a single commit and misses context when a branch has multiple commits). Encoded 2026-05-05 after suggesting `git show 9d9c7da` for a branch the user wanted to review holistically.
-21. **Verify research-tool claims with primary sources**: Research tools and AI assistants can be confidently wrong, especially about edge cases or features outside their training corpus. When such a tool says "X is not used / not parsed / doesn't exist", treat it as a hypothesis to verify, not a fact. Always cross-check against the primary source (the actual code or config) — and when a tool names files, check whether the indirection chain it describes actually exists. The cost of believing the tool when it's wrong is shipping a non-functional feature; the cost of double-checking is small. Encoded 2026-05-05 after a `colab url` first-cut shipped the wrong URL format because of unverified output.
-22. **Clean up orphaned assignments before finishing live tests**: After running a live integration test, `colab sessions` may show server-side assignments that the local `colab stop` couldn't see (e.g. assignments leaked from earlier in the session, or from crashed prior runs). Always run `colab sessions` as the final cleanup step, and for any `[?]`-marked orphan, run `python -c "from colab_cli.common import state; state.client.unassign('<endpoint>')"` (the CLI doesn't expose a direct unassign-by-endpoint command). Re-verify with `colab sessions` returning "No active sessions". Encoded 2026-05-05 after the first `colab url` live test left an orphan from a prior conversation turn that would have idled-out and billed compute units.
-23. **Forward unknown args through Typer with `context_settings`**: Typer/Click consumes any token starting with `-`/`--` as a flag of the parent command unless told otherwise. For a subcommand like `colab run script.py --some-script-flag` (where `--some-script-flag` belongs to the user's script, not to `colab`), declare it with `app.command(name="run", context_settings={"allow_extra_args": True, "ignore_unknown_options": True})` and accept the positional with `Annotated[Optional[List[str]], typer.Argument(...)] = None`. Also use `repr()` (not f-string interpolation) when embedding those forwarded strings into kernel-side Python source — `repr()` produces a safe round-trippable literal regardless of the user's shell-passed quotes, backslashes, or non-ASCII bytes. Encoded 2026-05-12 while implementing `colab run`.
+16. **Pydantic validation requires a schema**: `_issue_request` accepts an optional `schema=` and historically called `TypeAdapter(schema).validate_python(...)` unconditionally. When `schema=None` (a caller that doesn't care about the response body — e.g. unassign or fire-and-forget endpoints that return `[]`), this raises `pydantic.ValidationError: Input should be None`. Always guard with `if schema is None: return` after the empty-body short-circuit.
+17. **Suggest the branch-diff review command after committing**: The user reviews changes with `git diff main..<branch-name>` (full cumulative diff against `main`, not just the latest commit). After landing one or more commits on a feature branch, ALWAYS suggest the exact command — e.g. "Review with `git diff main..sort-help-commands`" — instead of `git show <sha>` (which only shows a single commit and misses context when a branch has multiple commits). Encoded 2026-05-05 after suggesting `git show 9d9c7da` for a branch the user wanted to review holistically.
+18. **Verify research-tool claims with primary sources**: Research tools and AI assistants can be confidently wrong, especially about edge cases or features outside their training corpus. When such a tool says "X is not used / not parsed / doesn't exist", treat it as a hypothesis to verify, not a fact. Always cross-check against the primary source (the actual code or config) — and when a tool names files, check whether the indirection chain it describes actually exists. The cost of believing the tool when it's wrong is shipping a non-functional feature; the cost of double-checking is small. Encoded 2026-05-05 after a `colab url` first-cut shipped the wrong URL format because of unverified output.
+19. **Clean up orphaned assignments before finishing live tests**: After running a live integration test, `colab sessions` may show server-side assignments that the local `colab stop` couldn't see (e.g. assignments leaked from earlier in the session, or from crashed prior runs). Always run `colab sessions` as the final cleanup step, and for any `[?]`-marked orphan, run `python -c "from colab_cli.common import state; state.client.unassign('<endpoint>')"` (the CLI doesn't expose a direct unassign-by-endpoint command). Re-verify with `colab sessions` returning "No active sessions". Encoded 2026-05-05 after the first `colab url` live test left an orphan from a prior conversation turn that would have idled-out and billed compute units.
+20. **Forward unknown args through Typer with `context_settings`**: Typer/Click consumes any token starting with `-`/`--` as a flag of the parent command unless told otherwise. For a subcommand like `colab run script.py --some-script-flag` (where `--some-script-flag` belongs to the user's script, not to `colab`), declare it with `app.command(name="run", context_settings={"allow_extra_args": True, "ignore_unknown_options": True})` and accept the positional with `Annotated[Optional[List[str]], typer.Argument(...)] = None`. Also use `repr()` (not f-string interpolation) when embedding those forwarded strings into kernel-side Python source — `repr()` produces a safe round-trippable literal regardless of the user's shell-passed quotes, backslashes, or non-ASCII bytes. Encoded 2026-05-12 while implementing `colab run`.
 
 ## Agent Execution Limitations (What I Can vs Cannot Run)
 As an AI agent operating via non-interactive shell tools (`run_shell_command`), there are strict limits on what I can test autonomously without human intervention:

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: google-colab-cli
-Version: 0.7.2
+Version: 0.7.3
 Summary: CLI for interacting with Colab.
 Project-URL: Homepage, https://github.com/googlecolab/google-colab-cli
 Project-URL: Repository, https://github.com/googlecolab/google-colab-cli
@@ -21,7 +21,7 @@ Requires-Dist: filelock>=3.29.2
 Requires-Dist: google-auth-oauthlib>=1.3.0
 Requires-Dist: google-auth>=2.49.1
 Requires-Dist: html2text>=2024.2.26
-Requires-Dist: jupyter-kernel-client==0.8
+Requires-Dist: jupyter-kernel-client
 Requires-Dist: nbformat>=5.10.4
 Requires-Dist: packaging>=24.0
 Requires-Dist: prompt-toolkit>=3.0.52
@@ -55,7 +55,7 @@ Designed to support seamless developer productivity, headless automation, and AI
 * **Instant VM Provisioning:** Spin up CPU, GPU (T4, L4, G4, H100, A100), or TPU (v5e1, v6e1) runtimes in seconds.
 * **Robust Code Execution:** Run local Python scripts, Jupyter Notebooks (`.ipynb`), or piped `stdin` code; launch interactive REPLs or raw TTY console shells.
 * **Ephemeral Job Runner (`colab run`):** Provision a fresh VM, execute a local script with forwarded arguments, retrieve output files, and automatically tear down the runtime in a single command.
-* **Automatic Keep-Alive:** Built-in background daemon automatically prevents idle VM termination, keeping resource allocations active without requiring open browser tabs.
+* **Activity-Based Liveness:** Sessions stay alive as long as the kernel is active, with the Colab backend automatically maintaining liveness.
 * **Seamless Workspace Automation:** Mount Google Drive, authenticate Google Cloud Platform (GCP) credentials, and install dependencies with high-performance `uv` package management.
 * **State & Log Archival:** Inspect local session states or export interactive history logs to standard Jupyter Notebooks, Markdown, or structured JSONL.
 
@@ -108,7 +108,7 @@ Run `colab <command> --help` to view specific options, defaults, and detailed he
 | `colab sessions` | List all active sessions currently active on the backend |
 | `colab status [-s NAME]` | Display hardware, machine shape, status, and local metadata for active sessions |
 | `colab restart-kernel [-s NAME]` | Restart the active session's Jupyter kernel |
-| `colab stop [-s NAME]` | Terminate a session VM and tear down its keep-alive daemon |
+| `colab stop [-s NAME]` | Terminate a session VM and release its resources |
 | `colab url [-s NAME] [--open]` | Print or open a browser URL connecting to the active session |
 
 ### Execution
@@ -213,7 +213,7 @@ Make the script executable (`chmod +x script.py`) and run it: `./script.py`. The
 
 For comprehensive architectural overviews and deep-dives into specific CLI sub-systems, refer to the detailed documentation:
 
-* [Session Management & Keep-Alive Architecture](docs/01_session_management.md)
+* [Session Management Architecture](docs/01_session_management.md)
 * [Interactive & Non-Interactive Execution Design](docs/02_execution_and_interactive.md)
 * [File Management & Jupyter Contents API](docs/03_file_management.md)
 * [Authentication Providers & VM Automation](docs/04_automation_and_utility.md)

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/README.md

@@ -19,7 +19,7 @@ Designed to support seamless developer productivity, headless automation, and AI
 * **Instant VM Provisioning:** Spin up CPU, GPU (T4, L4, G4, H100, A100), or TPU (v5e1, v6e1) runtimes in seconds.
 * **Robust Code Execution:** Run local Python scripts, Jupyter Notebooks (`.ipynb`), or piped `stdin` code; launch interactive REPLs or raw TTY console shells.
 * **Ephemeral Job Runner (`colab run`):** Provision a fresh VM, execute a local script with forwarded arguments, retrieve output files, and automatically tear down the runtime in a single command.
-* **Automatic Keep-Alive:** Built-in background daemon automatically prevents idle VM termination, keeping resource allocations active without requiring open browser tabs.
+* **Activity-Based Liveness:** Sessions stay alive as long as the kernel is active, with the Colab backend automatically maintaining liveness.
 * **Seamless Workspace Automation:** Mount Google Drive, authenticate Google Cloud Platform (GCP) credentials, and install dependencies with high-performance `uv` package management.
 * **State & Log Archival:** Inspect local session states or export interactive history logs to standard Jupyter Notebooks, Markdown, or structured JSONL.
 
@@ -72,7 +72,7 @@ Run `colab <command> --help` to view specific options, defaults, and detailed he
 | `colab sessions` | List all active sessions currently active on the backend |
 | `colab status [-s NAME]` | Display hardware, machine shape, status, and local metadata for active sessions |
 | `colab restart-kernel [-s NAME]` | Restart the active session's Jupyter kernel |
-| `colab stop [-s NAME]` | Terminate a session VM and tear down its keep-alive daemon |
+| `colab stop [-s NAME]` | Terminate a session VM and release its resources |
 | `colab url [-s NAME] [--open]` | Print or open a browser URL connecting to the active session |
 
 ### Execution
@@ -177,7 +177,7 @@ Make the script executable (`chmod +x script.py`) and run it: `./script.py`. The
 
 For comprehensive architectural overviews and deep-dives into specific CLI sub-systems, refer to the detailed documentation:
 
-* [Session Management & Keep-Alive Architecture](docs/01_session_management.md)
+* [Session Management Architecture](docs/01_session_management.md)
 * [Interactive & Non-Interactive Execution Design](docs/02_execution_and_interactive.md)
 * [File Management & Jupyter Contents API](docs/03_file_management.md)
 * [Authentication Providers & VM Automation](docs/04_automation_and_utility.md)

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/cloudbuild.yaml

@@ -53,10 +53,11 @@ steps:
       - -c
       - |
         set -e
-        gcloud artifacts packages delete "${_EG_PROJECT_NAME}" \
-          --project="${_AR_PROJECT}" \
-          --repository="${_AR_REPOSITORY}" \
-          --location="${_AR_LOCATION}" \
+        gcloud artifacts versions delete ${TAG_NAME}  \
+          --package={_EG_PROJECT_NAME} \
+          --project={_AR_PROJECT} \
+          --repository=${_AR_REPOSITORY} \
+          --location=us \
           --quiet || true
 
   # 2. Build sdist + wheel into dist/ using uv.
@@ -89,12 +90,6 @@ steps:
         set -e
         pip install --quiet --root-user-action=ignore \
           twine keyrings.google-artifactregistry-auth
-        gcloud artifacts versions delete ${TAG_NAME}  \
-          --package={_EG_PROJECT_NAME} \
-          --project={_AR_PROJECT} \
-          --repository=${_AR_REPOSITORY} \
-          --location=us \
-          --quiet
         twine upload \
           --repository-url "https://${_AR_LOCATION}-python.pkg.dev/${_AR_PROJECT}/${_AR_REPOSITORY}/" \
           --verbose \

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/docs/01_session_management.md

@@ -1,5 +1,7 @@
 ---
 log:
+2026-09-25: Runtime proxy tokens are now refreshed before expiry. `SessionState` persists `token_expires_at`, and `State.get_session` re-fetches the token via `GET /tun/m/assignments` when it's within 5 minutes of expiring.
+2026-09-25: Removed the keep-alive background daemon and related client/CLI ping functions. VM liveness is automatically maintained by the Colab backend based on kernel activity.
 2026-08-10: Added `colab usage` for account-level compute-unit rate/balance via `GET /tun/m/ccu-info` on the session backend (same bearer token as `colab new`).
 2026-08-09: Added `--high-mem` to `colab new`, `colab run`, and `colab ssh` (auto-create). Assign requests now send `shape=hm` when high-RAM is requested; `colab sessions` and `colab status` display machine shape.
 2026-06-15: Switched the keep-alive daemon from the `colab.pa.googleapis.com` `RuntimeService/KeepAliveAssignment` RPC to a Tunnel Frontend HTTP ping (`GET /tun/m/<endpoint>/keep-alive/` with `X-Colab-Tunnel: Google`) on `colab.research.google.com`. The RPC required `serviceusage` consumer access to Colab's internal project `1014160490159`, which ordinary user accounts lack, so every external user hit HTTP 403 `USER_PROJECT_DENIED` and their CLI sessions were idle-pruned within minutes (issue #14). Reproduced live with a third-party account; verified the tunnel ping is accepted by the same bearer-token credential that already works for `assign`. A `ReadTimeout` on the ping is treated as success (TFE records activity before forwarding to the often-non-responding VM). Generalized the pre-flight remediation messaging away from the now-irrelevant `colaboratory`/`pa.googleapis.com` framing, and removed the dead grpc-web client-registry/API-key code.
@@ -79,19 +81,11 @@ The CLI maps user flags to these backend parameters:
 - **API**: `GET https://colab.research.google.com/tun/m/assignments` (based on `colab-agent` implementation).
 - **Function**: Lists all active VM assignments for the user. This is useful for synchronizing local state with actual backend sessions.
 
-### 5. Keep-Alive Protocol
-To prevent Colab VMs from being deleted due to idle timeouts (standard is ~90 minutes), the CLI implements a background keep-alive mechanism.
-- **Daemon Process**: Since the CLI is a fire-and-forget tool, `colab new` spawns a detached background process running a hidden `keep-alive` command.
-- **Tunnel ping**: Every 60 seconds, the daemon issues `GET https://colab.research.google.com/tun/m/<endpoint>/keep-alive/` with the header `X-Colab-Tunnel: Google`, authenticated with the user's own Gaia bearer token (the same credential and host used for `/tun/m/assign`). The Tunnel Frontend (TFE) records `LastActiveTime` before forwarding the request, which refreshes the idle timer. This matches the official `colab-vscode` extension's `sendKeepAlive`. TFE notes the activity on arrival and then forwards to the VM, which often does not answer on this path — so the request commonly read-times-out even though the keep-alive succeeded; a `ReadTimeout` is therefore treated as success, while genuine HTTP errors (e.g. 404 for a deleted assignment) propagate.
-  - **Why not the RuntimeService RPC**: The previous implementation called `google.internal.colab.v1.RuntimeService/KeepAliveAssignment` at `colab.pa.googleapis.com` with `X-Goog-User-Project: 1014160490159`. That path requires the caller to be a `serviceusage` consumer of Colab's internal project `1014160490159`, which no ordinary user account is — so it returned HTTP 403 `USER_PROJECT_DENIED` for every external user, causing CLI sessions to be idle-pruned within minutes (issue #14). Dropping the header instead produced HTTP 400 `CONSUMER_INVALID` (public API-key project ≠ bearer-token quota project). The browser only succeeds because it rides the user's `google.com` cookie through an internal cookie-proxy (`colab.clients6.google.com`), which a headless bearer-token client cannot use. The TFE tunnel ping needs no project entitlement and works for any account that can assign a VM.
-- **Pre-flight (`colab new`, OAuth2/ADC only)**: Immediately after a successful `assign`, the CLI invokes `keep_alive_assignment` once synchronously. If the response is 403 with a `SCOPE_NOT_PERMITTED` body, it unassigns the new VM (to avoid leaking a billable assignment) and prints a per-provider remediation message before exiting non-zero. Other errors are tolerated — the daemon will retry and surface them via the structured event log. (Because keep-alive now uses the same backend/credential as `assign`, a scope failure at this stage is rare — assignment would normally have failed first.)
-- **Structured logging**: The daemon emits `keep_alive_started` (with `pid`, `endpoint`), one `keep_alive_error` per failed iteration (with `status_code`, `error_type`, truncated `error`, `response_body`, `iteration`, `consecutive_4xx`), and `keep_alive_stopped` (with `reason`, `iterations`, `duration_seconds`, optional `last_error`, optional `expected_endpoint`/`actual_endpoint`). All three are rendered specially by `colab log` so users get diagnostic context without parsing JSONL by hand.
-- **Termination**:
-    - **Explicit**: `colab stop` terminates the daemon using its stored PID.
-    - **Implicit**: If a session is pruned (e.g., during `sync_sessions`), its daemon is also terminated.
-    - **Safety Fallback**: The daemon automatically terminates after 24 hours to prevent permanent zombie processes.
-    - **State Check**: The daemon periodically verifies that its session still exists in the local state store; if missing, it exits.
-    - **Repeated 4xx**: After two consecutive 4xx responses, the daemon exits with `reason=consecutive_4xx_errors`. With the TFE tunnel ping, a normal read-timeout is not counted as a 4xx (it is treated as success), so this branch is now reached only by genuine HTTP errors such as a 404 for a deleted/expired assignment.
+### 5. Session Liveness
+VM liveness is automatically maintained by the Colab backend based on active kernel executions and interactions. The CLI runs no keep-alive daemon. A session persists until explicitly terminated via `colab stop` or until the backend reclaims it due to inactivity.
+
+### 6. Runtime Proxy Token Refresh
+The runtime proxy token (`runtimeProxyInfo.token`) authenticates every request to the VM and expires after `tokenExpiresInSeconds` (1 hour). `SessionState.token_expires_at` stores the absolute expiry, computed when the token is received. Commands that talk to the VM load their session through `State.get_session(name)`, which, when the token is missing an expiry or is within 5 minutes of it, calls `GET /tun/m/assignments`, takes the fresh token/URL for the matching endpoint, and persists it. If the endpoint is no longer assigned, the session is pruned. `sync_sessions` also writes back the fresh tokens it already fetches. Long-lived connections (`repl`, `console`, `ssh`) only present the token on initial connect and aren't refreshed mid-session.
 
 ## TODO / Future Work
 - **Backend Sync**: Implement a way to reconcile the local `sessions.json` with the output of `colab sessions`.
@@ -107,7 +101,7 @@ To prevent Colab VMs from being deleted due to idle timeouts (standard is ~90 mi
 - Handle authentication headers (likely `Authorization: Bearer <token>` or cookies).
 
 ### State Persistence & File Locking
-- **Stores**: `StateStore` (`sessions.json`) and `SettingsStore` (`settings.json`) both derive from `_LockedFileStore`, which guards concurrent access across independent `colab` invocations and the detached keep-alive daemon.
+- **Stores**: `StateStore` (`sessions.json`) and `SettingsStore` (`settings.json`) both derive from `_LockedFileStore`, which guards concurrent access across independent `colab` invocations.
 - **Cross-platform locking**: Locking uses the [`filelock`](https://pypi.org/project/filelock/) library rather than `fcntl.flock`. `fcntl` is POSIX-only and is unavailable on Windows, so the original implementation crashed on import there. `filelock` provides the same advisory cross-process locking on Linux, macOS, and Windows.
 - **Shared vs. exclusive**: Each store owns a `filelock.ReadWriteLock` bound to a sidecar file (`<path>.lock`). Reads acquire `read_lock()` (shared — multiple concurrent readers allowed) and writes acquire `write_lock()` (exclusive). This preserves the `LOCK_SH`/`LOCK_EX` distinction of the previous `fcntl` implementation.
 - **`is_singleton=False`**: The `ReadWriteLock` is created with `is_singleton=False`. With `filelock`'s default (`True`), two `ReadWriteLock` objects for the same path *within a single process* are deduplicated into one reentrant lock; its reentrancy guard then raises `RuntimeError` when two threads each construct their own `StateStore` and contend for the write lock. Disabling the singleton registry makes each store's lock independent so they serialize via the underlying file lock instead.

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/docs/04_automation_and_utility.md

@@ -1,5 +1,6 @@
 ---
 log:
+2026-09-25: Cleaned up references to keep-alive background daemon and pre-flight pings. VM liveness is automatically maintained by the Colab backend based on active kernel activity.
 2026-08-10: Added `colab usage` for account-level compute-unit rate/balance via `GET /tun/m/ccu-info` on the session backend (same bearer token as `colab new`).
 2026-06-11: Replaced the `oauth2` provider's `run_local_server()` (localhost redirect) with a remote copy-paste flow (`_run_remote_flow` in `auth.py`). The CLI now prints an authorization URL built with `redirect_uri=https://sdk.cloud.google.com/applicationdefaultauthcode.html` and `token_usage=remote`, then reads the pasted authorization code via `input()` and exchanges it with `flow.fetch_token(code=...)`. This is the same flow `gcloud auth application-default login` uses and works identically in local and remote/headless/container environments, removing the heuristic of whether to auto-open a browser. Confirmed server-side acceptance with a live GET-only check against the bundled cloud-SDK client (`764086051850-...`); the OOB redirect and a non-bundled client id were both verified to be rejected (`OOB flow has been blocked` / `redirect_uri_mismatch`). Unit tests in `tests/test_auth.py` assert no localhost server is started, the redirect URI + `token_usage=remote` are set, and the pasted code is exchanged.
 2026-06-01: Enabled `colab update --install` self-update on macOS in addition to Linux. Refactored platform check logic to keep the implementation DRY and updated both tests and documentation. Also, on these platforms, an additional message is shown recommending `colab update --install` to upgrade in place, positioned above the standard `pip`/`uv` installation command.
@@ -58,18 +59,9 @@ allowing the core `Client` to remain authentication-agnostic — it only sees a
 ### Required Scopes
 
 The CLI talks to the Colab session backend at `colab.research.google.com`
-for assignment, unassignment, the contents API, **and keep-alive** (the TFE
-tunnel ping — see `01_session_management.md`). The `userinfo.email` scope is
+for assignment, unassignment, CCU info, and the contents API. The `userinfo.email` scope is
 sufficient for this host.
 
-> Historical note: keep-alive previously used the `RuntimeService`
-> (`KeepAliveAssignment`) at `colab.pa.googleapis.com`, which required the
-> `https://www.googleapis.com/auth/colaboratory` scope **and** the caller to
-> be a `serviceusage` consumer of Colab's internal project `1014160490159`.
-> The latter is impossible for ordinary user accounts, which made keep-alive
-> fail with HTTP 403 `USER_PROJECT_DENIED` for all external users (issue #14).
-> Keep-alive no longer touches `colab.pa.googleapis.com`.
-
 How each provider supplies the scope:
 
 -   **`oauth2`**: `PUBLIC_SCOPES` already includes `colaboratory`, so the
@@ -92,16 +84,12 @@ How each provider supplies the scope:
     ```
 
     `userinfo.email` is required for the session backend at
-    `colab.research.google.com` (otherwise assign/unassign/sessions/keep-alive
+    `colab.research.google.com` (otherwise assign/unassign/sessions
     return HTTP 401); `colaboratory` is retained for forward compatibility and
     other Colab features; `openid` and `cloud-platform` are mandated by
     `gcloud` itself (`gcloud auth application-default login` rejects scope
     lists that omit `cloud-platform` with `Invalid value for [--scopes]`).
 
-`colab new` performs a one-shot keep-alive pre-flight after `assign`
-succeeds so missing-scope failures surface immediately (with per-provider
-remediation guidance) rather than silently after ~1 minute via the daemon.
-
 ## Approach
 
 ### 1. Authentication (`colab auth`)

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/docs/05_run_command.md

@@ -1,5 +1,6 @@
 ---
 log:
+2026-09-25: Removed keep-alive daemon spawning from session allocation and teardown. `--keep` remains supported and skips unassignment on completion.
 2026-08-09: Added `--high-mem` flag (passthrough to session creation; sends `shape=hm` on assign when supported).
 2026-05-12: Initial design and implementation of `colab run <script.py> [args...]`. Combines `colab new` + `colab exec` + `colab stop` into a single fire-and-forget invocation so a Python file can use `#!/usr/bin/env -S colab run` as a shebang line and execute on a freshly-allocated Colab VM. Adds `--keep` (skip auto-stop), `--gpu` / `--tpu` (passthrough to session creation), `-s/--session` (name the ephemeral session), and propagates the script's exit status (non-zero on any uncaught exception in the kernel). The script's `sys.argv` is re-set inside the kernel to mirror native `python script.py arg1 arg2` semantics, and `__name__` is set to `"__main__"`.
 2026-05-12: Native CPython exit-code semantics for `sys.exit()` / `raise SystemExit(...)` from the script body. The Colab kernel reports a `SystemExit` as `output_type=='error'`, which under the previous logic would have (a) printed the IPython traceback (`An exception has occurred, use %tb...`) and (b) flagged the run as a failure regardless of the integer exit code. Now: `sys.exit()` / `sys.exit(0)` exit 0 silently; `sys.exit(N)` exits N; `sys.exit('msg')` exits 1 (matching CPython). The IPython "To exit: use 'exit', 'quit', or Ctrl-D." UserWarning is filtered via the prelude. Encoded after running `examples/gpu_hello.py` end-to-end and seeing the noisy `SystemExit: 0` traceback at the end of an otherwise-successful GPU run.
@@ -49,7 +50,7 @@ print(torch.cuda.get_device_name(0))
 
 ## Behavior
 
-1. **Allocate**: Creates a fresh session (mirrors `colab new` end-to-end: `assign` → keep-alive pre-flight → spawn keep-alive daemon → persist `SessionState`). Session name defaults to `run-<6 hex>`.
+1. **Allocate**: Creates a fresh session (mirrors `colab new`: `assign` → persist `SessionState`). Session name defaults to `run-<6 hex>`.
 2. **Execute**: Reads the script file. Prepends a deterministic prelude that re-sets `sys.argv` and `__name__` so the script body sees the same execution context as `python script.py arg1 arg2`:
    ```python
    import sys
@@ -62,16 +63,13 @@ print(torch.cuda.get_device_name(0))
    - Sends `runtime.stop(shutdown_kernel=True)` (best-effort).
    - Calls `state.client.unassign(endpoint)` to free the billable VM.
    - Removes the session from `StateStore`.
-   - Kills the keep-alive daemon (`kill_process(s.keep_alive_pid)`).
    - Logs `session_terminated` with `reason="run_completed"` (or `"run_failed"`).
 
-If `--keep` is set, the session remains visible in `colab sessions` and `colab status` and can be reused with `colab exec -s <name>`, `colab repl -s <name>`, etc., until the user runs `colab stop` (or the keep-alive daemon hits its 24h cap).
+If `--keep` is set, the session remains visible in `colab sessions` and `colab status` and can be reused with `colab exec -s <name>`, `colab repl -s <name>`, etc., until the user runs `colab stop` (or the backend reclaims it due to inactivity).
 
 ## AGENTS.md Constraints Honoured
-- **Item 7 (no background threads)**: The keep-alive daemon is the existing detached process from `colab new`; this command introduces no new threads.
+- **Item 7 (fire-and-forget)**: No background threads or daemons are created; commands execute and exit cleanly.
 - **Item 10 (live probes allocate real resources)**: The teardown is in a `try/finally` so an exception during execution still releases the VM. Tests assert `unassign` is called even when the script errors.
-- **Item 16 (daemon flag propagation)**: Reuses `spawn_keep_alive(...)` which already propagates `--auth` and `--config`.
-- **Item 17 (persist-before-spawn)**: Uses the same persist-before-spawn pattern as `colab new`.
 
 ## Testing Strategy (TDD)
 

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/docs/06_ssh_access.md

@@ -1,5 +1,6 @@
 ---
 log:
+2026-09-25: Cleaned up keep-alive daemon references in session creation and teardown documentation.
 2026-08-09: Added `--high-mem` passthrough when `colab ssh` auto-creates a runtime (forwards to `colab new --high-mem`).
 2026-07-17: Initial design and implementation of `colab ssh` — client side of SSH-over-WebSocket runtime access. Adds three modes (interactive shell, `-s SESSION`, and `--proxy-mode` OpenSSH ProxyCommand bridge), `--identity/-i` key selection, and per-HTTP-status handshake error messages. Server side is out of scope for this repo; the subcommand is a no-op against runtimes that do not expose the `/colab/ssh` endpoint (surfaces an actionable HTTP 404 message).
 2026-07-22: Bare `colab ssh` now auto-creates a runtime (via `colab new`) when you have no active session, with `--gpu/--tpu` passthrough and `--rm` to stop an auto-created runtime on exit. Fixed two client bugs: the dead 403 branch (feature-off returns 404, not 403) and the RSA guidance (all `ssh-rsa` keys are server-rejected, so `id_rsa` is no longer auto-scanned and the 400 message no longer advertises `rsa-sha2`). Added `tests/test_ssh_wire_contract.py` (real loopback-server wire assertions) and `tests/test_ssh_autocreate.py`.
@@ -57,9 +58,8 @@ remote command, so to also land in `/content` add `RequestTTY yes` and
 1. **Session resolution / auto-create**: With `-s NAME`, resolves that session
    (via `state.resolve_session`, the same helper the other commands use). Bare
    `colab ssh` uses your only active session; with **no** session it auto-creates
-   one (mirrors `colab new` end-to-end: assign → keep-alive pre-flight → spawn
-   keep-alive daemon → persist `SessionState`); with **multiple** it errors and
-   asks you to pick one with `-s`.
+   one (mirrors `colab new`: assign → persist `SessionState`); with **multiple**
+   it errors and asks you to pick one with `-s`.
 2. **Connect**: Opens the WebSocket to `wss://<netloc>/colab/ssh?colab-runtime-proxy-token=<token>`
    and sends the resolved public key verbatim in the `X-Colab-Ssh-Pubkey` header
    (no transformation -- the bytes the user controls are exactly what the server
@@ -78,8 +78,8 @@ remote command, so to also land in `/content` add `RequestTTY yes` and
 5. **`--rm` teardown**: Stops the runtime when the session ends. In `--proxy-mode`
    this must survive how OpenSSH ends a `ProxyCommand`: on disconnect it sends
    **SIGHUP** (verified), not just stdin EOF, and Python's default SIGHUP action
-   would terminate the process before the teardown `finally` ran — leaking the
-   runtime and its keep-alive daemon. `--rm` therefore installs
+    would terminate the process before the teardown `finally` ran — leaking the
+    runtime. `--rm` therefore installs
    SIGHUP/SIGTERM/SIGINT handlers that run the stop, idempotent with the
    `finally`. `SIGKILL` cannot be intercepted, so a `kill -9`/hard crash can
    still leak; a normal disconnect is SIGHUP and is handled.

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/integration/README.md

@@ -12,8 +12,7 @@ End-to-end tests that run against a **live Colab backend** (unlike the mocked un
 | Directory | What it covers |
 | --- | --- |
 | `repro_plot_redirection/` | `colab exec` of a matplotlib script with `--output-image` redirection. |
-| `repro_keep_alive/` | Fast smoke test (~10s): keep-alive daemon spawns, persists its PID, no errors during the pre-flight ping, `colab stop` reaps it. |
-| `repro_keep_alive_scope/` | Slow soak test (~95s): runs the daemon long enough for one ping past the pre-flight, asserts no `keep_alive_error` events. |
+| `repro_run_command/` | `colab run <script.py>` runs a local script on a fresh VM and cleans up; `--keep` leaves the session alive. |
 | `repro_variable_persistence/` | Variables persist across `colab exec` calls in the same session. |
 | `repro_piped_console/` | Fast smoke test (~5s including session creation): `echo cmd \| colab console -s s` runs the command and exits within 30s. Regression test for the 2026-05-07 EOF-handler fix. |
 | `repro_bundled_oauth/` | Fast smoke test (~5s): verifies that the fallback OAuth configuration is loaded and starts the OAuth flow with the default client ID when local config is missing. |
@@ -22,7 +21,7 @@ End-to-end tests that run against a **live Colab backend** (unlike the mocked un
 
 ## Running
 ```bash
-uv run bash integration/repro_keep_alive/test.sh
+uv run bash integration/repro_run_command/test.sh
 ```
 `uv run` ensures the local `colab` entry point is on `PATH`.
 

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/integration/repro_piped_console/test.sh

@@ -28,7 +28,7 @@ TMP_DIR=$(mktemp -d)
 SESSION_FILE="$TMP_DIR/sessions.json"
 SESSION_NAME="test-piped-console"
 
-# Auth selection (same priority order as repro_keep_alive).
+# Auth selection (OAuth2 if token cached, else ADC).
 if [ -f "$HOME/.config/colab-cli/token.json" ]; then
     AUTH_FLAGS="--auth=oauth2"
 elif command -v gcloud > /dev/null && gcloud auth application-default print-access-token > /dev/null 2>&1; then

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/integration/repro_run_command/test.sh

@@ -23,7 +23,7 @@
 
 # Don't `set -e` so we can capture failures and clean up explicitly.
 
-# ---------- Auth detection (mirrors integration/repro_keep_alive/test.sh) ----
+# ---------- Auth detection ----------------------------------------------------
 if [ -f "$HOME/.config/colab-cli/token.json" ]; then
     AUTH_FLAGS="--auth=oauth2"
 elif command -v gcloud > /dev/null && gcloud auth application-default print-access-token > /dev/null 2>&1; then

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/pyproject.toml

@@ -28,7 +28,7 @@ dependencies = [
     "google-auth>=2.49.1",
     "google-auth-oauthlib>=1.3.0",
     "html2text>=2024.2.26",
-    "jupyter-kernel-client==0.8",
+    "jupyter-kernel-client",
     "nbformat>=5.10.4",
     "packaging>=24.0",
     "prompt-toolkit>=3.0.52",

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/skills/colab-operator/SKILL.md

@@ -20,10 +20,10 @@ by running `uv tool install google-colab-cli` or `pip install google-colab-cli`.
 - Exporting session history as a Jupyter notebook.
 
 ## Mental model (read this first)
-- **A session == a live Jupyter kernel on a rented VM.** `colab new` allocates a billable VM; `colab stop` releases it. Nothing reclaims it automatically except a 24h keep-alive cap, so an unstopped session burns compute units indefinitely.
+- **A session == a live Jupyter kernel on a rented VM.** `colab new` allocates a billable VM; `colab stop` releases it. Sessions stay alive as long as the kernel is active, with the Colab backend automatically maintaining liveness.
 - **Kernel state PERSISTS across `colab exec` / `colab repl` calls in the same session.** Each invocation reattaches to the *same* kernel (the kernel ID is cached in local state) and only closes the websocket on exit — it does **not** shut the kernel down. So imports, variables, and defined functions survive between separate `colab exec` commands. Build up state incrementally; don't re-import everything each call. (`colab stop` and `colab restart-kernel` are what actually reset it.)
 - **Default working directory is `/content`.** Every `exec`/`repl`/`run` `cd`s there first; prefer absolute paths (`/content/...`) for file work. For `colab ls/rm/upload/download`, absolute `/content/...` paths work and the default `ls` path is `content` (VM root).
-- **`colab` is fire-and-forget.** Each command authenticates, does one thing, and exits. A detached background daemon (spawned by `colab new`) handles keep-alive; you don't manage it.
+- **`colab` is fire-and-forget.** Each command authenticates, does one thing, and exits.
 
 ## Authentication (the #1 thing that blocks agents)
 - The global flag is `--auth={adc,oauth2}` and the **default is `adc`** (Application Default Credentials). It must come *before* the subcommand: `colab --auth=adc new -s x`.
@@ -35,10 +35,9 @@ by running `uv tool install google-colab-cli` or `pip install google-colab-cli`.
   https://www.googleapis.com/auth/userinfo.email,\
   https://www.googleapis.com/auth/colaboratory
   ```
-  Why all four: `userinfo.email` (session backend `colab.research.google.com`, else 401), `colaboratory` (RuntimeService `colab.pa.googleapis.com` keep-alive, else 403), `openid`+`cloud-platform` (mandated by gcloud itself; it rejects scope lists missing `cloud-platform`).
+  Why all four: `userinfo.email` (session backend `colab.research.google.com`, else 401), `colaboratory` (Colab features/APIs), `openid`+`cloud-platform` (mandated by gcloud itself; it rejects scope lists missing `cloud-platform`).
 - **oauth2 setup**: `colab --auth=oauth2 <anything>` triggers a browser consent flow on first use (token cached at `~/.config/colab-cli/token.json`). Requires a client config at `~/.colab-cli-oauth-config.json` (or `-c PATH`). The browser step means it usually needs a human; prefer ADC for agents.
-- **Verify auth in one shot**: `colab sessions` (read-only, lists server assignments) or `colab whoami` (hidden debug command: prints the active email, scopes, audience, and expiry). When any call 403s against `colab.pa.googleapis.com`, the cause is almost always a missing scope — `colab whoami` shows it instantly.
-- **`colab new` pre-flights the keep-alive RPC** right after allocating. If your token lacks the `colaboratory` scope it unassigns the fresh VM (so you don't leak a billable assignment) and prints the exact remediation. Follow that message rather than retrying blindly.
+- **Verify auth in one shot**: `colab sessions` (read-only, lists server assignments) or `colab whoami` (hidden debug command: prints the active email, scopes, audience, and expiry). When any call 401s or 403s, the cause is almost always missing credentials or scopes — `colab whoami` shows it instantly.
 - **Do NOT confuse `colab auth` with CLI authentication.** `colab auth` injects *VM-side* GCP credentials into the running kernel (so notebook code can call BigQuery/GCS); it is orthogonal to how the CLI itself authenticates. Never suggest "run `colab auth`" to fix a CLI 401/403 — that's a scope/identity problem fixed via the `gcloud` command above.
 
 ## Workflow
@@ -73,7 +72,7 @@ by running `uv tool install google-colab-cli` or `pip install google-colab-cli`.
 - `colab help` (or `colab help <cmd>`) lists/explains commands; the listing is alphabetical.
 - `colab sessions` lists server-side assignments and auto-prunes stale local entries. Orphans with no local record show as `[?]`.
 - `colab status [-s <name>]` shows hardware, IDLE/BUSY, and last execution.
-- `colab log -s <name> [-n 20] [-t TYPE]` shows recent structured events; invaluable when a task fails (keep-alive errors carry the raw `response_body`).
+- `colab log -s <name> [-n 20] [-t TYPE]` shows recent structured events; invaluable when a task fails.
 - `colab log -s <name> -o summary.ipynb` exports the session as a notebook (also `.md`, `.txt`, `.jsonl` by suffix).
 - `colab url -s <name>` prints a browser URL that attaches the Colab web UI to your existing CLI session instead of allocating a new VM (add `--open` to launch it).
 - `colab skill` / `colab readme` print this skill and the README (handy for self-discovery).
@@ -81,9 +80,8 @@ by running `uv tool install google-colab-cli` or `pip install google-colab-cli`.
 ## Safety
 - **Always `colab stop -s <name>` when done** — idle VMs burn compute units. `colab run` (without `--keep`) self-cleans even if the script errors.
 - Local state lives in `~/.config/colab-cli/sessions.json` (settings in `settings.json`, history in `history/*.jsonl`). Don't edit by hand.
-- **Isolate parallel/agent runs** with the global `--config <path>` flag to point session state at a scratch file (e.g. `colab --config /tmp/agent.json new -s job`). The keep-alive daemon inherits `--auth` and `--config` automatically.
+- **Isolate parallel/agent runs** with the global `--config <path>` flag to point session state at a scratch file (e.g. `colab --config /tmp/agent.json new -s job`).
 
 ## Recovery
 - "Session not found" / 404 / 401 on exec: the backend pruned the VM. `colab exec`/`repl` detect this and clean up local state automatically — run `colab sessions` and re-create with `colab new`.
 - Execution timeout or wedged kernel: `colab restart-kernel -s <name>` (keeps the VM, resets the kernel), or `colab stop` then `colab new`.
-- Keep-alive daemon died (`colab log` shows `keep_alive_stopped reason=consecutive_4xx_errors`): almost always the missing `colaboratory` scope — re-auth per the Authentication section.

{google_colab_cli-0.7.2 → google_colab_cli-0.7.3}/src/colab_cli/auth.py

@@ -157,15 +157,14 @@ def _get_adc_credentials() -> Credentials:
     Honors the standard ADC discovery chain (``GOOGLE_APPLICATION_CREDENTIALS``,
     ``gcloud auth application-default login``, GCE/GKE metadata server, etc.).
 
-    The RuntimeService at colab.pa.googleapis.com requires the
-    `colaboratory` scope (otherwise keep-alive returns 403 SCOPE_NOT_PERMITTED).
-    Most ADC credential types (service accounts, GCE/GKE, impersonated)
+    The RuntimeService at colab.pa.googleapis.com requires the `colaboratory`
+    scope. Most ADC credential types (service accounts, GCE/GKE, impersonated)
     support `with_scopes`; user credentials minted by
     `gcloud auth application-default login` do not. For the latter, the user
-    must re-run `gcloud auth application-default login` with
+    must run `gcloud auth application-default login` with
     `--scopes=openid,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/colaboratory`
     (`openid` and `cloud-platform` are required by `gcloud` itself; `userinfo.email`
-    is required by the session backend; `colaboratory` is required by this RPC).
+    is required by the session backend; `colaboratory` is retained for Colab APIs).
     """
     # `google.auth._default` emits a UserWarning when ADC user credentials
     # don't have a quota project pinned ("Your application has authenticated
@@ -173,13 +172,9 @@ def _get_adc_credentials() -> Credentials:
     # project. You might receive a 'quota exceeded' or 'API not enabled'
     # error.").
     #
-    # That heuristic does not apply to this CLI: every call we make to
-    # `colab.pa.googleapis.com` carries `X-Goog-User-Project: 1014160490159`
-    # (Colab's project id) — see AGENTS.md item 18 — so the user's
-    # quota-project setting is irrelevant. The warning shows up on every
-    # single `colab` invocation under ADC, which is pure noise. Filter it,
-    # but keep the scope as tight as possible: only this exact message,
-    # only during this one call.
+    # That warning shows up on every single `colab` invocation under ADC,
+    # which is pure noise. Filter it, but keep the scope as tight as possible:
+    # only this exact message, only during this one call.
     with warnings.catch_warnings():
         warnings.filterwarnings(
             "ignore",