google-colab-cli 0.7.5__tar.gz → 0.8.0__tar.gz

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/AGENTS.md

@@ -5,7 +5,7 @@
 - **Common**: `common.py` centralizes shared `State` (lazy-loading) and session resolution.
 - **Client**: `ColabClient` handles API interactions (assignment, unassignment).
 - **Auth**: `auth.py` exposes a single `get_credentials(config_path, provider)` facade that dispatches on the `AuthProvider` enum. Two providers are supported, selected via the global `--auth=oauth2|adc` flag (default `oauth2`):
-  - `oauth2`: public `google-auth-oauthlib` `InstalledAppFlow`, token cached at `~/.config/colab-cli/token.json`. Reads the client OAuth config from `-c/--client-oauth-config` (default `~/.colab-cli-oauth-config.json`), falling back to the **bundled** `src/colab_cli/oauth_config.json` resource (re-added in PR #41 / `9f44fe2`, 2026-05-29 — the earlier "removed in `20eb88e`" note was stale/incorrect; the file exists and `auth.py:_get_google_auth_credentials` loads it via `importlib.resources`). As of 2026-06-11 the flow is a **remote copy-paste flow**, not a localhost server: `_run_remote_flow` sets `redirect_uri=https://sdk.cloud.google.com/applicationdefaultauthcode.html` + `token_usage=remote`, prints the URL, and reads the pasted code via `input()`. NEVER revert to OOB (`urn:ietf:wg:oauth:2.0:oob`) — Google blocked it in 2022 ("OOB flow has been blocked"); the `sdk.cloud.google.com` redirect is registered only to the bundled cloud-SDK client (`764086051850-...`), so any other client id gets `redirect_uri_mismatch`. Server-side acceptance/rejection of these variants is verifiable GET-only by building the authorization URL and inspecting whether Google reaches sign-in vs. an OAuth error page (no resources allocated).
+  - `oauth2`: public `google-auth-oauthlib` `InstalledAppFlow`, token cached in the OS keyring via the Python `keyring` library (`service_name="google-colab-cli"`, `username="oauth_token"`), falling back to `_run_remote_flow` if the keyring is not accessible or locked (keyring failures are echoed to stderr; logs alone go to a file; on Linux, `_check_linux_keyring_locked` checks `secretstorage` beforehand and warns immediately when the `SecretService` collection is locked). A pre-keyring `~/.config/colab-cli/token.json` (`LEGACY_TOKEN_PATH`) is migrated into the keyring and deleted only after a successful keyring write, or immediately when the keyring already holds a token. Integration scripts assume no legacy file exists and detect OAuth2 availability via a keyring-only `has_oauth2_token` helper; the migration path is covered by `integration/repro_legacy_token_migration/test.sh` (isolated HOME + file-backed test keyring). Tests that must ignore the host keyring set `PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring`. Reads the client OAuth config from `-c/--client-oauth-config` (default `~/.colab-cli-oauth-config.json`), falling back to the **bundled** `src/colab_cli/oauth_config.json` resource (re-added in PR #41 / `9f44fe2`, 2026-05-29 — the earlier "removed in `20eb88e`" note was stale/incorrect; the file exists and `auth.py:_get_google_auth_credentials` loads it via `importlib.resources`). As of 2026-06-11 the flow is a **remote copy-paste flow**, not a localhost server: `_run_remote_flow` sets `redirect_uri=https://sdk.cloud.google.com/applicationdefaultauthcode.html` + `token_usage=remote`, prints the URL, and reads the pasted code via `input()`. NEVER revert to OOB (`urn:ietf:wg:oauth:2.0:oob`) — Google blocked it in 2022 ("OOB flow has been blocked"); the `sdk.cloud.google.com` redirect is registered only to the bundled cloud-SDK client (`764086051850-...`), so any other client id gets `redirect_uri_mismatch`. Server-side acceptance/rejection of these variants is verifiable GET-only by building the authorization URL and inspecting whether Google reaches sign-in vs. an OAuth error page (no resources allocated).
   - `adc`: Google Application Default Credentials via `google.auth.default()`. The CLI passes `scopes=PUBLIC_SCOPES` (which includes `colaboratory`) and re-applies via `creds.with_scopes()` for credential types that support it. **User credentials minted by `gcloud auth application-default login` ignore the `scopes=` kwarg AND raise `NotImplementedError` on `with_scopes`**: ADC users must explicitly re-authenticate with `gcloud auth application-default login --scopes=openid,https://www.googleapis.com/auth/cloud-platform,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/colaboratory`. `userinfo.email` is required by the session backend at `colab.research.google.com` (assign/unassign/sessions return 401 without it); `colaboratory` is retained for forward compatibility and other Colab features; `openid` and `cloud-platform` are mandated by `gcloud` itself, which rejects scope lists that omit `cloud-platform` with `Invalid value for [--scopes]`. Service-account / GCE / GKE / impersonated creds get the right scopes transparently via `with_scopes`.
 - **Backend Hosts**: Two distinct backends with different requirements:
   - `colab.research.google.com` (session backend / `tun/m/...`): accepts the `userinfo.email` scope. Handles assign, unassign, CCU info, and the contents API.
@@ -37,7 +37,7 @@
 
 ## Subcommand Workflows
 - **Session Management**: `new`, `sessions`, `status`, `stop`.
-- **Execution**: `repl`, `exec`, `console`.
+- **Execution**: `repl`, `exec`, `ssh`.
 - **Files**: `ls`, `rm`, `upload`, `download`, `edit`.
 - **Automation**: `auth`, `drivemount`, `install`, `log`, `pay`, `version`, `update`.
 
@@ -69,9 +69,9 @@ The package version is derived from the git tag via `hatch-vcs` (see `pyproject.
 10. **Live Probes Allocate Real Resources**: Probing the Colab API to debug an issue creates real, billable assignments — every successful POST `/tun/m/assign` reserves a VM. Prefer GET-only (read) probes whenever possible. For any state-mutating call, (a) record every endpoint you create as you go, and (b) clean up via `client.unassign(endpoint)` (or `colab stop`) before declaring the investigation done. Then verify with `colab sessions` that nothing was orphaned.
 11. **Push Freshness**: The remote may have advanced during a session (other contributors land commits while you work). ALWAYS `git fetch <remote>` immediately before pushing or merging. If `git log main..<remote>/main` is non-empty, reset local `main` to the remote, rebase feature branches onto it, retest, then push. NEVER force-push `main` to recover from divergence.
 12. **Amend Safety**: Before `git commit --amend`, explicitly verify all three preconditions: (a) the user requested amend OR a pre-commit hook auto-modified files for an otherwise-successful commit, (b) HEAD was created by you in this conversation (`git log -1 --format='%an %ae'`), (c) the commit has not been pushed to a remote. If any precondition fails, create a new commit instead. NEVER amend a failed/rejected commit — fix the issue and create a new one.
-13. **Run Integration Tests Yourself**: Re-read AGENTS.md "Agent Execution Limitations" before claiming you can't run a test. The CANNOT-run list is exclusively interactive commands (`colab auth`, `colab drivemount`, `colab repl`, `colab console`). Tests built on `colab new` / `colab stop` / `colab log` are non-interactive and you MUST run them yourself before declaring a fix complete. Encoded after running through a full implement-and-document cycle for a fix that the integration test would have falsified in 30 seconds — the cure was `uv run bash integration/repro_run_command/test.sh`.
+13. **Run Integration Tests Yourself**: Re-read AGENTS.md "Agent Execution Limitations" before claiming you can't run a test. The CANNOT-run list is exclusively interactive commands (`colab auth`, `colab drivemount`, unpiped `colab repl`, interactive `colab ssh`). Tests built on `colab new` / `colab stop` / `colab log` are non-interactive and you MUST run them yourself before declaring a fix complete. Encoded after running through a full implement-and-document cycle for a fix that the integration test would have falsified in 30 seconds — the cure was `uv run bash integration/repro_run_command/test.sh`.
 14. **Heed Research Caveats**: When a research subagent surfaces a caveat ("the proto allows it but the policy may reject"), treat it as a TODO to verify, not as a footnote. The validation pattern: shell out to the actual service with the proposed inputs and confirm the response matches expectations BEFORE writing the code that depends on it. For policy-gated paths, run a one-shot probe before committing to a design.
-15. **Two distinct auths — never confuse them**: This codebase has two unrelated authentication concerns: (a) **CLI-to-Colab-control-plane**: how `Client` authenticates HTTP requests to `colab.research.google.com` and `colab.pa.googleapis.com`. Driven by the global `--auth={oauth2,adc}` flag and `auth.py:get_credentials`. The OAuth2 path is bootstrapped automatically by `google_auth_oauthlib`'s `InstalledAppFlow` on first invocation when `~/.config/colab-cli/token.json` doesn't exist; **no separate command is needed** — any `--auth=oauth2` invocation (e.g. `colab --auth=oauth2 sessions`) triggers the browser consent flow. (b) **VM-side credentials**: how the `colab auth` subcommand injects user GCP credentials *into* the running Colab kernel so notebook code (e.g. `gcloud`, BigQuery client) can make authenticated calls from inside the VM. This uses the `USE_AUTH_EPHEM='0'` gcloud-fallback path executed on the kernel via `ColabRuntime`. **NEVER tell a user to "run `colab auth`" as a prerequisite for fixing CLI-side auth issues** — they're orthogonal layers and the suggestion misleads.
+15. **Two distinct auths — never confuse them**: This codebase has two unrelated authentication concerns: (a) **CLI-to-Colab-control-plane**: how `Client` authenticates HTTP requests to `colab.research.google.com` and `colab.pa.googleapis.com`. Driven by the global `--auth={oauth2,adc}` flag and `auth.py:get_credentials`. The OAuth2 path is bootstrapped automatically by `google_auth_oauthlib`'s `InstalledAppFlow` on first invocation when no token is stored in `keyring` (or `keyring` is inaccessible); **no separate command is needed** — any `--auth=oauth2` invocation (e.g. `colab --auth=oauth2 sessions`) triggers the browser consent flow. (b) **VM-side credentials**: how the `colab auth` subcommand injects user GCP credentials *into* the running Colab kernel so notebook code (e.g. `gcloud`, BigQuery client) can make authenticated calls from inside the VM. This uses the `USE_AUTH_EPHEM='0'` gcloud-fallback path executed on the kernel via `ColabRuntime`. **NEVER tell a user to "run `colab auth`" as a prerequisite for fixing CLI-side auth issues** — they're orthogonal layers and the suggestion misleads.
 16. **Pydantic validation requires a schema**: `_issue_request` accepts an optional `schema=` and historically called `TypeAdapter(schema).validate_python(...)` unconditionally. When `schema=None` (a caller that doesn't care about the response body — e.g. unassign or fire-and-forget endpoints that return `[]`), this raises `pydantic.ValidationError: Input should be None`. Always guard with `if schema is None: return` after the empty-body short-circuit.
 17. **Suggest the branch-diff review command after committing**: The user reviews changes with `git diff main..<branch-name>` (full cumulative diff against `main`, not just the latest commit). After landing one or more commits on a feature branch, ALWAYS suggest the exact command — e.g. "Review with `git diff main..sort-help-commands`" — instead of `git show <sha>` (which only shows a single commit and misses context when a branch has multiple commits). Encoded 2026-05-05 after suggesting `git show 9d9c7da` for a branch the user wanted to review holistically.
 18. **Verify research-tool claims with primary sources**: Research tools and AI assistants can be confidently wrong, especially about edge cases or features outside their training corpus. When such a tool says "X is not used / not parsed / doesn't exist", treat it as a hypothesis to verify, not a fact. Always cross-check against the primary source (the actual code or config) — and when a tool names files, check whether the indirection chain it describes actually exists. The cost of believing the tool when it's wrong is shipping a non-functional feature; the cost of double-checking is small. Encoded 2026-05-05 after a `colab url` first-cut shipped the wrong URL format because of unverified output.
@@ -83,11 +83,12 @@ As an AI agent operating via non-interactive shell tools (`run_shell_command`),
 - **I CAN Run:**
   - Automated tests (`pytest`), linting (`ruff`), and headless execution scripts.
   - Subcommands that don't pause for user input (e.g., `colab new`, `colab status`, `colab stop`, `colab ls`, `colab install`, `colab exec <file.py>`).
-  - **Piped `colab repl` and `colab console`**: as of 2026-05-07 both commands support piped stdin and exit cleanly on EOF (`echo 'cmd' | colab console -s s` returns in ~1.2s). The unpiped, interactive variants still cannot be run autonomously.
+  - **Piped `colab repl`**: as of 2026-05-07 it supports piped stdin and exits cleanly on EOF (`echo 'print(1)' | colab repl -s s`). The unpiped, interactive variant still cannot be run autonomously.
+  - **One-shot shell commands**: `colab ssh -s s -- cmd` runs `cmd` non-interactively and exits with its status (add `</dev/null` if my shell tool runs commands in a terminal); where ssh can't work, use `echo '!cmd' | colab exec -s s` (each `!` line is its own shell and IPython expands `{expr}` and `$name` in it).
   - Specially crafted mock scripts that simulate timeouts or API calls.
 - **I CANNOT Run (Requires User Assistance):**
   - **`colab auth`**: This command relies on the traditional Gcloud fallback `input_request` (via `USE_AUTH_EPHEM='0'`), which prompts the user via Python's `input()` to click a URL, sign in, and paste back an authorization code. My shell tool will hang indefinitely on this `input()`.
   - **`colab drivemount`**: This command prompts the user via `sys.stdin.readline()` (specifically querying `/dev/tty` to ensure input is captured) to press `Enter` after granting OAuth consent in the browser. My shell tool will timeout/hang waiting for `Enter`.
-  - **Interactive (TTY) `colab repl` / `colab console`**: When stdin is a real terminal these commands drop into interactive raw-TTY modes that require real-time keystroke streaming. My shell tools cannot support this. (Piped stdin is fine — see above.)
+  - **Interactive (TTY) `colab repl` / `colab ssh`**: When stdin is a real terminal these commands open interactive sessions that require real-time keystroke streaming. My shell tools cannot support this. (Piped stdin to `colab repl` or `colab exec` is fine — see above.)
 
 Whenever working on interactive commands, I must build the core logic, write mock tests, and explicitly ask the user to run the live test in their terminal to verify success.

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/CHANGELOG.md

@@ -8,6 +8,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 The package version is derived from the git tag via `hatch-vcs`; each release
 below corresponds to a tag of the same name.
 
+## [Unreleased]
+
+### Changed
+
+- **ssh:** Every `colab ssh` WebSocket now sends `X-Colab-Client-Agent: colab-cli` and the query values `colab-client-agent=colab-cli` and `colab-client-agent=ssh-mode-<mode>`, where the mode is `interactive` for plain `colab ssh`, `cmd` for `colab ssh -- CMD`, and `proxy` otherwise.
+- **ssh:** When the server rejects a connection with HTTP 429 and reports its
+  SSH session limit, `colab ssh` now says the limit is reached (N of N sessions
+  in use on the runtime) and that the limit depends on the Colab plan. A 429
+  without a usable limit keeps the previous message.
+
+### Added
+
+- **ssh:** `colab ssh [-s NAME] -- CMD` runs one command on an existing runtime and exits with its status. Output is byte-exact when piped (a PTY only when stdin and stdout are terminals), and it never creates a runtime. Its handshake is tagged `ssh-mode-cmd`.
+
+### Removed
+
+- **console:** **BREAKING:** Remove the `colab console` subcommand. Use
+  `colab ssh -s NAME` for an interactive shell on the runtime and
+  `colab ssh -s NAME -- CMD` for one-shot commands; where ssh can't work, pipe
+  `!`-prefixed shell commands to `colab exec -s NAME`. Each `!` line is its own
+  shell and IPython expands `{expr}` and `$name` in it, so start a multi-line
+  script with `%%bash` and keep untrusted text out of `!` lines. Breaking, so
+  it ships in a new major version.
+
+### Fixed
+
+- **ssh:** `colab --config FILE ssh` and `colab --auth adc ssh` now keep those options in the ProxyCommand that `colab ssh` starts, so it uses the same sessions file and auth provider instead of the defaults.
+- **security:** Create the debug log (`~/.config/colab-cli/colab.log`),
+  `sessions.json`, `settings.json` and session history files readable only by
+  their owner, and tighten copies that older versions left readable by others
+  the next time the CLI uses them.
+- **security:** Stop writing access tokens, runtime proxy tokens, OAuth codes,
+  client secrets, auth headers and cookies to the debug log. Logs written by
+  older versions may contain credentials and can be deleted; if other users
+  could read an old log, revoke the CLI's access in your Google Account and
+  sign in again.
+
 ## [0.7.1] - 2026-09-14
 
 ### Fixed

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: google-colab-cli
-Version: 0.7.5
+Version: 0.8.0
 Summary: CLI for interacting with Colab.
 Project-URL: Homepage, https://github.com/googlecolab/google-colab-cli
 Project-URL: Repository, https://github.com/googlecolab/google-colab-cli
@@ -22,6 +22,7 @@ Requires-Dist: google-auth-oauthlib>=1.3.0
 Requires-Dist: google-auth>=2.49.1
 Requires-Dist: html2text>=2024.2.26
 Requires-Dist: jupyter-kernel-client==0.9.0
+Requires-Dist: keyring>=25.7.0
 Requires-Dist: nbformat>=5.10.4
 Requires-Dist: packaging>=24.0
 Requires-Dist: prompt-toolkit>=3.0.52
@@ -53,7 +54,7 @@ Designed to support seamless developer productivity, headless automation, and AI
 ## Key Features
 
 * **Instant VM Provisioning:** Spin up CPU, GPU (T4, L4, G4, H100, A100), or TPU (v5e1, v6e1) runtimes in seconds.
-* **Robust Code Execution:** Run local Python scripts, Jupyter Notebooks (`.ipynb`), or piped `stdin` code; launch interactive REPLs or raw TTY console shells.
+* **Robust Code Execution:** Run local Python scripts, Jupyter Notebooks (`.ipynb`), or piped `stdin` code; launch interactive REPLs or SSH shells (`colab ssh`).
 * **Ephemeral Job Runner (`colab run`):** Provision a fresh VM, execute a local script with forwarded arguments, retrieve output files, and automatically tear down the runtime in a single command.
 * **Activity-Based Liveness:** Sessions stay alive as long as the kernel is active, with the Colab backend automatically maintaining liveness.
 * **Seamless Workspace Automation:** Mount Google Drive, authenticate Google Cloud Platform (GCP) credentials, and install dependencies with high-performance `uv` package management.
@@ -117,8 +118,7 @@ Run `colab <command> --help` to view specific options, defaults, and detailed he
 | `colab run [--gpu GPU] [--tpu TPU] [--high-mem] [--keep] SCRIPT [ARGS...]` | Run a local script on a fresh VM, forwarding arguments, then release it |
 | `colab exec [-s NAME] [-f FILE] [--output-image PATH]` | Execute Python code from stdin, a local `.py` file, or a `.ipynb` notebook |
 | `colab repl [-s NAME] [--output-image PATH]` | Start an interactive Python REPL on the VM (exits cleanly on piped EOF) |
-| `colab console [-s NAME]` | Connect to a raw interactive TTY shell (tmux) on the remote VM |
-| `colab ssh [-s NAME] [--proxy-mode] [-i KEY] [--gpu GPU] [--tpu TPU] [--high-mem]` | Open an SSH shell to the runtime over WebSocket, or act as an OpenSSH `ProxyCommand` bridge for IDE remote-dev |
+| `colab ssh [-s NAME] [--proxy-mode] [-i KEY] [--gpu GPU] [--tpu TPU] [--high-mem] [-- CMD ...]` | Open an SSH shell to the runtime over WebSocket, run a one-shot remote command (`-- CMD`), or act as an OpenSSH `ProxyCommand` bridge for IDE remote-dev |
 
 ### File Operations
 | Command | Description |
@@ -180,7 +180,7 @@ colab stop -s analysis
 ## Usage Notes
 
 * **Machine shape:** Use `--high-mem` with `colab new`, `colab run`, or `colab ssh` (when auto-creating a runtime) to request a high-RAM machine shape. Requires Colab Pro or Pro+ entitlement for supported accelerators (CPU, T4, A100, etc.). L4 and TPU runtimes ignore this flag because they only offer one shape. Machine shape is shown in `colab sessions` and `colab status`.
-* **TTY Requirements:** The interactive commands `repl` and `console` require a local TTY. When running inside automated scripts or pipelines, make sure to pipe stdin (e.g., `echo "print(1)" | colab repl`) to trigger non-interactive execution modes.
+* **TTY Requirements:** `colab repl` and interactive `colab ssh` expect a local TTY. When running inside automated scripts or pipelines, pipe stdin to `colab repl` (e.g., `echo "print(1)" | colab repl`) to trigger its non-interactive mode, or run one-shot shell commands with `colab ssh -s NAME -- CMD` (e.g., `colab ssh -s NAME -- ls /content`); pass untrusted values as separate arguments (after `--` where the program accepts it), since a single argument is a remote shell line. `colab ssh` needs a local `ssh` client and key, so where it can't work (no client or key, a tool not on the minimal ssh environment's `PATH`, or the runtime's concurrent SSH connections in use), pipe the commands to `colab exec` with IPython's `!` prefix (e.g., `echo '!ls /content' | colab exec`); each `!` line is its own shell and IPython expands `{expr}` and `$name` in it, so start a multi-line script with `%%bash` and keep untrusted text out of `!` lines.
 * **Transparent Code Execution:** When calling `colab exec -f file.py`, the CLI reads the file locally and transmits its content to the remote kernel. You do not need to manually upload files before execution.
 * **Storage & State Paths:** Session tokens and metadata are stored at `~/.config/colab-cli/sessions.json`. Global CLI settings are located at `~/.config/colab-cli/settings.json`. These can be customized or isolated via the global `--config` flag.
 

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/README.md

@@ -17,7 +17,7 @@ Designed to support seamless developer productivity, headless automation, and AI
 ## Key Features
 
 * **Instant VM Provisioning:** Spin up CPU, GPU (T4, L4, G4, H100, A100), or TPU (v5e1, v6e1) runtimes in seconds.
-* **Robust Code Execution:** Run local Python scripts, Jupyter Notebooks (`.ipynb`), or piped `stdin` code; launch interactive REPLs or raw TTY console shells.
+* **Robust Code Execution:** Run local Python scripts, Jupyter Notebooks (`.ipynb`), or piped `stdin` code; launch interactive REPLs or SSH shells (`colab ssh`).
 * **Ephemeral Job Runner (`colab run`):** Provision a fresh VM, execute a local script with forwarded arguments, retrieve output files, and automatically tear down the runtime in a single command.
 * **Activity-Based Liveness:** Sessions stay alive as long as the kernel is active, with the Colab backend automatically maintaining liveness.
 * **Seamless Workspace Automation:** Mount Google Drive, authenticate Google Cloud Platform (GCP) credentials, and install dependencies with high-performance `uv` package management.
@@ -81,8 +81,7 @@ Run `colab <command> --help` to view specific options, defaults, and detailed he
 | `colab run [--gpu GPU] [--tpu TPU] [--high-mem] [--keep] SCRIPT [ARGS...]` | Run a local script on a fresh VM, forwarding arguments, then release it |
 | `colab exec [-s NAME] [-f FILE] [--output-image PATH]` | Execute Python code from stdin, a local `.py` file, or a `.ipynb` notebook |
 | `colab repl [-s NAME] [--output-image PATH]` | Start an interactive Python REPL on the VM (exits cleanly on piped EOF) |
-| `colab console [-s NAME]` | Connect to a raw interactive TTY shell (tmux) on the remote VM |
-| `colab ssh [-s NAME] [--proxy-mode] [-i KEY] [--gpu GPU] [--tpu TPU] [--high-mem]` | Open an SSH shell to the runtime over WebSocket, or act as an OpenSSH `ProxyCommand` bridge for IDE remote-dev |
+| `colab ssh [-s NAME] [--proxy-mode] [-i KEY] [--gpu GPU] [--tpu TPU] [--high-mem] [-- CMD ...]` | Open an SSH shell to the runtime over WebSocket, run a one-shot remote command (`-- CMD`), or act as an OpenSSH `ProxyCommand` bridge for IDE remote-dev |
 
 ### File Operations
 | Command | Description |
@@ -144,7 +143,7 @@ colab stop -s analysis
 ## Usage Notes
 
 * **Machine shape:** Use `--high-mem` with `colab new`, `colab run`, or `colab ssh` (when auto-creating a runtime) to request a high-RAM machine shape. Requires Colab Pro or Pro+ entitlement for supported accelerators (CPU, T4, A100, etc.). L4 and TPU runtimes ignore this flag because they only offer one shape. Machine shape is shown in `colab sessions` and `colab status`.
-* **TTY Requirements:** The interactive commands `repl` and `console` require a local TTY. When running inside automated scripts or pipelines, make sure to pipe stdin (e.g., `echo "print(1)" | colab repl`) to trigger non-interactive execution modes.
+* **TTY Requirements:** `colab repl` and interactive `colab ssh` expect a local TTY. When running inside automated scripts or pipelines, pipe stdin to `colab repl` (e.g., `echo "print(1)" | colab repl`) to trigger its non-interactive mode, or run one-shot shell commands with `colab ssh -s NAME -- CMD` (e.g., `colab ssh -s NAME -- ls /content`); pass untrusted values as separate arguments (after `--` where the program accepts it), since a single argument is a remote shell line. `colab ssh` needs a local `ssh` client and key, so where it can't work (no client or key, a tool not on the minimal ssh environment's `PATH`, or the runtime's concurrent SSH connections in use), pipe the commands to `colab exec` with IPython's `!` prefix (e.g., `echo '!ls /content' | colab exec`); each `!` line is its own shell and IPython expands `{expr}` and `$name` in it, so start a multi-line script with `%%bash` and keep untrusted text out of `!` lines.
 * **Transparent Code Execution:** When calling `colab exec -f file.py`, the CLI reads the file locally and transmits its content to the remote kernel. You do not need to manually upload files before execution.
 * **Storage & State Paths:** Session tokens and metadata are stored at `~/.config/colab-cli/sessions.json`. Global CLI settings are located at `~/.config/colab-cli/settings.json`. These can be customized or isolated via the global `--config` flag.
 

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/docs/01_session_management.md

@@ -1,5 +1,11 @@
 ---
 log:
+2026-11-04: `--gpu` and `--tpu` now reject unknown values instead of silently selecting `A100` (GPU) or `v6e1` (TPU). `resolve_runtime_options()` uses the new `GPU_ACCELERATORS` / `TPU_ACCELERATORS` maps and raises a `typer.BadParameter` that names the flag and the supported list; `--help` is generated from the same maps so it cannot drift. Validation runs before any assignment request, in both `colab new` and `colab run`.
+2026-11-02: Added `_check_linux_keyring_locked` in `auth.py` to check if the active Linux `SecretService` keyring collection is locked via `secretstorage` before reading or writing OAuth tokens, logging and printing an immediate warning with the collection label, timeout, and `--auth=adc` hint while still attempting the keyring operation.
+2026-10-29: Removed the `colab console` subcommand; the long-lived connections that skip mid-session token refresh are now `repl` and `ssh`.
+2026-10-29: The debug log (`colab.log` and `--logtostderr`) masks access tokens, runtime proxy tokens, OAuth codes, client secrets, auth headers and cookies, including in percent-encoded URLs; `jupyter_kernel_client` logs at INFO so kernel stdin replies are not recorded.
+2026-10-29: `sessions.json`, `settings.json`, `colab.log` and session history files are created with mode 0600, and existing copies are tightened to 0600 the next time the CLI uses them (history files whenever the history logger starts).
+2026-10-21: Stored OAuth2 credentials in the OS native keyring via the `keyring` library instead of `~/.config/colab-cli/token.json`, falling back to `_run_remote_flow` if keyring is inaccessible. Keyring failures are reported on stderr, and an existing `token.json` is migrated into the keyring and then deleted (or deleted outright if the keyring already has a token).
 2026-10-14: Added configurable HTTP request timeouts and bounded allocation retries. Uncertain allocation POSTs are reconciled by GET using the same notebook hash before resubmission.
 2026-09-25: Runtime proxy tokens are now refreshed before expiry. `SessionState` persists `token_expires_at`, and `State.get_session` re-fetches the token via `GET /tun/m/assignments` when it's within 5 minutes of expiring.
 2026-09-25: Removed the keep-alive background daemon and related client/CLI ping functions. VM liveness is automatically maintained by the Colab backend based on kernel activity.
@@ -30,6 +36,7 @@ Defines the specific hardware model.
 - **GPU Accelerators**:
     - `T4`: NVIDIA T4 (standard free-tier GPU).
     - `L4`: NVIDIA L4 (cost-effective modern GPU).
+    - `G4`: NVIDIA G4 (entry-level datacenter GPU).
     - `A100`: NVIDIA A100 (high-performance GPU).
     - `H100`: NVIDIA H100 (latest-gen performance GPU).
 - **TPU Accelerators**:
@@ -52,6 +59,12 @@ The CLI maps user flags to these backend parameters:
 - `colab new my-session --high-mem` -> adds `shape=hm` (when supported)
 - `colab new my-session --gpu A100 --high-mem` -> `variant=GPU`, `accelerator=A100`, `shape=hm`
 
+`--gpu` accepts `T4`, `L4`, `G4`, `H100`, and `A100`; `--tpu` accepts `v5e1`
+and `v6e1` (case-insensitive). Anything else is rejected with the supported
+list before a VM is requested. Unknown values used to fall back silently to
+`A100`/`v6e1`, which turned a typo in an automation script into a real,
+billable allocation.
+
 ## Approach
 
 ### HTTP Timeout and Allocation Recovery
@@ -86,7 +99,7 @@ end to end; mocked timeouts alone do not establish backend compatibility.
     - `nsa`: 1 (Standard flag observed in browser traces, typically for "next-gen session architecture").
     - `variant`: Selected from the list above.
     - `accelerator`: Selected from the list above.
-- **State Persistence**: The response contains a `token` and potentially a backend URL or identifier. We will store this in a local JSON file (default `~/.config/colab-cli/sessions.json`).
+- **State Persistence**: The response contains a `token` and potentially a backend URL or identifier. We will store this in a local JSON file (default `~/.config/colab-cli/sessions.json`), readable only by its owner (mode 0600).
     - Format: `{ "session_name": { "token": "...", "backend_url": "...", "hardware": "..." } }`
 
 ### 2. Session Status (`colab status`)
@@ -111,7 +124,7 @@ end to end; mocked timeouts alone do not establish backend compatibility.
 VM liveness is automatically maintained by the Colab backend based on active kernel executions and interactions. The CLI runs no keep-alive daemon. A session persists until explicitly terminated via `colab stop` or until the backend reclaims it due to inactivity.
 
 ### 6. Runtime Proxy Token Refresh
-The runtime proxy token (`runtimeProxyInfo.token`) authenticates every request to the VM and expires after `tokenExpiresInSeconds` (1 hour). `SessionState.token_expires_at` stores the absolute expiry, computed when the token is received. Commands that talk to the VM load their session through `State.get_session(name)`, which, when the token is missing an expiry or is within 5 minutes of it, calls `GET /tun/m/assignments`, takes the fresh token/URL for the matching endpoint, and persists it. If the endpoint is no longer assigned, the session is pruned. `sync_sessions` also writes back the fresh tokens it already fetches. Long-lived connections (`repl`, `console`, `ssh`) only present the token on initial connect and aren't refreshed mid-session.
+The runtime proxy token (`runtimeProxyInfo.token`) authenticates every request to the VM and expires after `tokenExpiresInSeconds` (1 hour). `SessionState.token_expires_at` stores the absolute expiry, computed when the token is received. Commands that talk to the VM load their session through `State.get_session(name)`, which, when the token is missing an expiry or is within 5 minutes of it, calls `GET /tun/m/assignments`, takes the fresh token/URL for the matching endpoint, and persists it. If the endpoint is no longer assigned, the session is pruned. `sync_sessions` also writes back the fresh tokens it already fetches. Long-lived connections (`repl`, `ssh`) only present the token on initial connect and aren't refreshed mid-session.
 
 ## TODO / Future Work
 - **Backend Sync**: Implement a way to reconcile the local `sessions.json` with the output of `colab sessions`.
@@ -122,7 +135,7 @@ The runtime proxy token (`runtimeProxyInfo.token`) authenticates every request t
 - **Global Flags**:
     - `-c`, `--client-oauth-config`: Path to the client secrets JSON file (default: `~/.colab-cli-oauth-config.json`).
     - `--config`: Path to the session state JSON file (default: `~/.config/colab-cli/sessions.json`).
-- **Token Storage**: Credentials are persisted to `~/.config/colab-cli/token.json` after the initial flow.
+- **Token Storage**: Credentials are persisted to the OS native keyring via the `keyring` library (`service_name="google-colab-cli"`, `username="oauth_token"`) after the initial flow. If the keyring is inaccessible or locked, `_run_remote_flow` is invoked for the current run and a warning is printed to stderr. A legacy `~/.config/colab-cli/token.json` is migrated into the keyring on first use and deleted after a successful keyring write, or immediately if the keyring already holds a token.
 - Use `requests` for robust HTTP interactions and `pydantic` for schema validation.
 - Handle authentication headers (likely `Authorization: Bearer <token>` or cookies).
 

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/docs/02_execution_and_interactive.md

@@ -5,9 +5,13 @@ log:
 2026-05-07: Fixed `print_kitty` (used by `colab exec --output-image` and any image-producing exec) to no-op when `sys.stdout.isatty()` is false. The Kitty Graphics Protocol escape sequence is meaningless when stdout is a file or pipe and was visually corrupting captured output (a multi-KB base64 PNG blob would land in log files, grep targets, or showboat captures). Image bytes are still saved to disk via `handle_image`'s file-write path; only the inline-render attempt is suppressed.
 
 2026-06-04: Bumped the default `--timeout` for `colab exec` from 10s to 30s (and the matching `colab run` default) so brief silent tasks are less likely to hit a premature `TimeoutError`. Explicit `--timeout` overrides are unaffected.
+
+2026-10-24: `--timeout` for `colab exec` (and `colab run`) is now an idle limit: execution fails only after that many seconds without output, so code that keeps printing is never cut off, and `--timeout 0` means no limit. On 0.8.0/0.9.0 the old deadline was never enforced, so code that is silent for longer than `--timeout` now fails where it used to complete. `colab repl` and `colab install`, which pass no timeout, wait until execution finishes, or until the kernel dies or restarts. None of them busy-loop while waiting. `colab auth` and `colab drivemount` are unchanged. See #183.
+
+2026-10-29: Removed the `colab console` subcommand and its `/colab/tty` transport (`console.py`, its tests, and `integration/repro_piped_console/`). Use `colab ssh -s NAME` for an interactive shell and `colab ssh -s NAME -- CMD` for one-shot commands, or pipe `!`-prefixed shell commands to `colab exec` where ssh can't work.
 ---
 
-# Design: Execution and Interactive Interaction (`repl`, `exec`, `console`)
+# Design: Execution and Interactive Interaction (`repl`, `exec`)
 
 ## Overview
 Execution involves sending Python code (or shell commands) to the Jupyter kernel running on the Colab VM and processing the stream of output messages.
@@ -28,13 +32,7 @@ Execution involves sending Python code (or shell commands) to the Jupyter kernel
     - If file path is local: Read content, send as code.
     - If file path is remote: Execute `!python <path>`.
 - **Multi-Modal Output**: Handle `display_data` messages (e.g., `image/png`, `text/html`). For the CLI, we'll save images to temporary files and print their paths, or if the terminal supports it (e.g., iTerm2), inline them.
-- **Timeout Configuration**: Exposes a `--timeout` flag (default 30s) to allow long-running silent tasks (like model compilation or data downloading) to execute without being prematurely killed.
-
-### 3. Console (`colab console`)
-- **Implementation**: Connects directly to the backend terminal endpoint (`/colab/tty`) via WebSockets using `websocket-client`.
-- **Interactive**: Bypasses the Jupyter kernel entirely to provide a raw, PTY-backed bash session on the Colab VM.
-- **Terminal Management**: Configures `sys.stdin` to raw mode using `termios` and `tty`, passing single characters to the socket and writing raw ANSI escape sequences directly to `sys.stdout.buffer`. Hooks into `SIGWINCH` to communicate local terminal dimensions (`cols`/`rows`) to the remote bash environment so output rendering works perfectly during resizing.
-- **Piped stdin**: Detected via `sys.stdin.isatty()`. When piped, the input characters are forwarded one at a time to the remote pty, and on EOF the client sends `exit\n` and then closes the websocket itself after `PIPED_EOF_GRACE_SECONDS` (0.5s) so the user's shell goodbye text drains back. The remote `/colab/tty` endpoint wraps bash in tmux, which intercepts a bare `\x04` as a literal character — that is why we send `exit\n` rather than Ctrl-D.
+- **Timeout Configuration**: Exposes a `--timeout` flag (default 30s): the longest silence allowed before execution fails. Raise it so long-running silent tasks (like model compilation or data downloading) are not prematurely killed; `0` means no limit.
 
 ## Implementation Details
 - **Kernel Management**: `ColabRuntime` (from `colab-agent`) already handles message signing and message types.
@@ -52,6 +50,4 @@ TDD is mandatory for all execution features.
 ### 2. TTY and Piping
 - **Test Case**: Mock `sys.stdin.isatty()` to verify `colab repl` correctly switches between interactive mode and one-shot piped execution.
 - **Test Case**: Verify large piped inputs are handled without buffer overflow or truncation.
-- **Test Case**: `colab console` with piped stdin sends `exit\n` and calls `ws.close()` on EOF (regression: previously sent `\x04` only and hung).
-- **Test Case**: `colab console` in TTY mode does not synthesize an exit on EOF (the user owns the session lifecycle).
 - **Test Case**: `print_kitty` is a no-op when `sys.stdout.isatty()` is false (regression: previously emitted ANSI/base64 into pipes and files).

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/docs/03_file_management.md

@@ -31,7 +31,7 @@ File management on the Colab VM will be implemented using the Jupyter Contents A
 ### 3. Downloading Files (`colab download`)
 - **API**: `GET /api/contents/<remote_path>?content=1` (as seen in HAR).
 - **Response**: JSON with `content` field.
-- **Handling**: Decodes content based on `format` (text or base64) and saves it locally.
+- **Handling**: Decodes content based on `format` (`base64`, `json`, or plain `text`) and saves it locally.
 
 ### 4. Deleting Files (`colab rm`)
 - **API**: `DELETE /api/contents/<remote_path>`.
@@ -59,4 +59,4 @@ TDD is mandatory for all file management features.
 
 ### 2. Error Cases
 - **Test Case**: Verify 404 responses are correctly caught and presented as a "File not found" error to the user.
-- **Test Case**: Verify correct handling of large file uploads exceeding API limits via kernel streaming.
+- **Test Case**: Verify correct handling of large file uploads exceeding API limits via kernel streaming.

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/docs/04_automation_and_utility.md

@@ -1,5 +1,10 @@
 ---
 log:
+2026-11-07: Optimized `_get_adc_credentials` in `auth.py` to stub `google.auth._cloud_sdk.get_project_id` around `google.auth.default()`. When ADC user credentials lack a `quota_project_id`, `google-auth` otherwise spawns `gcloud config get-value project` on every CLI invocation to resolve a `project_id` that `_get_adc_credentials` immediately discards (`_`).
+2026-11-04: `colab drivemount` now opens the Drive consent URL in the system browser (`utils.open_url_in_browser`, also used by `colab pay`/`colab url --open`) instead of only printing it. The URL is still printed first, and a missing browser opener (SSH, containers, CI) is treated as normal: the flow keeps waiting for the user exactly as before. The prompt moved into `commands/automation.py::show_drive_auth_url` so it can be unit-tested.
+2026-11-02: Added `_check_linux_keyring_locked` in `auth.py` to inspect the active Linux `SecretService` collection via `secretstorage` before `get_password`/`set_password`, warning immediately when the keyring is locked so users know why the CLI is waiting on an unlock prompt (and suggesting `--auth=adc`).
+2026-10-29: Session history files are created readable only by the owner (0600), and existing ones are tightened to 0600 whenever the history logger starts.
+2026-10-21: Migrated OAuth2 token persistence in `auth.py` from `~/.config/colab-cli/token.json` to the OS native password storage via the `keyring` library (`KEYRING_SERVICE_NAME = "google-colab-cli"`, `KEYRING_USERNAME = "oauth_token"`). When the keyring is empty, locked, or inaccessible, `_run_remote_flow` is invoked; keyring read/write failures print a warning to stderr without failing the command. An existing `~/.config/colab-cli/token.json` (`LEGACY_TOKEN_PATH`) is used when the keyring has no token, copied into the keyring, and deleted only after the keyring write succeeds; it is also deleted when the keyring already holds a token.
 2026-09-25: Cleaned up references to keep-alive background daemon and pre-flight pings. VM liveness is automatically maintained by the Colab backend based on active kernel activity.
 2026-08-10: Added `colab usage` for account-level compute-unit rate/balance via `GET /tun/m/ccu-info` on the session backend (same bearer token as `colab new`).
 2026-06-11: Replaced the `oauth2` provider's `run_local_server()` (localhost redirect) with a remote copy-paste flow (`_run_remote_flow` in `auth.py`). The CLI now prints an authorization URL built with `redirect_uri=https://sdk.cloud.google.com/applicationdefaultauthcode.html` and `token_usage=remote`, then reads the pasted authorization code via `input()` and exchanges it with `flow.fetch_token(code=...)`. This is the same flow `gcloud auth application-default login` uses and works identically in local and remote/headless/container environments, removing the heuristic of whether to auto-open a browser. Confirmed server-side acceptance with a live GET-only check against the bundled cloud-SDK client (`764086051850-...`); the OOB redirect and a non-bundled client id were both verified to be rejected (`OOB flow has been blocked` / `redirect_uri_mismatch`). Unit tests in `tests/test_auth.py` assert no localhost server is started, the redirect URI + `token_usage=remote` are set, and the pasted code is exchanged.
@@ -32,7 +37,18 @@ backend, selected via the global `--auth=<provider>` flag:
     `token_usage=remote`) using the registered HTTPS landing page
     `https://sdk.cloud.google.com/applicationdefaultauthcode.html`; the user
     signs in, copies the code Google displays, and pastes it back at the
-    prompt. The refresh token is cached at `~/.config/colab-cli/token.json`.
+    prompt. The refresh token is cached in the OS native keyring via the
+    `keyring` library (`service_name="google-colab-cli"`, `username="oauth_token"`),
+    falling back to running `_run_remote_flow` if the keyring is not accessible
+    or locked. On Linux, `_check_linux_keyring_locked` inspects the active
+    `SecretService` collection via `secretstorage` before reading or writing
+    and prints an immediate warning if the collection is locked while waiting
+    on the unlock prompt. Keyring read/write failures are printed to stderr, since a
+    failed write means the user is prompted again on every command. A token
+    left at the pre-keyring path `~/.config/colab-cli/token.json` is migrated
+    into the keyring on first use and the file is deleted once the keyring
+    write succeeds (it is kept if the write fails). If the keyring already
+    holds a token, the legacy file is deleted.
     This is the same mechanism `gcloud auth application-default login` uses,
     and it behaves identically on local, remote, headless, and container
     hosts (no auto-opened browser, no bound port). We deliberately do **not**
@@ -65,9 +81,9 @@ sufficient for this host.
 How each provider supplies the scope:
 
 -   **`oauth2`**: `PUBLIC_SCOPES` already includes `colaboratory`, so the
-    InstalledAppFlow consent screen lists it. Existing cached tokens at
-    `~/.config/colab-cli/token.json` that were minted before this change must
-    be deleted to trigger a fresh consent flow.
+    InstalledAppFlow consent screen lists it. Existing cached tokens in the
+    keyring that were minted before a scope change must be cleared to trigger a
+    fresh consent flow.
 -   **`adc`**: `google.auth.default(scopes=PUBLIC_SCOPES)` is called, and for
     credential subclasses that support `with_scopes` (service accounts,
     GCE/GKE metadata, impersonated) we re-apply via `creds.with_scopes(...)`.
@@ -123,7 +139,10 @@ How each provider supplies the scope:
     automatically interacts with the Colab backend
     (`/tun/m/credentials-propagation/`), prompts the user with the Google OAuth
     consent URL if needed, and dispatches the required `colab_reply` message to
-    the `stdin` channel to unlock the kernel thread.
+    the `stdin` channel to unlock the kernel thread. The consent URL is also
+    passed to `webbrowser.open` (see `utils.open_url_in_browser`); when no
+    browser opener is available (SSH, containers, CI) the URL stays on screen
+    and the flow continues to wait for the user, unchanged.
 -   **Timeout**: The kernel is silent (no iopub traffic) the entire time the
     user is OAuthing in their browser. To avoid the upstream 10s
     `jupyter_kernel_client` default raising `TimeoutError` mid-flow, this
@@ -135,7 +154,8 @@ How each provider supplies the scope:
 -   **Action**: Capture the session's command history and outputs.
 -   **Storage**: Maintain a local JSON-L file of all major operations,
     executions, and stdin interactions in
-    `~/.config/colab-cli/history/<session_name>.jsonl`.
+    `~/.config/colab-cli/history/<session_name>.jsonl`, readable only by the
+    owner (mode 0600).
 -   **Viewing**: `colab log list` and `colab log show <session>`.
 -   **Conversion (Planned)**: Future expansion to convert history logs to
     `.ipynb` or `.html`.

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/docs/05_run_command.md

@@ -5,6 +5,7 @@ log:
 2026-05-12: Initial design and implementation of `colab run <script.py> [args...]`. Combines `colab new` + `colab exec` + `colab stop` into a single fire-and-forget invocation so a Python file can use `#!/usr/bin/env -S colab run` as a shebang line and execute on a freshly-allocated Colab VM. Adds `--keep` (skip auto-stop), `--gpu` / `--tpu` (passthrough to session creation), `-s/--session` (name the ephemeral session), and propagates the script's exit status (non-zero on any uncaught exception in the kernel). The script's `sys.argv` is re-set inside the kernel to mirror native `python script.py arg1 arg2` semantics, and `__name__` is set to `"__main__"`.
 2026-05-12: Native CPython exit-code semantics for `sys.exit()` / `raise SystemExit(...)` from the script body. The Colab kernel reports a `SystemExit` as `output_type=='error'`, which under the previous logic would have (a) printed the IPython traceback (`An exception has occurred, use %tb...`) and (b) flagged the run as a failure regardless of the integer exit code. Now: `sys.exit()` / `sys.exit(0)` exit 0 silently; `sys.exit(N)` exits N; `sys.exit('msg')` exits 1 (matching CPython). The IPython "To exit: use 'exit', 'quit', or Ctrl-D." UserWarning is filtered via the prelude. Encoded after running `examples/gpu_hello.py` end-to-end and seeing the noisy `SystemExit: 0` traceback at the end of an otherwise-successful GPU run.
 2026-06-04: Bumped the default value of the `--timeout` flag from 10.0s to 30.0s so short-but-silent tasks aren't prematurely killed out of the box. Mirrors the same change for `colab exec`.
+2026-10-24: `--timeout` is now an idle limit: the run fails only after that many seconds without output, so a script that keeps printing can run as long as it needs, and `--timeout 0` means no limit. The CLI waits for kernel messages itself, in blocking slices of up to one second, instead of using `jupyter-kernel-client`'s `execute_interactive`. That deadline covered the whole run: 1.0.2 enforces it, failing runs after 30s even while they print, and 0.8.0/0.9.0 never enforce it, polling at ~100% CPU once it passes. As a result, on 0.8.0/0.9.0 a script that is silent for longer than `--timeout` now fails (and, without `--keep`, its session is stopped) where it used to complete. See #183.
 ---
 
 # Design: `colab run` — Shebang-Compatible One-Shot Execution
@@ -33,7 +34,7 @@ colab run [OPTIONS] SCRIPT [SCRIPT_ARGS]...
 | `--tpu` | str | None | Same set as `colab new --tpu` (v5e1, v6e1). |
 | `--high-mem` | bool | False | Same as `colab new --high-mem` — request high-RAM when supported. |
 | `--keep` | bool | False | Do **not** stop the session after the script finishes. |
-| `--timeout` | float | 30.0 | Timeout in seconds for code execution to prevent hanging on silent tasks. |
+| `--timeout` | float | 30.0 | Seconds to wait without output before failing, so a hung or silent task doesn't block forever. Raise it for long silent tasks; `0` means no limit. |
 
 ### Shebang usage
 With `--keep` and `--gpu` baked into the shebang line, an entire one-file workload becomes:

{google_colab_cli-0.7.5 → google_colab_cli-0.8.0}/docs/06_ssh_access.md

@@ -1,5 +1,12 @@
 ---
 log:
+2026-11-09: `--identity/-i` now accepts either a `.pub` public key or a private key path: `_resolve_pubkey` reads the `.pub` file directly when present before falling back to `_pubkey_from_identity` (`ssh-keygen -y -f`), and `_ssh_base_args` strips `.pub` for `ssh -i` when the sibling private key exists on disk.
+2026-10-29: Removed the `colab console` subcommand, so `colab ssh` is the interactive shell on the runtime. Dropped console behavior from the Windows validation caveat.
+2026-10-29: Added `colab ssh [-s NAME] -- CMD`, which runs one command on an existing session and exits with its status (a PTY only when stdin and stdout are terminals; neither it nor its ProxyCommand child creates a runtime). Its handshake carries `ssh-mode-cmd`, and the update banner is skipped for `ssh` when stdout is not a terminal. Added `tests/test_ssh_cmd.py`.
+2026-10-29: HTTP 429 now shows the server's SSH session limit ("N of N SSH sessions in use") when the body carries one.
+2026-10-29: The ProxyCommand that `colab ssh` starts now keeps a non-default global `--auth` and `--config` (as an absolute path), so `colab --config ALT ssh -s S` bridges S from ALT instead of the default sessions file.
+2026-10-29: Every `/colab/ssh` handshake now sends `X-Colab-Client-Agent: colab-cli` and the two `colab-client-agent` query values `colab-cli` and `ssh-mode-<mode>`. The mode is `interactive` for plain `colab ssh`, `cmd` for `colab ssh -- CMD` (both pass it to their ProxyCommand re-invocation through a hidden `--colab-internal-mode` option), and `proxy` otherwise.
+2026-10-20: Fixed Windows SSH ProxyCommand quoting, local null-device selection, and cleanup signal registration. Added native Windows regression and live integration tests; verified the SSH suite on Windows, Linux, and macOS, plus live Windows remote execution and runtime cleanup.
 2026-09-25: Cleaned up keep-alive daemon references in session creation and teardown documentation.
 2026-08-09: Added `--high-mem` passthrough when `colab ssh` auto-creates a runtime (forwards to `colab new --high-mem`).
 2026-07-17: Initial design and implementation of `colab ssh` — client side of SSH-over-WebSocket runtime access. Adds three modes (interactive shell, `-s SESSION`, and `--proxy-mode` OpenSSH ProxyCommand bridge), `--identity/-i` key selection, and per-HTTP-status handshake error messages. Server side is out of scope for this repo; the subcommand is a no-op against runtimes that do not expose the `/colab/ssh` endpoint (surfaces an actionable HTTP 404 message).
@@ -22,18 +29,18 @@ runtime-proxy token so no separate credential handling is needed.
 ## User Surface
 
 ```
-colab ssh [OPTIONS]
+colab ssh [OPTIONS] [-- CMD [ARGS]...]
 ```
 
 | Flag | Type | Default | Purpose |
 |---|---|---|---|
-| `-s`, `--session` | str | auto | Session to connect to. If omitted, uses your only active session, auto-creates one when you have none, or errors when you have several. |
+| `-s`, `--session` | str | auto | Session to connect to. If omitted, uses your only active session, auto-creates one when you have none (not with `-- CMD`), or errors when you have several. |
 | `--proxy-mode` | bool | False | Act as an OpenSSH `ProxyCommand`-compatible WebSocket↔stdio bridge (reads stdin, writes stdout) for `~/.ssh/config`. Every other flag still applies. |
-| `-i`, `--identity` | str | auto | Private key for the public key sent to Colab. Default: first of `~/.ssh/id_ed25519`, `id_ecdsa`. |
+| `-i`, `--identity` | str | auto | SSH key (`.pub` public key or private key path) for the public key sent to Colab. Default: first of `~/.ssh/id_ed25519.pub`, `id_ecdsa.pub`. |
 | `--gpu` | str | None | GPU accelerator for a runtime this command creates (T4, L4, G4, H100, A100). |
 | `--tpu` | str | None | TPU accelerator for a runtime this command creates (v5e1, v6e1). |
 | `--high-mem` | bool | False | Request high-RAM when this command auto-creates a runtime (ignored when connecting to an existing session). |
-| `--rm` | bool | False | Stop the runtime when the session ends. Interactive: only a runtime `colab ssh` auto-created (a reused session is never removed). `--proxy-mode`: the bridged session, on disconnect. |
+| `--rm` | bool | False | Stop the runtime when the session ends. Interactive: only a runtime `colab ssh` auto-created (a reused session is never removed). `--proxy-mode`: the bridged session, on disconnect. Ignored with `-- CMD`. |
 
 ### `~/.ssh/config` usage
 `--proxy-mode` turns `colab ssh` into a transport any SSH-based tool can drive:
@@ -53,24 +60,50 @@ where a bare `colab` may not be on `PATH`. External SSH tools run their own
 remote command, so to also land in `/content` add `RequestTTY yes` and
 `RemoteCommand cd /content 2>/dev/null; exec bash -l`.
 
+### Remote command
+`colab ssh [-s NAME] -- CMD [ARGS]...` runs CMD in `/content` on an existing
+session (never creating one); every word from CMD on goes to CMD, flags too.
+
+- One argument is a shell command line (`-- 'ls / | wc -l'`); several are an
+  exact argv, so `*`, `~` and `$VAR` in separate arguments are not expanded.
+  Scripts can also pipe in: `echo cmd | colab ssh -s NAME -- bash`.
+- `-t` only when stdin and stdout are terminals; otherwise no PTY, so piped
+  output is byte-exact. The exit status is CMD's; 255 means an ssh error or a
+  remote command killed by a signal (e.g. SIGPIPE from `| head`).
+- A runtime accepts a limited number of concurrent SSH connections; past it
+  the server answers 429 and `colab ssh` prints its message.
+- With a terminal on both ends, Ctrl-C reaches the remote command and ssh
+  exits 255. Piped, Ctrl-C stops only `colab ssh` (exit 130) and the remote
+  command may keep running: stop it with a pattern that cannot match its own
+  shell, e.g. `colab ssh -s NAME -- pkill -f '[s]leep 300'`, or `colab stop`.
+- SSH sessions use a minimal environment: Python, pip and `nvidia-smi` work;
+  tools such as `gcloud` or `node` may need their directories added to `PATH`.
+
 ## Behavior
 
 1. **Session resolution / auto-create**: With `-s NAME`, resolves that session
    (via `state.resolve_session`, the same helper the other commands use). Bare
    `colab ssh` uses your only active session; with **no** session it auto-creates
-   one (mirrors `colab new`: assign → persist `SessionState`); with **multiple**
+   one (mirrors `colab new`: assign → persist `SessionState`; never with
+   `-- CMD`); with **multiple**
    it errors and asks you to pick one with `-s`.
 2. **Connect**: Opens the WebSocket to `wss://<netloc>/colab/ssh?colab-runtime-proxy-token=<token>`
    and sends the resolved public key verbatim in the `X-Colab-Ssh-Pubkey` header
    (no transformation -- the bytes the user controls are exactly what the server
    receives). Only `ssh-ed25519` / `ecdsa-sha2-nistp{256,384,521}` keys are
-   accepted.
+   accepted. It also sends `X-Colab-Client-Agent: colab-cli` and appends
+   `&colab-client-agent=colab-cli&colab-client-agent=ssh-mode-<mode>`: `interactive`
+   for plain `colab ssh`, `cmd` for `colab ssh -- CMD`, `proxy` for a
+   user-written `ProxyCommand` or an IDE.
 3. **Interactive shell**: Spawns the system `ssh` binary with the CLI re-invoked
-   as its own `ProxyCommand` (`python -m colab_cli.cli ssh --proxy-mode`), so the
+   as its own `ProxyCommand` (`python -m colab_cli.cli ssh --proxy-mode --colab-internal-mode interactive`), so the
    WebSocket bridge and the interactive shell share one code path. It forces a
    PTY (`-t`) and runs `cd /content 2>/dev/null; exec $SHELL -l` so you land in
    `/content` (Colab's working dir) rather than root's home; a missing `/content`
    falls back to the login home.
+   On native Windows, the ProxyCommand uses `subprocess.list2cmdline` for
+   CreateProcessW argument parsing; POSIX systems continue to use `shlex.join`.
+   The local `UserKnownHostsFile` uses `os.devnull` (`nul` on Windows).
 4. **`--proxy-mode` bridge**: Bridges the WebSocket ↔ stdin/stdout for use as an
    OpenSSH `ProxyCommand`. Honors every flag: `-s NAME` creates the session if
    missing (creation output routed to stderr so stdout stays the clean ssh byte
@@ -80,9 +113,12 @@ remote command, so to also land in `/content` add `RequestTTY yes` and
    **SIGHUP** (verified), not just stdin EOF, and Python's default SIGHUP action
     would terminate the process before the teardown `finally` ran — leaking the
     runtime. `--rm` therefore installs
-   SIGHUP/SIGTERM/SIGINT handlers that run the stop, idempotent with the
-   `finally`. `SIGKILL` cannot be intercepted, so a `kill -9`/hard crash can
+   handlers for the available SIGHUP/SIGTERM/SIGINT signals that run the stop,
+   idempotent with the `finally`. `SIGKILL` cannot be intercepted, so a `kill -9`/hard crash can
    still leak; a normal disconnect is SIGHUP and is handled.
+   Windows has no SIGHUP, so registration skips it and retains SIGTERM/SIGINT.
+   Forced process termination on Windows can also bypass cleanup; this change
+   does not guarantee teardown after a killed process.
 6. **Error handling**: The WebSocket upgrade maps each common HTTP status to an
    actionable message:
 
@@ -92,17 +128,22 @@ remote command, so to also land in `/content` add `RequestTTY yes` and
    | 401 | Token invalid/expired — try `colab new` |
    | 403 | Forbidden — token lacks permission for this action (feature-off returns 404, not 403) |
    | 404 | SSH not exposed on this runtime — SSH is baked in at creation, so run `colab new` |
-   | 429 | Another `colab ssh` is already connected — disconnect first |
+   | 429 | Session limit reached: "N of N SSH sessions in use", and the limit depends on the Colab plan; without a usable limit in the body, another `colab ssh` is connected: disconnect it first |
    | 502 | Runtime `sshd` unreachable — runtime may be unhealthy |
    | other / none | Raw status or a network-check hint |
 
+7. **Remote command**: `-- CMD` resolves an existing session and runs CMD as
+   described under Remote command. Its ProxyCommand child (mode `cmd`) only
+   resolves the session too, so neither process creates a runtime.
 
 ## Testing Strategy (TDD)
 
 ### Unit tests (`tests/test_ssh.py`)
 1. WebSocket URL construction (`wss` for https, `ws` for http; token query param).
-2. Pubkey resolution — `--identity` (via `ssh-keygen -y -f`) and the `~/.ssh`
-   default scan; missing-key and missing-identity exit paths.
+2. Pubkey resolution — `--identity` (reads the matching `.pub` file first,
+   whether passed with or without `.pub`, before falling back to `ssh-keygen -y
+   -f`) and the `~/.ssh` default scan; missing-key and missing-identity exit
+   paths.
 3. The full status→message map (400/401/403/404/429/502/other/none).
 4. Shell quoting for the `ProxyCommand` string.
 5. Session resolution (existing vs missing).
@@ -111,10 +152,13 @@ remote command, so to also land in `/content` add `RequestTTY yes` and
 
 ### Wire-contract tests (`tests/test_ssh_wire_contract.py`)
 Stands up a loopback WebSocket server and drives the real connect path (no mock)
-to assert the request path, the `colab-runtime-proxy-token` query param, and the
-`X-Colab-Ssh-Pubkey` header reach the wire verbatim. Includes mutation tests that
-fail if `_SSH_PATH`/`_PUBKEY_HEADER` drift, plus real HTTP 400/429 mapping via a
-genuine `WebSocketBadStatusException`.
+to assert the request path, the `colab-runtime-proxy-token` query param, the
+`X-Colab-Ssh-Pubkey` header, the `X-Colab-Client-Agent` header and the two
+`colab-client-agent` query values reach the wire verbatim. Includes mutation tests
+that fail if `_SSH_PATH`/`_PUBKEY_HEADER`, the client-agent header or a mode value
+drift, real HTTP 400/429 mapping (including a 429 body with a limit) via a genuine
+`WebSocketBadStatusException`, and real CLI child processes for plain `colab ssh`
+(`interactive`) and a user `ProxyCommand` (`proxy`).
 
 ### Auto-create & proxy-mode tests (`tests/test_ssh_autocreate.py`)
 Bare `colab ssh` create vs reuse vs ambiguous; `--gpu/--tpu` passthrough; `--rm`
@@ -126,6 +170,11 @@ SIGHUP cleanup handler being installed only under `--rm`.
 Interactive `ssh` forces a PTY (`-t`) and runs a `cd /content` remote command
 (host before the command, `2>/dev/null` tolerance for a missing directory).
 
+### Remote-command tests (`tests/test_ssh_cmd.py`)
+`-- CMD` forwarding and quoting, the `-t`/`-T` rule, exit status, no
+auto-create in the parent or its ProxyCommand child, `ssh-mode-cmd` at
+`websocket.WebSocket.connect`, closed stdio, and the update-banner gate.
+
 ### Integration test (`integration/repro_ssh/`)
 Two parts. An offline smoke that always runs (no VM): ``--help`` advertises the
 documented flags, and an unknown session exits 2 with an actionable message. A
@@ -135,3 +184,26 @@ command over the WebSocket bridge -- exercising the same connect ->
 pubkey-header auth -> handshake -> bridge -> remote-exec path as the interactive
 shell, minus the TTY -- asserts the RSA-key rejection, and verifies ``colab
 stop`` leaves no orphan VM.
+
+### Windows regression tests (`tests/test_windows_ssh.py`)
+Real Windows child processes run the generated ProxyCommand using a Python
+interpreter installed under a path with spaces. Assertions check exact argv
+preservation for spaces, quotes, shell punctuation, backslashes, Unicode, and
+identity paths. A separate test launches the real Windows OpenSSH client with
+a local stub ProxyCommand and verifies its argv, without contacting Colab.
+Portable tests also check registration without SIGHUP and the local null
+device. Existing POSIX shell quoting and teardown tests remain in the suite.
+
+These and the existing SSH tests passed on Windows, Linux, and macOS with
+Python 3.12. This covers SSH compatibility only; local file editing and
+self-update require separate platform validation.
+
+`uv run python integration/repro_windows_ssh/test.py` provides a native Windows
+live test using cached OAuth2 credentials. It creates a CPU runtime, executes a
+remote command through the generated ProxyCommand with a key path containing
+spaces, exercises `--rm`, and cleans up the recorded endpoint on failure. Exit
+77 is a prerequisite skip, not a successful live validation.
+
+The native Windows live test passed on 2026-10-20 with Python 3.12.14 and the
+Windows OpenSSH client. This validates remote command execution and cleanup
+for this SSH path, not overall native Windows feature parity.