boxd 0.2.19.dev901__tar.gz → 0.2.20__tar.gz

{boxd-0.2.19.dev901/src/boxd.egg-info → boxd-0.2.20}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boxd
-Version: 0.2.19.dev901
+Version: 0.2.20
 Summary: Python SDK for the boxd cloud VM platform
 Author: Azin
 License-Expression: MIT
@@ -23,6 +23,7 @@ License-File: LICENSE
 Requires-Dist: grpcio>=1.80
 Requires-Dist: protobuf>=6.31.1
 Requires-Dist: pydantic>=2
+Requires-Dist: typing-extensions>=4.12
 Requires-Dist: httpx<1,>=0.27
 Provides-Extra: dev
 Requires-Dist: grpcio-tools<=1.80,>=1.60; extra == "dev"
@@ -118,6 +119,12 @@ The first of these that is present wins:
 If your key is revoked mid-session, the SDK fails fast with
 `AuthenticationError` rather than retrying.
 
+`AuthenticationError` means only that no credential was found or boxd refused
+it; its `.reason` says which (`"missing_credentials"`, `"invalid_api_key"`,
+`"invalid_token"`). Failing to reach boxd is an `APIConnectionError`, an
+account that can't use the API a `PermissionDeniedError`, and an error answer
+from the key exchange an `APIStatusError` with its `.http_status`.
+
 The exchange endpoint is **rate-limited per source IP** (30 requests / 60s). A
 session token is valid for one hour, so a fleet of workers behind one NAT
 should **mint one token and share it** (`Boxd(token=...)`) rather than each
@@ -141,6 +148,9 @@ One limit: inside a **shared** machine the automatic credential can manage the
 organization's shared machines, but cannot read environment variables or
 secrets, and cannot reach private machines. Pass an API key for those.
 
+An **isolated** machine has no automatic credential: `Boxd()` raises
+`AuthenticationError` there, so pass an API key.
+
 ## Sync and async
 
 `Boxd` and `AsyncBoxd` are the same surface — same namespaces, same method
@@ -204,7 +214,7 @@ boxd.machines.create(
     disk="100G",
     auto_suspend_timeout=300,     # seconds; 0 disables
     auto_destroy_timeout=0,
-    ssh=True,                     # give the machine an SSH port
+    ssh=True,                     # False: SSH off, nothing answers on its port
     proxies=[ProxyEntry(name="api", port=3000)],
     volumes=[VolumeMount(disk_id="d_...", mount_path="/data", read_only=False)],
 )
@@ -238,6 +248,7 @@ boxd.machines.unshare(id)
 boxd.machines.set_auto_suspend_timeout(id, 300) # seconds idle; 0 disables
 boxd.machines.set_auto_hibernate_timeout(id, 0)
 boxd.machines.set_idle_ignore_outbound(id, True) # only inbound connections keep it awake
+boxd.machines.set_ssh(id, False)                # nothing answers SSH; exec still works
 boxd.machines.wait_until_ready(id)
 boxd.machines.wait_until_ready(id, timeout=180.0, poll_interval=1.0)   # seconds
 boxd.machines.suggest_name()
@@ -246,6 +257,14 @@ boxd.machines.suggest_name()
 A fork inherits the source's sizing for anything you leave unset, and is
 private to you unless you pass `shared=True`.
 
+With SSH off (`ssh=False` on create, or `set_ssh(id, False)`), nothing
+answers SSH on the machine's port, on the public address or the org's
+network, and SSH sessions already open to it are closed. The port stays
+allocated, so turning SSH back on keeps the address. `exec`, `boxd connect`
+and the console terminal don't use SSH and keep working. A fork or a restore
+keeps its source's setting unless you pass `ssh`. `set_ssh` can't be called
+from inside the machine itself.
+
 `create` and `fork` return once the machine is scheduled, not once it is
 usable. Call `wait_until_ready` before doing anything that depends on it
 running — especially before forking it again.
@@ -265,7 +284,6 @@ engineering = (
     .network("eng")
     .label("team", "eng")
     .env("NODE_ENV", "development")
-    .egress("github.com", "*.atlassian.net", "*.ubuntu.com")
     .packages("apt-get update", "apt-get install -y nginx")
     .with_(github(shared="eng-bot"), claude())
 )
@@ -332,6 +350,14 @@ the org's secrets count. Only org owners and admins may list an org
 shared machines, so a private machine can't list one. A name that resolves to
 nothing, or that the create may not list, fails the create, naming it.
 
+Only org owners and admins may set, move or delete an org `unattached` secret;
+your own (`private-unattached`) are yours. From inside a machine (with its
+in-VM credentials) a spec can't list an unattached secret, yours or the org's,
+on any machine, and `update_spec` can't change that machine's own `egress` or
+`secrets`; make those changes from outside the machine, with an API key or in
+the console. Names a fork or a restore takes over from its source's spec are
+not affected.
+
 On the machine, the spec's `env` and its `secrets` win over your `private`
 vars, which win over the org's `shared` vars (shared machines only), which win
 over the org's `all` vars. A name can't be in both `env` and `secrets`.
@@ -366,6 +392,12 @@ with an org API key or from inside a machine, for someone who isn't a member,
 and with a snapshot or disk the member couldn't use. Your own id is a plain
 create. `assignTo` is never inherited: forks, snapshots and restores drop it.
 
+The machine runs the spec you wrote as the member's own machine: its packages
+as root, its env and its base, alongside the member's GitHub token, agent
+logins and personal connections. Assign a spec only if the member would run it
+themselves. The member sees who created it (`created_by`) and can read the
+spec (`machine.spec`).
+
 #### Integrations the machine needs
 
 A spec's `connections` and `agents` are the machine's requirements. Reading the
@@ -398,10 +430,17 @@ can't connect (`spec.connections.<toolkit>: ...`), and the GitHub App
 
 `packages` are shell commands the machine runs once, in order, after its first
 boot: as root, with bash (sh on an image without it),
-`DEBIAN_FRONTEND=noninteractive`, no stdin and a 30-minute timeout each. Their
-output is appended to `/var/log/boxd/packages.log` in the machine. The first
-failure (a non-zero exit, a timeout) stops the rest; the machine stays usable.
-Reading the machine tells you how each one went:
+`DEBIAN_FRONTEND=noninteractive`, no stdin, the env and secrets an exec
+session on the machine gets, and a 30-minute timeout each. Their output is
+appended to `/var/log/boxd/packages.log` in the machine, which only root can
+read. A command is done when its shell exits, so one that leaves a process
+running in the background (`my-server &`) doesn't hold up the next. The first
+failure (a non-zero exit, a timeout) stops the rest; the machine stays usable,
+and `update_spec(id, {}, retry_packages=True)` runs the failed command and the ones after it again (so does
+Retry in the console's Packages section). To give up on a command instead, or
+to replace a wrong one, remove it from the spec: a command that leaves the spec
+before it finished is skipped, and the ones after it run. Reading the machine
+tells you how each one went:
 
 ```python
 m = boxd.machines.get("mike-dev")
@@ -410,18 +449,44 @@ for p in m.packages:
     print(p.state, p.exit_code, p.command)
 ```
 
-`state` is `"pending"`, `"running"`, `"done"` or `"failed"`; the commands
-after a failure stay `"pending"`. `output_tail` is the last 4 KiB of the
-command's output, and `reason` says why a failed one has no exit code (it
-timed out, or the run was interrupted). Nothing runs again: not on a reboot, a
-resume, a wake or a move. Restoring a snapshot whose spec lists packages marks
-those done and runs only the ones the restore's spec adds; a fork keeps its
-source's states and runs nothing. The machine doesn't auto-suspend or
+`state` is `"pending"`, `"running"`, `"done"`, `"failed"` or `"skipped"`; the
+commands after a failure stay `"pending"`. A pending or failed command that an
+`update_spec` removes from the spec becomes `"skipped"`: it never runs and no
+longer stops the ones after it, which run right away (or when the machine next
+runs). A skipped one that failed keeps its `exit_code` and `output_tail`, and
+`reason` says when it was skipped. Listing it in the spec again puts it back to
+`"pending"`, in its place. So replacing a broken command is one patch: drop it
+and add the corrected one, and the corrected one runs, with everything that
+was waiting. A running command can't be removed (the patch is refused until it
+ends), and removing a done one undoes nothing. `output_tail` is the last 4 KiB of the
+command's output, with the value of every secret the machine gets shown as
+`[secret NAME]` (only the value as it is: printed encoded or in pieces, it isn't
+recognised). The log file isn't masked: it is root's, and root on the machine
+can read its env anyway. `output_tail` is empty on the machines `list()` returns,
+which leave it out to keep the list small; `get()` has it. `reason` says why a failed one has no exit code (it
+timed out, or was interrupted too often), or why a pending one is waiting. A
+command the platform interrupts (the machine stops, reboots, hibernates or
+moves, or boxd restarts on the machine's host) goes back to `"pending"` and runs
+again from the start once the machine is running, and the rest follow; after 3
+interruptions it fails. What finished never runs again: not on a reboot, a
+resume, a wake or a move. Restoring a snapshot marks done the commands that had
+run on the machine the snapshot was taken from and runs the snapshot spec's
+others (one that was running when the snapshot was taken runs again from the
+start), then the ones the restore's spec adds; a fork keeps its source's states
+and runs nothing. The machine doesn't auto-suspend or
 hibernate while a command runs. At most 50 commands (a restore counts the
 snapshot's too), each at most 4 KiB; the server names the one it refuses
 (`spec.packages[2]: ...`). Commands that download something need the hosts in
 the spec's `egress`, if it has one.
 
+A spec has size limits, checked before anything else and named with what was
+given (`spec.env: at most 128 variables (got 129)`): at most 256 KiB as JSON
+in all; `env` at most 128 variables, each value at most 32 KiB and all values
+together at most 64 KiB; `secrets` at most 64 names; `networks` at most 16, each
+name at most 63 bytes; `egress` at most 256 entries, none empty (leave `egress`
+out for no allowlist); `connections` at most 64; `packages` at most 200 listed
+and 50 once duplicates are dropped. An `update_spec` patch is at most 256 KiB.
+
 #### Updating a machine's spec
 
 `update_spec` changes the spec of a machine created from one and applies what
@@ -451,23 +516,33 @@ Each change in `changes` says how it applies (`apply`):
 
 | Field | `apply` |
 |---|---|
-| `labels`, `networks`, `egress` | `"live"` |
+| `labels`, `networks`, `egress` | `"live"` (new egress rules are in force within about a second of the call returning) |
 | `connections`, `agents` | `"live"`: what the machine needs (`requirements`) changes |
-| `packages` | `"live"`: added commands run now, after the ones before them. Removing one undoes nothing (a warning says so); a command that already ran isn't run again; commands after a failed one don't run (a warning says so) |
+| `packages` | `"live"`: added commands run now, after the ones before them. Removing one undoes nothing (a warning says so); a pending or failed one the patch removes is skipped (listed in `skipped`), so the ones after it run, and a skipped one listed again goes back to pending in its place (`unskipped`); a command that already ran isn't run again; commands after a failed one don't run (a warning says so) unless the update also retries (`retry_packages=True`: the failed command and the rest run again first, listed in `changed`, skipped ones excepted; a failed command the patch removes is skipped, not retried). Removing a running command is `"refused"` until it ends |
 | `env`, `secrets` | `"reboot_recommended"`: new shells and exec sessions get them now; processes already running keep the old values until they restart (`reboot_recommended`) |
 | `size` | `"reboot"` on a running machine, which reboots to apply it (as `resize`), so it needs `reboot=True`; `"live"` on a stopped one |
 | `base`, `isolated`, `assignTo` | `"refused"`: fixed when the machine is created |
 
-A refused change (also: new packages on a fork, a size change in standby or
+A refused change (also: new packages or a retry on a fork, a size change in standby or
 hibernated) refuses the whole patch, and so does a reboot without
-`reboot=True`; nothing is applied. Otherwise the spec and the labels,
+`reboot=True`; nothing is applied. So is an update during which the spec
+changed (another update, a label edit): it raises `ConflictError`, and
+reading the machine again and retrying is safe. The builder form does that
+itself: it sends the spec it read, so a change made in between can't be
+undone by its lists, and on a conflict reads the spec again and reapplies the
+function, up to 3 times. A patch you built from a spec you read can say so
+too, `expected_spec=machine.spec`: if the spec is no longer that one, nothing
+is applied and it raises `ConflictError`. Otherwise the spec and the labels,
 networks, egress and secrets it drives change together, then the size, then
-the new packages run. The patched spec is checked like a create's (an
+the new packages run. If the size then fails (the org's quota changed, the
+machine went to standby), the rest stays applied and the call still returns:
+`resize_error` says why, and `spec` has the size the machine has. The patched spec is checked like a create's (an
 unknown toolkit, a secret you may not list, more than 50 commands in all are
 refused, naming the field). `machine.created_spec` never changes, so the
 console shows the spec badge as "updated". Allowed to whoever may edit the
 machine's labels: its owner, or an org owner or admin on a shared machine.
-The builder form only adds: a `size` that differs from the spec's raises, so
+The builder form only adds: a `size` that differs from the spec's raises
+(`"2X8"` is the same as `"2x8"`), so
 use a patch to replace or remove. `spec_patch(from_spec, to_spec)` computes the
 patch the builder form sends. `AsyncBoxd` has the same `await
 boxd.machines.update_spec(...)`.
@@ -479,15 +554,20 @@ A machine can be limited to what it may reach on the internet:
 ```python
 boxd.machines.set_egress_allow("alpha", ["api.stripe.com", "*.github.com", "203.0.113.0/24"])
 boxd.machines.get("alpha").egress_allow
+boxd.machines.set_egress_allow("alpha", ["none"])   # nothing is reachable
 boxd.machines.set_egress_allow("alpha", [])   # clear: unrestricted again
 ```
 
-Entries are hostnames or wildcards, or public IPv4 addresses and CIDRs. Web
+Entries are hostnames or wildcards, or public IPv4 addresses and CIDRs, at
+most 256; an internationalized name is kept in its ASCII form (`bücher.de`
+becomes `xn--bcher-kva.de`), which is what the machine looks up. Web
 requests are admitted by hostname, everything else by address, with the
 addresses of allowlisted names learned as the machine resolves them. Nothing
 that is not named is reachable. The hosts of any bound secret the machine
-holds are always admitted. Takes effect within a second and is not visible or
-changeable from inside the machine. A fork inherits its source's allowlist; a
+holds are always admitted. `"none"` on its own leaves the allowlist on with
+nothing in it, so the machine reaches only those; next to other entries it is
+dropped. Takes effect within a second. From inside the
+machine it can be read (`boxd machine egress`) but not changed. A fork inherits its source's allowlist; a
 machine restored from a snapshot starts unrestricted, unless its machine spec
 lists `egress`.
 
@@ -554,6 +634,7 @@ machine.shared                  # shared with that org, or private to you
 machine.access.ssh_port         # int | None — None until allocated
 machine.access.domain
 machine.access.url              # https://<name>.<domain>
+machine.ssh                     # bool — False: SSH is off, nothing answers on ssh_port
 
 machine.idle.suspend_after      # seconds; 0 = that timer is disabled
 machine.idle.hibernate_after
@@ -576,6 +657,7 @@ machine.requirements            # list[MachineRequirement] — the spec's connec
                                 #   kind, name, scope ("personal"/"shared"), tag, met, can_fix, reason
 machine.packages                # list[MachinePackage] — the spec's packages and how each went:
                                 #   command, state, exit_code, output_tail, reason, started_at, finished_at
+                                #   (output_tail is empty from list(): get() has it)
 ```
 
 `None` always means "not set": a port that was never allocated, a boot that
@@ -703,6 +785,47 @@ relative to the VM user's home directory (`/home/boxd`) — a bare filename
 like `"notes.txt"` resolves to `/home/boxd/notes.txt`, same as `"./notes.txt"`
 or `"sub/notes.txt"` would.
 
+### Filesystem
+
+`machines.fs` works on a machine's files the way code works on local ones: as
+the machine's user, seeing the same files its commands do, with failures
+raised as the builtin `OSError`s.
+
+```python
+fs = boxd.machines.fs
+
+fs.write(id, "/app/config.json", '{"debug": true}')     # creates missing parents
+fs.write(id, "big.bin", chunks)                          # or stream an iterable of bytes
+data = fs.read(id, "/app/config.json")                   # bytes
+for chunk in fs.read_stream(id, "big.bin"): ...          # without holding it all
+
+entry = fs.stat(id, "/app")          # FsEntry: name, path, type, size, mode, mtime, ...
+entry.is_dir, entry.is_symlink
+fs.list(id, "/app")                  # one level, every entry
+fs.exists(id, "/app/missing")        # False
+fs.mkdir(id, "/app/a/b")             # mkdir -p
+fs.rename(id, "/app/a", "/app/c")
+fs.remove(id, "/app/c", recursive=True)
+fs.realpath(id, "/app/link")
+
+try:
+    fs.read(id, "/app/nope")
+except FileNotFoundError:
+    ...
+```
+
+A write replaces the file all at once, so readers see the old contents or the
+new, never a mix; only a file that can't be swapped by a rename (several hard
+links, another user's file, a mount point) is overwritten in place. It writes
+through a symlink to its target, and an existing file keeps its mode and owner
+(or takes `mode=`). Only regular files can be read. Paths are `str`, `bytes` or
+`os.PathLike`; a relative path starts in the user's home directory, and names
+that aren't valid UTF-8 round-trip. Errors are
+`MachineFilesystemError` subclasses that are also `FileNotFoundError`,
+`PermissionError`, `FileExistsError`, `IsADirectoryError` or
+`NotADirectoryError`, with `errno` and `filename` set. A machine started before
+`machines.fs` existed raises `MachineAgentOutdatedError` until it is rebooted.
+
 ### Ports and proxies
 
 ```python
@@ -794,18 +917,28 @@ boxd.secrets.delete("API_TOKEN", scope="shared")
 ```
 
 A secret can be bound to the hosts it may be sent to. The machine then only
-ever holds a placeholder, and the real value is substituted into requests to
-those hosts on the way out:
+ever holds a placeholder, and the real value is substituted into HTTPS
+requests to those hosts on the way out:
 
 ```python
 boxd.secrets.set("STRIPE_KEY", "sk_live_...", scope="all", domains=["api.stripe.com"])
 ```
 
-Inside the machine, `STRIPE_KEY` is an opaque `bxds_…` string. A request to
-`api.stripe.com` carrying it, in a header, the query, the body or Basic auth,
-arrives at Stripe with the real key; a request anywhere else carries the
-useless placeholder. Nothing to configure in the machine, and the placeholder
-does not change when the value is rotated. `domains` is the full desired set:
+Inside the machine, `STRIPE_KEY` is an opaque `bxds_…` string. An HTTPS
+request to `api.stripe.com` carrying it anywhere (the URL, a header, Basic auth
+included, or the body) arrives at Stripe with the real key. It is replaced only
+over HTTPS, and only when both the server name the connection was opened for
+and the request's own `Host` are hosts the secret is bound to: over plain HTTP
+and in a request to any other host it stays the useless placeholder. A body
+over 16 MiB with a `Content-Length` is sent as written, placeholder included.
+Responses on those connections carry the placeholder instead of the value:
+wherever the real value (8 characters or more) appears in a response's headers
+or body exactly as it was sent, it is replaced, so a server that echoes the
+request doesn't hand the value to the machine. A value the server sends back
+re-encoded (base64, URL- or JSON-escaped) isn't caught, and neither is one in
+a body compressed with anything but gzip or deflate (the proxy asks for an
+uncompressed response). Nothing to configure in the machine, and the
+placeholder does not change when the value is rotated. `domains` is the full desired set:
 setting the secret again without it makes it a plain secret. Wildcards such as
 `*.stripe.com` are accepted; provider-wide ones such as `*.amazonaws.com` are
 refused.
@@ -826,7 +959,7 @@ Scope decides which machines a name applies to:
 | `private` | only your machines in that organization |
 | `shared` | the organization's shared machines |
 | `all` | every machine in the organization |
-| `unattached` | secrets only: just the machines whose spec lists it ([Secrets](#secrets)); the organization's with `org`, else your own |
+| `unattached` | secrets only: just the machines whose spec lists it ([Secrets](#secrets)); the organization's with `org` (set, moved and deleted by its owners and admins only), else your own |
 | `private-unattached` | secrets only: your own unattached secret in the organization named by `org` |
 
 `list` reports the organization's unattached secrets as `unattached` and your
@@ -1098,7 +1231,7 @@ from boxd import (
     AuthenticationError,     # no usable credential, or it was rejected
     PermissionDeniedError,   # authenticated, but not allowed
     NotFoundError,
-    ConflictError,           # already exists, or fights the current state
+    ConflictError,           # already exists, fights the current state, or it changed meanwhile
     RateLimitError,          # rate limit or quota
     APIStatusError,          # any other error from the server
     APIConnectionError,      # could not reach the server
@@ -1112,9 +1245,14 @@ except NotFoundError:
 
 Every error carries `.message`, `.code` (the canonical status name, e.g.
 `"not_found"`) and `.grpc_code`, the numeric
-[status code](https://grpc.github.io/grpc/core/md_doc_statuscodes.html).
-
-Connection failures are retried with exponential backoff, `max_retries` times.
+[status code](https://grpc.github.io/grpc/core/md_doc_statuscodes.html). Some
+also carry `.reason`, a name for the exact failure (`"invalid_api_key"`,
+`"is_a_directory"`, ...), and errors from the key exchange carry
+`.http_status`. `machines.fs` errors are also `OSError`s (see
+[Filesystem](#filesystem)).
+
+Connection failures are retried with exponential backoff, `max_retries` times,
+including failing to reach the key exchange or a machine's automatic credential.
 Timeouts are never retried — the server may already have applied the request —
 and neither is `AuthenticationError`.