boxd 0.2.18.dev896__tar.gz → 0.2.19__tar.gz

{boxd-0.2.18.dev896/src/boxd.egg-info → boxd-0.2.19}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: boxd
-Version: 0.2.18.dev896
+Version: 0.2.19
 Summary: Python SDK for the boxd cloud VM platform
 Author: Azin
 License-Expression: MIT
@@ -23,6 +23,7 @@ License-File: LICENSE
 Requires-Dist: grpcio>=1.80
 Requires-Dist: protobuf>=6.31.1
 Requires-Dist: pydantic>=2
+Requires-Dist: typing-extensions>=4.12
 Requires-Dist: httpx<1,>=0.27
 Provides-Extra: dev
 Requires-Dist: grpcio-tools<=1.80,>=1.60; extra == "dev"
@@ -250,6 +251,284 @@ private to you unless you pass `shared=True`.
 usable. Call `wait_until_ready` before doing anything that depends on it
 running — especially before forking it again.
 
+### Machine specs
+
+A machine spec describes a machine in one value: what it boots from, its size,
+labels, networks, isolation, egress, env, the secrets it gets, the
+integrations and agent logins it needs, and the packages it installs on first boot. Build one from reusable pieces and create from it:
+
+```python
+from boxd import Machine, team, snapshot, github, slack, jira, claude, secrets
+
+engineering = (
+    Machine.spec()
+    .from_(team("eng"))                  # a team's default snapshot; or snapshot("eng-base")
+    .network("eng")
+    .label("team", "eng")
+    .env("NODE_ENV", "development")
+    .packages("apt-get update", "apt-get install -y nginx")
+    .with_(github(shared="eng-bot"), claude())
+)
+
+mike = engineering.size("2x8").with_(slack(), jira(shared="jira"), secrets("NPM_TOKEN"))
+m = boxd.machines.create("mike-dev", org="acme", spec=mike)
+m.spec           # the spec as it stands (update_spec, and label, network, egress and size edits, update it)
+m.created_spec   # the spec it was created with, never changed
+mike.resolve()   # the same spec as a dict, without creating anything
+```
+
+| Builder | Sets |
+|---|---|
+| `.from_(team(name))` / `.from_(snapshot(name))` | `base`. Omit for the default image. |
+| `.size("1x4" \| "2x8" \| "4x16" \| …)` | `size`, `<vcpu>x<memory GiB>` |
+| `.assign_to(user_id)` | `assignTo`: create the machine for another member of the org (below) |
+| `.label(key, value)` / `.label({...})` | `labels` |
+| `.network(*names)` | `networks` |
+| `.isolated()` | `isolated` |
+| `.egress(*hosts)` | `egress`: the machine's egress allowlist from birth |
+| `.env(key, value)` / `.env({...})` | `env`: wins over org and personal vars of the same name |
+| `.packages(*commands)` | `packages`: shell commands run once after the first boot (below) |
+| `.with_(*pieces)` | merges pieces or other builders |
+
+Pieces: `slack()`, `jira()`, `github()`, `asana()` (pass `shared="tag"` to use
+the org connection shared under that tag instead of the owner's own),
+`claude()`, `codex()`, and `secrets(*names)` (below). Integrations and agent logins are recorded on the
+machine; a missing one never blocks the create (see "Integrations the machine
+needs"). A piece is a plain dict (`{"labels": {...}}`, or
+`{"connections": {"notion": "personal"}}` for any toolkit), so you can write your own.
+
+Pieces merge in order: maps (`labels`, `env`, `connections`) merge with the
+later piece winning, lists (`networks`, `egress`, `secrets`, `agents`,
+`packages`) combine in order without duplicates, `isolated` only turns on, and `base`, `size` or `assignTo`
+set twice to different values raises `MachineSpecError` (a `ValueError`). So
+does a key a spec doesn't have, and a name in both `env` and `secrets`. The server checks every field and names it in its error
+(`spec.size: ...`).
+
+A spec carries its machine's base, size, labels, networks, isolation and env,
+so `create(spec=...)` takes none of `image`, `env`, `labels`, `networks`,
+`isolated`, `vcpu` or `memory` beside it. A spec whose base is a snapshot (or a
+team with a default snapshot) restores it. A snapshot keeps its machine's spec:
+restoring it gives the new machine that spec, and
+`create(from_snapshot=..., spec=...)` merges a spec on top by the same rules;
+its `size`, if set, must be the snapshot's. A fork keeps its source's spec.
+
+#### Secrets
+
+A spec names the secrets its machine gets; their values stay in
+`boxd.secrets`. Usually they are `unattached` secrets, which reach no machine
+until a spec lists them:
+
+```python
+boxd.secrets.set("NPM_TOKEN", "npm_…", scope="unattached", org="acme")
+
+m = boxd.machines.create("mike-dev", org="acme", spec=engineering.with_(secrets("NPM_TOKEN")))
+```
+
+A name resolves, on your own machine, to your own secret first
+(`private-unattached`, then `private`), else the org's (`unattached`, `all`,
+`shared`). On a machine created for someone else, or a shared machine, only
+the org's secrets count. Only org owners and admins may list an org
+`unattached` secret in a spec, and an org `shared` secret only reaches
+shared machines, so a private machine can't list one. A name that resolves to
+nothing, or that the create may not list, fails the create, naming it.
+
+Only org owners and admins may set, move or delete an org `unattached` secret;
+your own (`private-unattached`) are yours. From inside a machine (with its
+in-VM credentials) a spec can't list an unattached secret, yours or the org's,
+on any machine, and `update_spec` can't change that machine's own `egress` or
+`secrets`; make those changes from outside the machine, with an API key or in
+the console. Names a fork or a restore takes over from its source's spec are
+not affected.
+
+On the machine, the spec's `env` and its `secrets` win over your `private`
+vars, which win over the org's `shared` vars (shared machines only), which win
+over the org's `all` vars. A name can't be in both `env` and `secrets`.
+
+The machine keeps a reference, not a copy: rotating the secret reaches it, and
+deleting the secret takes it away
+(setting it again with the same name brings it back). Forks, snapshots and
+restores resolve the
+names again, and fail naming any that no longer resolve. A secret's
+`bound_machines` says how many machines list it.
+
+#### Creating a machine for someone else
+
+An org owner or admin can create a machine for another member of the org with
+`.assign_to(user_id)` (the member's user id, as `orgs.get_members()` lists it):
+
+```python
+m = boxd.machines.create(
+    "mike-dev",
+    org="acme",                          # required: the org the machine is for
+    spec=engineering.size("2x8").assign_to("usr_e393f70e…"),
+)
+```
+
+The machine is the member's from the start: private to them, billed to the
+org, and set up with their own integrations and agent logins. You can't see or
+reach it afterwards, so `create()` answers with its id and name only, and the
+create can't also set `delete_after`. The member sees `created_by` (your user
+id) and `created_by_name` on it. Refused, naming `spec.assignTo`: without
+`org`, with `shared=True`, for a caller who isn't an owner or admin of the org,
+with an org API key or from inside a machine, for someone who isn't a member,
+and with a snapshot or disk the member couldn't use. Your own id is a plain
+create. `assignTo` is never inherited: forks, snapshots and restores drop it.
+
+The machine runs the spec you wrote as the member's own machine: its packages
+as root, its env and its base, alongside the member's GitHub token, agent
+logins and personal connections. Assign a spec only if the member would run it
+themselves. The member sees who created it (`created_by`) and can read the
+spec (`machine.spec`).
+
+#### Integrations the machine needs
+
+A spec's `connections` and `agents` are the machine's requirements. Reading the
+machine (`get`, `list`, the machine `create` returns) tells you, for each one,
+whether it is connected right now:
+
+```python
+m = boxd.machines.get("mike-dev")
+for r in m.requirements:
+    # kind="connection" name="jira" scope="shared" tag="jira" met=False can_fix=True reason=None
+    print(r.name, r.scope, r.tag, "connected" if r.met else "connect it" if r.can_fix else r.reason or "ask an org admin")
+```
+
+A `personal` connection is the machine owner's own connection for that toolkit
+in the machine's org (on a machine created with `assign_to`, the member's), a
+`shared` one the org's connection shared under `tag`, and an agent the owner's
+Claude Code or Codex sign-in. `can_fix` says whether you could connect it: the
+owner for personal ones and agents, an org owner or admin for shared ones.
+Connect from the console: the machine's page lists them with a Connect button.
+
+An unmet requirement is only unavailable on the machine; nothing blocks
+creating, booting or using it. Some can't be met on the machine as it is, and
+`reason` says why: an isolated machine gets no integrations or agent logins,
+and a shared machine never gets anyone's personal connection or agent login
+(require `shared:<tag>` instead). The create refuses a toolkit this cluster
+can't connect (`spec.connections.<toolkit>: ...`), and the GitHub App
+(`github-app`) only as `"shared:org"`.
+
+#### Packages
+
+`packages` are shell commands the machine runs once, in order, after its first
+boot: as root, with bash (sh on an image without it),
+`DEBIAN_FRONTEND=noninteractive`, no stdin, the env and secrets an exec
+session on the machine gets, and a 30-minute timeout each. Their output is
+appended to `/var/log/boxd/packages.log` in the machine, which only root can
+read. A command is done when its shell exits, so one that leaves a process
+running in the background (`my-server &`) doesn't hold up the next. The first
+failure (a non-zero exit, a timeout) stops the rest; the machine stays usable,
+and `update_spec(id, {}, retry_packages=True)` runs the failed command and the ones after it again (so does
+Retry in the console's Packages section). To give up on a command instead, or
+to replace a wrong one, remove it from the spec: a command that leaves the spec
+before it finished is skipped, and the ones after it run. Reading the machine
+tells you how each one went:
+
+```python
+m = boxd.machines.get("mike-dev")
+for p in m.packages:
+    # command="apt-get install -y nginx" state="failed" exit_code=100 output_tail="E: Unable to locate…" reason=None …
+    print(p.state, p.exit_code, p.command)
+```
+
+`state` is `"pending"`, `"running"`, `"done"`, `"failed"` or `"skipped"`; the
+commands after a failure stay `"pending"`. A pending or failed command that an
+`update_spec` removes from the spec becomes `"skipped"`: it never runs and no
+longer stops the ones after it, which run right away (or when the machine next
+runs). A skipped one that failed keeps its `exit_code` and `output_tail`, and
+`reason` says when it was skipped. Listing it in the spec again puts it back to
+`"pending"`, in its place. So replacing a broken command is one patch: drop it
+and add the corrected one, and the corrected one runs, with everything that
+was waiting. A running command can't be removed (the patch is refused until it
+ends), and removing a done one undoes nothing. `output_tail` is the last 4 KiB of the
+command's output, with the value of every secret the machine gets shown as
+`[secret NAME]` (only the value as it is: printed encoded or in pieces, it isn't
+recognised). The log file isn't masked: it is root's, and root on the machine
+can read its env anyway. `output_tail` is empty on the machines `list()` returns,
+which leave it out to keep the list small; `get()` has it. `reason` says why a failed one has no exit code (it
+timed out, or was interrupted too often), or why a pending one is waiting. A
+command the platform interrupts (the machine stops, reboots, hibernates or
+moves, or boxd restarts on the machine's host) goes back to `"pending"` and runs
+again from the start once the machine is running, and the rest follow; after 3
+interruptions it fails. What finished never runs again: not on a reboot, a
+resume, a wake or a move. Restoring a snapshot marks done the commands that had
+run on the machine the snapshot was taken from and runs the snapshot spec's
+others (one that was running when the snapshot was taken runs again from the
+start), then the ones the restore's spec adds; a fork keeps its source's states
+and runs nothing. The machine doesn't auto-suspend or
+hibernate while a command runs. At most 50 commands (a restore counts the
+snapshot's too), each at most 4 KiB; the server names the one it refuses
+(`spec.packages[2]: ...`). Commands that download something need the hosts in
+the spec's `egress`, if it has one.
+
+A spec has size limits, checked before anything else and named with what was
+given (`spec.env: at most 128 variables (got 129)`): at most 256 KiB as JSON
+in all; `env` at most 128 variables, each value at most 32 KiB and all values
+together at most 64 KiB; `secrets` at most 64 names; `networks` at most 16, each
+name at most 63 bytes; `egress` at most 256 entries, none empty (leave `egress`
+out for no allowlist); `connections` at most 64; `packages` at most 200 listed
+and 50 once duplicates are dropped. An `update_spec` patch is at most 256 KiB.
+
+#### Updating a machine's spec
+
+`update_spec` changes the spec of a machine created from one and applies what
+changes. Pass a JSON Merge Patch, or a function that adds to the current spec:
+
+```python
+# Patch: a field left out is untouched, a scalar replaces, maps merge key by
+# key (None removes a key), a list replaces the whole list, None removes a field.
+boxd.machines.update_spec("mike-dev", {
+    "labels": {"owner": "mike", "team": None},
+    "networks": ["eng", "ops"],
+    "packages": ["apt-get install -y nginx", "npm i -g pnpm"],
+})
+
+# Builder: reads the spec, adds with the composition rules, sends the difference.
+boxd.machines.update_spec("mike-dev", lambda s: s.network("ops").env("LOG_LEVEL", "debug"))
+
+# Dry run: the diff, nothing changed.
+diff = boxd.machines.update_spec("mike-dev", {"size": "4x16"}, dry_run=True)
+for c in diff.changes:
+    print(c.field, c.apply, c.note, c.warnings)
+if diff.reboot_required:
+    boxd.machines.update_spec("mike-dev", {"size": "4x16"}, reboot=True)
+```
+
+Each change in `changes` says how it applies (`apply`):
+
+| Field | `apply` |
+|---|---|
+| `labels`, `networks`, `egress` | `"live"` (new egress rules are in force within about a second of the call returning) |
+| `connections`, `agents` | `"live"`: what the machine needs (`requirements`) changes |
+| `packages` | `"live"`: added commands run now, after the ones before them. Removing one undoes nothing (a warning says so); a pending or failed one the patch removes is skipped (listed in `skipped`), so the ones after it run, and a skipped one listed again goes back to pending in its place (`unskipped`); a command that already ran isn't run again; commands after a failed one don't run (a warning says so) unless the update also retries (`retry_packages=True`: the failed command and the rest run again first, listed in `changed`, skipped ones excepted; a failed command the patch removes is skipped, not retried). Removing a running command is `"refused"` until it ends |
+| `env`, `secrets` | `"reboot_recommended"`: new shells and exec sessions get them now; processes already running keep the old values until they restart (`reboot_recommended`) |
+| `size` | `"reboot"` on a running machine, which reboots to apply it (as `resize`), so it needs `reboot=True`; `"live"` on a stopped one |
+| `base`, `isolated`, `assignTo` | `"refused"`: fixed when the machine is created |
+
+A refused change (also: new packages or a retry on a fork, a size change in standby or
+hibernated) refuses the whole patch, and so does a reboot without
+`reboot=True`; nothing is applied. So is an update during which the spec
+changed (another update, a label edit): it raises `ConflictError`, and
+reading the machine again and retrying is safe. The builder form does that
+itself: it sends the spec it read, so a change made in between can't be
+undone by its lists, and on a conflict reads the spec again and reapplies the
+function, up to 3 times. A patch you built from a spec you read can say so
+too, `expected_spec=machine.spec`: if the spec is no longer that one, nothing
+is applied and it raises `ConflictError`. Otherwise the spec and the labels,
+networks, egress and secrets it drives change together, then the size, then
+the new packages run. If the size then fails (the org's quota changed, the
+machine went to standby), the rest stays applied and the call still returns:
+`resize_error` says why, and `spec` has the size the machine has. The patched spec is checked like a create's (an
+unknown toolkit, a secret you may not list, more than 50 commands in all are
+refused, naming the field). `machine.created_spec` never changes, so the
+console shows the spec badge as "updated". Allowed to whoever may edit the
+machine's labels: its owner, or an org owner or admin on a shared machine.
+The builder form only adds: a `size` that differs from the spec's raises
+(`"2X8"` is the same as `"2x8"`), so
+use a patch to replace or remove. `spec_patch(from_spec, to_spec)` computes the
+patch the builder form sends. `AsyncBoxd` has the same `await
+boxd.machines.update_spec(...)`.
+
 ### Egress allowlist
 
 A machine can be limited to what it may reach on the internet:
@@ -260,13 +539,16 @@ boxd.machines.get("alpha").egress_allow
 boxd.machines.set_egress_allow("alpha", [])   # clear: unrestricted again
 ```
 
-Entries are hostnames or wildcards, or public IPv4 addresses and CIDRs. Web
+Entries are hostnames or wildcards, or public IPv4 addresses and CIDRs, at
+most 256; an internationalized name is kept in its ASCII form (`bücher.de`
+becomes `xn--bcher-kva.de`), which is what the machine looks up. Web
 requests are admitted by hostname, everything else by address, with the
 addresses of allowlisted names learned as the machine resolves them. Nothing
 that is not named is reachable. The hosts of any bound secret the machine
-holds are always admitted. Takes effect within a second and is not visible or
-changeable from inside the machine. A fork inherits its source's allowlist; a
-machine restored from a snapshot starts unrestricted.
+holds are always admitted. Takes effect within a second. From inside the
+machine it can be read (`boxd machine egress`) but not changed. A fork inherits its source's allowlist; a
+machine restored from a snapshot starts unrestricted, unless its machine spec
+lists `egress`.
 
 ### Labels
 
@@ -345,6 +627,15 @@ machine.source.id               # str | None — provenance; may not resolve
 machine.hibernated_at           # datetime | None — None = not hibernated
 machine.last_connected_at       # datetime | None — None = never connected
 machine.boot_time_ms            # int | None — last boot; None = never booted
+machine.spec                    # dict | None — the machine spec it runs under
+machine.created_spec            # dict | None — the spec it was created with
+machine.created_by              # str | None — who created it for you (spec assignTo); None = you did
+machine.created_by_name         # str | None — a name to show for created_by
+machine.requirements            # list[MachineRequirement] — the spec's connections and agents:
+                                #   kind, name, scope ("personal"/"shared"), tag, met, can_fix, reason
+machine.packages                # list[MachinePackage] — the spec's packages and how each went:
+                                #   command, state, exit_code, output_tail, reason, started_at, finished_at
+                                #   (output_tail is empty from list(): get() has it)
 ```
 
 `None` always means "not set": a port that was never allocated, a boot that
@@ -558,23 +849,33 @@ boxd.env.list(org="acme")
 boxd.env.delete("MODE", scope="all")
 
 boxd.secrets.set("API_TOKEN", "s3cr3t", scope="shared")
-boxd.secrets.list()                # Secret(name, scope, domains) — no value
+boxd.secrets.list()                # Secret(name, scope, domains, bound_machines) — no value
 boxd.secrets.delete("API_TOKEN", scope="shared")
 ```
 
 A secret can be bound to the hosts it may be sent to. The machine then only
-ever holds a placeholder, and the real value is substituted into requests to
-those hosts on the way out:
+ever holds a placeholder, and the real value is substituted into HTTPS
+requests to those hosts on the way out:
 
 ```python
 boxd.secrets.set("STRIPE_KEY", "sk_live_...", scope="all", domains=["api.stripe.com"])
 ```
 
-Inside the machine, `STRIPE_KEY` is an opaque `bxds_…` string. A request to
-`api.stripe.com` carrying it, in a header, the query, the body or Basic auth,
-arrives at Stripe with the real key; a request anywhere else carries the
-useless placeholder. Nothing to configure in the machine, and the placeholder
-does not change when the value is rotated. `domains` is the full desired set:
+Inside the machine, `STRIPE_KEY` is an opaque `bxds_…` string. An HTTPS
+request to `api.stripe.com` carrying it anywhere (the URL, a header, Basic auth
+included, or the body) arrives at Stripe with the real key. It is replaced only
+over HTTPS, and only when both the server name the connection was opened for
+and the request's own `Host` are hosts the secret is bound to: over plain HTTP
+and in a request to any other host it stays the useless placeholder. A body
+over 16 MiB with a `Content-Length` is sent as written, placeholder included.
+Responses on those connections carry the placeholder instead of the value:
+wherever the real value (8 characters or more) appears in a response's headers
+or body exactly as it was sent, it is replaced, so a server that echoes the
+request doesn't hand the value to the machine. A value the server sends back
+re-encoded (base64, URL- or JSON-escaped) isn't caught, and neither is one in
+a body compressed with anything but gzip or deflate (the proxy asks for an
+uncompressed response). Nothing to configure in the machine, and the
+placeholder does not change when the value is rotated. `domains` is the full desired set:
 setting the secret again without it makes it a plain secret. Wildcards such as
 `*.stripe.com` are accepted; provider-wide ones such as `*.amazonaws.com` are
 refused.
@@ -595,6 +896,13 @@ Scope decides which machines a name applies to:
 | `private` | only your machines in that organization |
 | `shared` | the organization's shared machines |
 | `all` | every machine in the organization |
+| `unattached` | secrets only: just the machines whose spec lists it ([Secrets](#secrets)); the organization's with `org` (set, moved and deleted by its owners and admins only), else your own |
+| `private-unattached` | secrets only: your own unattached secret in the organization named by `org` |
+
+`list` reports the organization's unattached secrets as `unattached` and your
+own as `private-unattached`, each with `bound_machines`, the number of machines
+whose spec lists it (0 for every other scope). `unattached` is for secrets
+only: a plain value a machine needs belongs in its spec's `env`.
 
 Scope is part of a name's identity — the same name can exist in several scopes
 at once — so changing it is a `move` between two addresses, and both ends are
@@ -860,7 +1168,7 @@ from boxd import (
     AuthenticationError,     # no usable credential, or it was rejected
     PermissionDeniedError,   # authenticated, but not allowed
     NotFoundError,
-    ConflictError,           # already exists, or fights the current state
+    ConflictError,           # already exists, fights the current state, or it changed meanwhile
     RateLimitError,          # rate limit or quota
     APIStatusError,          # any other error from the server
     APIConnectionError,      # could not reach the server