devlaunch 0.0.26__tar.gz → 0.0.28__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {devlaunch-0.0.26 → devlaunch-0.0.28}/PKG-INFO +626 -22
- {devlaunch-0.0.26 → devlaunch-0.0.28}/README.md +625 -21
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/devpod_provider.py +22 -5
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/dl.py +1118 -127
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/gh_auth.py +4 -1
- devlaunch-0.0.28/devlaunch/timing.py +410 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/tools.py +477 -58
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/workspace_id.py +3 -2
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/workspace_state.py +5 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/worktree/branch_manager.py +32 -9
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/worktree/config.py +1 -4
- devlaunch-0.0.28/devlaunch/worktree/git_errors.py +35 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/worktree/locks.py +19 -3
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/worktree/migration.py +40 -10
- devlaunch-0.0.28/devlaunch/worktree/repo_manager.py +640 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/worktree/storage.py +10 -3
- devlaunch-0.0.28/devlaunch/worktree/workspace_clone.py +880 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/pyproject.toml +14 -1
- devlaunch-0.0.26/devlaunch/timing.py +0 -91
- devlaunch-0.0.26/devlaunch/worktree/repo_manager.py +0 -383
- devlaunch-0.0.26/devlaunch/worktree/workspace_clone.py +0 -566
- {devlaunch-0.0.26 → devlaunch-0.0.28}/.gitignore +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/LICENSE +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/__init__.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/aid.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/completion.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/completion_loader.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/completions/__init__.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/completions/dl.bash +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/devpod_ssh.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/disk_usage.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/tty_session.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/worktree/__init__.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/worktree/models.py +0 -0
- {devlaunch-0.0.26 → devlaunch-0.0.28}/devlaunch/xdg.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: devlaunch
|
|
3
|
-
Version: 0.0.
|
|
3
|
+
Version: 0.0.28
|
|
4
4
|
Summary: DevLaunch - A streamlined CLI for devpod workspaces
|
|
5
5
|
Project-URL: Source, https://github.com/blooop/devlaunch
|
|
6
6
|
Project-URL: Home, https://github.com/blooop/devlaunch
|
|
@@ -208,12 +208,23 @@ this is a plain rename: branches, history and **uncommitted changes all survive*
|
|
|
208
208
|
the folder name changes. `metadata.json` is updated in the same pass, so nothing is left
|
|
209
209
|
pointing at the old name.
|
|
210
210
|
|
|
211
|
-
**Your existing devpod containers keep their old ids and are orphaned
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
211
|
+
**Your existing devpod containers keep their old ids and are orphaned — and they can
|
|
212
|
+
often be repaired rather than replaced.** An orphaned container is sourced at the path this
|
|
213
|
+
migration just renamed, with the real clone sitting next to it under the new name, which is
|
|
214
|
+
precisely what [`dl --reconcile`](#reconciling-records-that-disagree) is for: it re-points
|
|
215
|
+
devpod's record at the renamed clone, and `dl <workspace> recreate` finishes the repair.
|
|
216
|
+
That gives you back the clone association and the workspace's identity — not state that
|
|
217
|
+
lived only inside the old container, which nothing can bring back. The repair is
|
|
218
|
+
order-dependent: relaunching the branch claims the renamed clone for a fresh container,
|
|
219
|
+
and reconcile never re-points a clone a live container holds — so reconcile first, then
|
|
220
|
+
relaunch. Left alone, the next `dl user/repo@branch` simply builds a fresh container
|
|
221
|
+
under the new id, and deleting the old one is all that remains for it.
|
|
222
|
+
|
|
223
|
+
dl does not delete containers for you — deleting by id is how a running sidecar got
|
|
224
|
+
destroyed the last time something tried ([kinisi_ros#9766](https://github.com/kinisi-robotics/kinisi_ros/pull/9766)) —
|
|
215
225
|
so it prints a one-line notice with the count and writes the old ids to
|
|
216
|
-
`~/.cache/devlaunch/orphaned-workspaces.txt`.
|
|
226
|
+
`~/.cache/devlaunch/orphaned-workspaces.txt`. For the workspaces you are finished with,
|
|
227
|
+
the disposal command reads from that listing:
|
|
217
228
|
|
|
218
229
|
```bash
|
|
219
230
|
xargs -r -n1 devpod delete < ~/.cache/devlaunch/orphaned-workspaces.txt
|
|
@@ -229,7 +240,10 @@ Running dl again changes nothing: the migration is keyed on the `version` field
|
|
|
229
240
|
`metadata.json`, not on directory names, so a branch that happens to look like a new-scheme
|
|
230
241
|
id is never mistaken for one. If a migration is interrupted, the next run finishes it — the
|
|
231
242
|
version is written last, in the same atomic save as the new paths, so it never claims more
|
|
232
|
-
than the filesystem has actually done.
|
|
243
|
+
than the filesystem has actually done. A rename the filesystem refuses (a read-only mount,
|
|
244
|
+
tightened permissions) is treated the same way: the version stays put and every later run
|
|
245
|
+
retries the refused directories and repeats the notice until the underlying refusal is
|
|
246
|
+
fixed by hand.
|
|
233
247
|
|
|
234
248
|
## Workspace Commands
|
|
235
249
|
|
|
@@ -251,11 +265,43 @@ than the filesystem has actually done.
|
|
|
251
265
|
|--------|-------------|
|
|
252
266
|
| `--devcontainer <variant\|path>` | Use a non-default `devcontainer.json`. A bare name means `.devcontainer/<name>/devcontainer.json`. Stored with the workspace, so pass it once. |
|
|
253
267
|
| `DEVLAUNCH_NO_TTY=1` | Never give a workspace command a terminal; always use the plain `devpod ssh` transport. |
|
|
268
|
+
| `DEVLAUNCH_DOTFILES_ON_ATTACH=1` | Refresh dotfiles before handing over an interactive shell. Off by default; see below. |
|
|
254
269
|
|
|
255
270
|
Projects with demanding devcontainers — several variants, compose sidecars, or a
|
|
256
271
|
host-side `initializeCommand` that has to tell branch workspaces apart — are
|
|
257
272
|
covered in [docs/devcontainer-projects.md](docs/devcontainer-projects.md).
|
|
258
273
|
|
|
274
|
+
### Refreshing dotfiles on attach
|
|
275
|
+
|
|
276
|
+
devpod applies dotfiles when it *provisions* a workspace, so a workspace that has
|
|
277
|
+
been up for a fortnight still has the dotfiles it was born with. `dl <ws>
|
|
278
|
+
dotfiles` fixes that when you think of it; `DEVLAUNCH_DOTFILES_ON_ATTACH=1` makes
|
|
279
|
+
`dl` think of it for you, running the same refresh just before it hands you the
|
|
280
|
+
shell.
|
|
281
|
+
|
|
282
|
+
```bash
|
|
283
|
+
DEVLAUNCH_DOTFILES_ON_ATTACH=1 dl someone/repo
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
It is off unless you set it, and that is the point rather than caution. The
|
|
287
|
+
refresh is a `devpod ssh` round-trip — measured at ~1.7s, almost all of it
|
|
288
|
+
connection setup — with a `git pull` behind it, and it would otherwise be charged
|
|
289
|
+
to every attach on every machine to close a gap most people do not have.
|
|
290
|
+
|
|
291
|
+
Two things it deliberately does not do:
|
|
292
|
+
|
|
293
|
+
- **It never runs for `dl <ws> -- <command>`.** A one-shot command renders no
|
|
294
|
+
prompt and sources no interactive shell, so refreshing in front of it would
|
|
295
|
+
buy that command nothing and cost it the round-trip. That path is the one
|
|
296
|
+
agent launchers use, and it stays exactly as fast as it was.
|
|
297
|
+
- **It never holds the shell hostage.** The refresh gets 60 seconds; an
|
|
298
|
+
unreachable dotfiles remote, or one that wants a password nobody is there to
|
|
299
|
+
type, means a pause and then your shell, not a hang. Failure is a warning —
|
|
300
|
+
you get the workspace either way.
|
|
301
|
+
|
|
302
|
+
Refreshes run every time you attach, with no cooldown, because you asked for
|
|
303
|
+
them. If that is too often, unset the variable and use `dl <ws> dotfiles`.
|
|
304
|
+
|
|
259
305
|
## GitHub Authentication
|
|
260
306
|
|
|
261
307
|
Every workspace `dl` opens inherits the host's GitHub login, so `gh` is already
|
|
@@ -318,7 +364,15 @@ would not be a guarantee.
|
|
|
318
364
|
|
|
319
365
|
On `devpod up`, at most three round trips, each one earning the next.
|
|
320
366
|
|
|
321
|
-
**1.
|
|
367
|
+
**1. The setup pass — the only trip a ready workspace ever pays.** One trip
|
|
368
|
+
carries everything the host wants done on the way into a running container: the
|
|
369
|
+
stages first, then the probe. Naming the container — the hostname your shell
|
|
370
|
+
prompt shows — is the one stage today, and it costs nothing extra because the
|
|
371
|
+
probe was paying for the trip anyway. Each stage reports `ok`, `failed` with its
|
|
372
|
+
exit status, or *not reached*; one that fails stops neither the stages behind it
|
|
373
|
+
nor the probe, and `dl` says which one it was.
|
|
374
|
+
|
|
375
|
+
The probe is the tail of that trip. The container reports
|
|
322
376
|
what only it can know: whether both tools answer at all, where its `claude`
|
|
323
377
|
resolves to, and where `~/.local/share/claude/versions` in its own home resolves
|
|
324
378
|
to. It reports those and names no verdict; the host reads them, so "a real
|
|
@@ -410,12 +464,191 @@ DEVLAUNCH_NO_TOOLS=1 dl someone/repo
|
|
|
410
464
|
|
|
411
465
|
| Variable | Description |
|
|
412
466
|
|----------|-------------|
|
|
413
|
-
| `DEVLAUNCH_NO_TOOLS=1` | Do not install `gh` or `claude` into workspaces |
|
|
467
|
+
| `DEVLAUNCH_NO_TOOLS=1` | Do not install `gh` or `claude` into workspaces. The setup pass still runs — one trip per `up`, which still names the container; only the installing is skipped |
|
|
414
468
|
|
|
415
469
|
Attaching to a workspace that is *already running* skips `devpod up`, and so skips
|
|
416
470
|
this too. A workspace started by something other than `dl` — or created before this
|
|
417
471
|
existed — picks the tools up on its next `dl <workspace> restart`.
|
|
418
472
|
|
|
473
|
+
## A terminal beside the agent
|
|
474
|
+
|
|
475
|
+
Every workspace `dl` opens also has [zellij](https://zellij.dev) on `PATH`, which
|
|
476
|
+
buys one thing the other tools do not: an agent running in a container can open a
|
|
477
|
+
**second terminal next to itself**, in the same container, and you can attach to it
|
|
478
|
+
from anywhere to watch or to type.
|
|
479
|
+
|
|
480
|
+
Nothing has to cooperate for this. It does not come from your dotfiles, it does not
|
|
481
|
+
need an edit to any repo's `devcontainer.json`, and it works in images `dl` has never
|
|
482
|
+
seen — the same argument the rest of "Tools in every workspace" makes, for the same
|
|
483
|
+
reason: `dl` launches arbitrary repos.
|
|
484
|
+
|
|
485
|
+
### Opening a pane from inside a session
|
|
486
|
+
|
|
487
|
+
From anywhere inside the container — including from a completely non-interactive
|
|
488
|
+
command, with no terminal attached to anything:
|
|
489
|
+
|
|
490
|
+
```bash
|
|
491
|
+
zellij -s devlaunch action new-pane -- htop
|
|
492
|
+
```
|
|
493
|
+
|
|
494
|
+
`-s <name>` is the form to use and the only one worth depending on. Bare
|
|
495
|
+
`zellij action new-pane` happens to work by falling back to the single running
|
|
496
|
+
session, which stops being a single session the moment there are two of them.
|
|
497
|
+
|
|
498
|
+
`devlaunch` is the session name `dl` creates and the one to name here.
|
|
499
|
+
|
|
500
|
+
### Switching the wrap on
|
|
501
|
+
|
|
502
|
+
The session an agent opens panes into has to exist first, and creating it is
|
|
503
|
+
**off by default**:
|
|
504
|
+
|
|
505
|
+
```bash
|
|
506
|
+
DEVLAUNCH_ZELLIJ=1 dl someone/repo -- claude -p "do the thing"
|
|
507
|
+
```
|
|
508
|
+
|
|
509
|
+
| Variable | Description |
|
|
510
|
+
|----------|-------------|
|
|
511
|
+
| `DEVLAUNCH_ZELLIJ=1` | Before running `dl <spec> -- <command>`, make sure a zellij session named `devlaunch` exists in the container, so the command can open panes into it |
|
|
512
|
+
|
|
513
|
+
With it off, no invocation changes meaning at all — that is what off means here, and
|
|
514
|
+
it is why the switch exists rather than the behaviour simply being on.
|
|
515
|
+
|
|
516
|
+
**The command runs beside the session, not inside a pane of it.** That is deliberate.
|
|
517
|
+
Putting the command in a pane would hand its stdin, stdout and exit status to zellij,
|
|
518
|
+
and all three are things `dl` promises to leave alone: `dl <ws> -- cmd > file` has to
|
|
519
|
+
put the command's own output in the file, and a failing command has to come back with
|
|
520
|
+
its own status. Since `zellij -s <name> action new-pane` works perfectly well from a
|
|
521
|
+
command that is in no session at all, running beside the session costs nothing and
|
|
522
|
+
delivers the same pane.
|
|
523
|
+
|
|
524
|
+
**The interactive session of a bare `dl <workspace>` is untouched, switched on or
|
|
525
|
+
off.** An interactive attach sends no command for the wrap to attach to — that is
|
|
526
|
+
exactly what gets it a terminal from devpod — and giving it one would cost either the
|
|
527
|
+
terminal or a round trip in front of every shell. You land in an ordinary login shell
|
|
528
|
+
with `zellij` on `PATH`, so `zellij attach -c devlaunch` gets you the session, and any
|
|
529
|
+
panes an agent has opened in it, whenever you want them.
|
|
530
|
+
|
|
531
|
+
There is one exception, and it is a pleasant one: if you also run with
|
|
532
|
+
`DEVLAUNCH_DOTFILES_ON_ATTACH=1`, that refresh is a command, so it gets wrapped like
|
|
533
|
+
any other and the session is already there when the shell arrives.
|
|
534
|
+
|
|
535
|
+
### Existing workspaces
|
|
536
|
+
|
|
537
|
+
zellij arrives on the setup pass, which runs on every `devpod up`. So a workspace
|
|
538
|
+
that predates this picks it up on its next **`dl <workspace> restart`** — a full
|
|
539
|
+
`dl <workspace> recreate` also works but is not needed, because nothing here is a
|
|
540
|
+
bind mount and mounts are the thing that only lands at container creation.
|
|
541
|
+
|
|
542
|
+
Attaching to a workspace that is *already running* skips `devpod up` and so skips
|
|
543
|
+
this too, which is what makes the restart necessary rather than automatic.
|
|
544
|
+
|
|
545
|
+
### What it costs
|
|
546
|
+
|
|
547
|
+
Almost nothing, and that was measured rather than assumed. zellij is a conda-forge
|
|
548
|
+
package installed by pixi into the container, so it lands in the shared package cache
|
|
549
|
+
above and every container after the first extracts rather than downloads:
|
|
550
|
+
|
|
551
|
+
| | |
|
|
552
|
+
|---|---|
|
|
553
|
+
| **Warm install** (shared cache populated) | 0.56s / 0.23s / 0.23s over three fresh containers |
|
|
554
|
+
| **Cold install** (empty cache) | 3.0s, filling 167MB of shared cache |
|
|
555
|
+
| **Every launch after the first** | one `command -v`; the whole setup pass measured at 50ms |
|
|
556
|
+
|
|
557
|
+
**It can never fail a launch.** Provisioning zellij is a stage of the setup pass, so a
|
|
558
|
+
container with no network, no pixi and no way to get either reports the stage as
|
|
559
|
+
failed, by name, and then opens exactly as it would have. A container that ends up
|
|
560
|
+
without zellij still works; with the wrap on, the command still runs, because the
|
|
561
|
+
session setup is allowed to fail and the command runs regardless.
|
|
562
|
+
|
|
563
|
+
`DEVLAUNCH_NO_TOOLS=1` turns this off along with the rest of tool provisioning —
|
|
564
|
+
installing zellij is tool provisioning, where naming a container is not.
|
|
565
|
+
|
|
566
|
+
## The shared pixi package cache
|
|
567
|
+
|
|
568
|
+
Every container `dl` creates gets one host directory bound into it, and
|
|
569
|
+
`PIXI_CACHE_DIR` pointed at it, so that dotfiles which provision their tools with
|
|
570
|
+
`pixi global sync` download each package once per machine instead of once per
|
|
571
|
+
container:
|
|
572
|
+
|
|
573
|
+
| | |
|
|
574
|
+
|---|---|
|
|
575
|
+
| **On the host** | `$XDG_CACHE_HOME`, or `~/.cache`, then `devlaunch/pixi` |
|
|
576
|
+
| **In the container** | `/var/tmp/devlaunch-pixi` |
|
|
577
|
+
|
|
578
|
+
Measured on the profile this was built for — 23 pixi-global environments — a
|
|
579
|
+
container with a cold cache spends 62–113 s and downloads 1.2 GB; one that finds
|
|
580
|
+
the packages already there finishes in 18–28 s and fetches nothing. Two containers
|
|
581
|
+
syncing against it at the same time is fine: the downloads are content-addressed
|
|
582
|
+
and rattler takes a lock per package.
|
|
583
|
+
|
|
584
|
+
**Deleting it is always safe, at any moment, including while containers are
|
|
585
|
+
running.** It holds nothing but downloaded package archives — every one of them
|
|
586
|
+
re-fetchable from the network, and none of them referenced by a path anything
|
|
587
|
+
inside a container has stored. The worst a deletion costs is the next container's
|
|
588
|
+
download.
|
|
589
|
+
|
|
590
|
+
```bash
|
|
591
|
+
rm -rf ~/.cache/devlaunch/pixi
|
|
592
|
+
```
|
|
593
|
+
|
|
594
|
+
`dl --purge` takes it away with the rest of `~/.cache/devlaunch/`, for the same
|
|
595
|
+
reason.
|
|
596
|
+
|
|
597
|
+
Two things it deliberately is not. It is **not the host's own**
|
|
598
|
+
`~/.cache/rattler/cache`: containers write into it as their own remote user,
|
|
599
|
+
whose uid only happens to match yours, and a `pixi clean cache` you run for your
|
|
600
|
+
own reasons must not be able to pull packages out from under a running container.
|
|
601
|
+
And it is **not a shared `PIXI_HOME`** — the installed environments and their
|
|
602
|
+
trampolines are baked with absolute paths, and two containers sharing one
|
|
603
|
+
environment tree is [pixi#5476](https://github.com/prefix-dev/pixi/issues/5476).
|
|
604
|
+
Only the download cache is shared, which is the part that is safe to share.
|
|
605
|
+
|
|
606
|
+
If the directory cannot be created, or is not there when the launch reaches it —
|
|
607
|
+
a full disk, a read-only cache home, a cache swept between the two — the launch
|
|
608
|
+
goes ahead without the mount and the container downloads its own packages,
|
|
609
|
+
exactly as it did before this existed.
|
|
610
|
+
|
|
611
|
+
**Sharing requires the container's user to be able to write the directory**,
|
|
612
|
+
which in practice means its uid matches yours or it is root. The mount carries
|
|
613
|
+
host ownership through unchanged, and pixi does not degrade to reading a cache
|
|
614
|
+
it cannot write: pointing `PIXI_CACHE_DIR` at a directory owned by another uid
|
|
615
|
+
fails the install outright (`Permission denied` on the repodata, exit 1) even
|
|
616
|
+
when every package it wants is already in there. So an image whose remote user
|
|
617
|
+
is neither root nor your uid does not merely lose the sharing — its `pixi global
|
|
618
|
+
sync` fails, and its tools do not get provisioned.
|
|
619
|
+
|
|
620
|
+
`dl` cannot see the container's uid before it launches, so it cannot decide this
|
|
621
|
+
for you. In practice the common case is safe: every mainstream base declares a
|
|
622
|
+
remote user at uid 1000, which is the first human user on a Linux host. If you
|
|
623
|
+
hit the failure, the fixes available to you are to run that image as your own
|
|
624
|
+
uid, or to take the cache out of play for it (`rm -rf ~/.cache/devlaunch/pixi`
|
|
625
|
+
recovers a directory an earlier container left owned by someone else).
|
|
626
|
+
|
|
627
|
+
### Existing containers, and what a recreate is for
|
|
628
|
+
|
|
629
|
+
**A mount lands only when a container is created.** devpod re-applies
|
|
630
|
+
`--workspace-env` on every `up`, but it will not add a bind mount to a container
|
|
631
|
+
that already exists — passing `--mount` there is a silent no-op. So a container
|
|
632
|
+
built before this feature, or before a change to where the mount lands, keeps
|
|
633
|
+
whatever it was created with until `dl <workspace> recreate`, and only then
|
|
634
|
+
picks the current arrangement up.
|
|
635
|
+
|
|
636
|
+
In between, `PIXI_CACHE_DIR` points at `/var/tmp/devlaunch-pixi` with nothing
|
|
637
|
+
mounted on it. That is a working private cache, not a failure — `/var/tmp` is
|
|
638
|
+
world-writable in every image, so pixi creates the directory and fills it. The
|
|
639
|
+
container re-warms itself and simply never shares, abandoning whatever pixi had
|
|
640
|
+
already warmed in its default location. **This is the reason the container-side
|
|
641
|
+
path is under `/var/tmp` rather than somewhere tidier like `/var/cache`:** a
|
|
642
|
+
target whose parent is root-owned is a hard `pixi global sync` failure on every
|
|
643
|
+
container that predates it, not a lost optimisation.
|
|
644
|
+
|
|
645
|
+
One older breakage needs the recreate rather than a restart. Devlaunch briefly
|
|
646
|
+
mounted this cache inside `~/.cache`, which left that directory root-owned in
|
|
647
|
+
any image that ships no `~/.cache` of its own. `$HOME` lives on the container's
|
|
648
|
+
own layer, so `dl <workspace> stop` and a fresh `up` keep the root-owned
|
|
649
|
+
directory; `dl <workspace> recreate` gets a new layer where `~/.cache` is the
|
|
650
|
+
user's own again.
|
|
651
|
+
|
|
419
652
|
## Global Commands
|
|
420
653
|
|
|
421
654
|
| Command | Description |
|
|
@@ -425,6 +658,7 @@ existed — picks the tools up on its next `dl <workspace> restart`.
|
|
|
425
658
|
| `dl --ls --size` | Add [what deleting each workspace would free](#how-much-disk-a-workspace-costs). Opt-in: it walks every file in the clone |
|
|
426
659
|
| `dl --install` | Install shell completions |
|
|
427
660
|
| `dl --prune [-y] [--force]` | Remove [the clone directories no workspace opens any more](#pruning-the-clones-nothing-opens) — and nothing else |
|
|
661
|
+
| `dl --reconcile [-y]` | Re-point [devpod workspaces whose recorded source folder no longer holds a checkout](#reconciling-records-that-disagree) at the clone that does |
|
|
428
662
|
| `dl --purge [-y]` | Remove all devlaunch data — [the workspaces devlaunch created](#what-purge-deletes), and its caches |
|
|
429
663
|
| `dl --refresh` | Refresh completion cache |
|
|
430
664
|
| `dl --help, -h` | Show this help |
|
|
@@ -462,7 +696,7 @@ Anything it is leaving is named before it asks:
|
|
|
462
696
|
$ dl --purge
|
|
463
697
|
This will remove all devlaunch data:
|
|
464
698
|
- 4 DevPod workspace(s)
|
|
465
|
-
- /home/you/.cache/devlaunch/ (workspace clones, repo caches, completions)
|
|
699
|
+
- /home/you/.cache/devlaunch/ (workspace clones, repo caches, the shared pixi cache, completions)
|
|
466
700
|
|
|
467
701
|
Leaving 2 workspace(s) devlaunch did not create:
|
|
468
702
|
- pythontemplate
|
|
@@ -498,13 +732,36 @@ Removed what was permitted under /home/you/.cache/devlaunch. These refused:
|
|
|
498
732
|
Usually this means a container wrote them as a different user, and:
|
|
499
733
|
sudo rm -rf '/home/you/.cache/devlaunch'
|
|
500
734
|
clears them. Check the reasons above first -- it does not fix all of them.
|
|
735
|
+
devlaunch does not manage Docker images or volumes: the containers these workspaces used may still hold disk, and `docker system df` shows what Docker is holding.
|
|
501
736
|
```
|
|
502
737
|
|
|
738
|
+
That last line ends every purge, including one that found nothing to purge and
|
|
739
|
+
one you answered `n` to — `dl --prune` ends on the same one, in the same words.
|
|
740
|
+
See [the disk neither command frees](#the-disk-neither-command-frees).
|
|
741
|
+
|
|
503
742
|
Exit status is `1`, because a clone you were told would go is still on disk. It
|
|
504
743
|
used to be `1` with the *whole* cache still standing: the first refusal stopped
|
|
505
744
|
the purge, so the completion caches, `metadata.json` and every other clone
|
|
506
745
|
survived on account of one directory.
|
|
507
746
|
|
|
747
|
+
When **none** of it goes — nothing under the cache came away at all, which is
|
|
748
|
+
what a symlinked cache root gives you, or one that cannot even be looked at, or
|
|
749
|
+
one whose every entry refused — the headline says that instead of claiming a
|
|
750
|
+
partial success:
|
|
751
|
+
|
|
752
|
+
```
|
|
753
|
+
$ dl --purge -y
|
|
754
|
+
Removed nothing under /home/you/.cache/devlaunch. These refused:
|
|
755
|
+
- /home/you/.cache/devlaunch: Permission denied
|
|
756
|
+
```
|
|
757
|
+
|
|
758
|
+
The report underneath is the same one, and so is the exit status: `0` means the
|
|
759
|
+
cache is gone and nothing else does, which is the only distinction a script can
|
|
760
|
+
act on. Removed everything, removed what it was permitted to and removed nothing
|
|
761
|
+
are three outcomes rather than two, and the sentence is where the third one
|
|
762
|
+
lives — because it is the one that decides whether you still have clones to go
|
|
763
|
+
and look for.
|
|
764
|
+
|
|
508
765
|
What is listed is the directory, once — not the hundreds of files inside it.
|
|
509
766
|
Unlinking needs write permission on the directory rather than on the file, so
|
|
510
767
|
every entry in that clone refuses separately and they are all the same fact.
|
|
@@ -551,6 +808,7 @@ the next clone of a repo fast), and never looks outside
|
|
|
551
808
|
stub was the only thing anything pointed at. `--prune` will not guess which
|
|
552
809
|
clone such a workspace needs: it keeps every clone of that repository and
|
|
553
810
|
names the record to go and fix. `--force` does not move any of them.
|
|
811
|
+
[`dl --reconcile`](#reconciling-records-that-disagree) is what fixes them.
|
|
554
812
|
|
|
555
813
|
Note that *every* directory two levels under `<cache>/devlaunch/repos` is a
|
|
556
814
|
candidate — a stray directory somebody left there is looked at like any other.
|
|
@@ -612,6 +870,110 @@ It also drops the `metadata.json` records of directories that are already gone.
|
|
|
612
870
|
That file was append-only in practice — 49 records for 17 live workspaces on the
|
|
613
871
|
same host — and this is the first thing that prunes it.
|
|
614
872
|
|
|
873
|
+
#### The disk neither command frees
|
|
874
|
+
|
|
875
|
+
Both commands end on the same line, in the same words:
|
|
876
|
+
|
|
877
|
+
```
|
|
878
|
+
$ dl --prune -y
|
|
879
|
+
...
|
|
880
|
+
Removed 2 clone director(ies) -- 1.4 GiB.
|
|
881
|
+
devlaunch does not manage Docker images or volumes: the containers these workspaces used may still hold disk, and `docker system df` shows what Docker is holding.
|
|
882
|
+
```
|
|
883
|
+
|
|
884
|
+
The gigabytes a cleanup reports are usually not the ones you are looking for. On
|
|
885
|
+
the host this was measured, `--prune` had 4.00 GB of stale clones to give back
|
|
886
|
+
while `docker system df` read **86.5 GB of reclaimable images, 43.18 GB of
|
|
887
|
+
volumes and 13.88 GB of build cache** — an order of magnitude more, sitting
|
|
888
|
+
behind a command that had just said "Removed". Saying nothing is what makes a
|
|
889
|
+
freed figure read as *all* of it, so both commands say this instead, whether they
|
|
890
|
+
removed 40 clones, found nothing to remove, or were answered `n` at the
|
|
891
|
+
confirmation. The report you get for saying `n` is a reason to print it, not an
|
|
892
|
+
exception: that is where somebody is deciding what is worth deleting.
|
|
893
|
+
|
|
894
|
+
**It is a sentence, not a measurement.** `dl` runs no `docker` command to print
|
|
895
|
+
it, so there is nothing to be slow and nothing to fail where Docker is absent,
|
|
896
|
+
stopped, or reachable only as another user. The figures above are this README's,
|
|
897
|
+
from the host it was measured on, not from your machine — `docker system df` is
|
|
898
|
+
where yours are.
|
|
899
|
+
|
|
900
|
+
**And it points rather than offers.** There is deliberately no `dl` flag that
|
|
901
|
+
removes an image, and no list of image ids here to paste into `docker image rm`.
|
|
902
|
+
Images devpod builds carry no devlaunch or devpod label, so any list `dl` printed
|
|
903
|
+
would be a guess at which of them belong to these workspaces, and `docker image
|
|
904
|
+
prune -a` is not scoped to devlaunch at all — it would take images built by
|
|
905
|
+
everything else on the machine. Deleting them is a decision with your own
|
|
906
|
+
containers on the other side of it, and `docker system df` is the tool that shows
|
|
907
|
+
you what it costs.
|
|
908
|
+
|
|
909
|
+
### Reconciling records that disagree
|
|
910
|
+
|
|
911
|
+
`dl` keeps its own record of every workspace, and devpod keeps one too. They
|
|
912
|
+
agree until the naming that connects them moves — and it did move once, when
|
|
913
|
+
workspace ids and clone-directory names gained a hashed suffix. `dl`'s records
|
|
914
|
+
were migrated to the new naming; devpod's were not, because nothing knew to
|
|
915
|
+
touch them. On the host that reported it, **36 of 39 devpod workspaces recorded
|
|
916
|
+
a source folder that was missing, or was a stub with no `.git` in it**, while the
|
|
917
|
+
real checkout sat next to it under the new name. Nothing was deleted and nothing
|
|
918
|
+
was corrupted: `dl` was simply asking devpod about workspaces devpod had never
|
|
919
|
+
been given, and devpod was answering correctly that there were none.
|
|
920
|
+
|
|
921
|
+
Two things fix that, and they are different jobs. `dl` now **writes the devpod
|
|
922
|
+
workspace id down** when it creates a workspace, so the naming can move again
|
|
923
|
+
without taking anything with it — that is automatic and needs no command. It
|
|
924
|
+
does nothing for the records that already disagree, because they were written
|
|
925
|
+
before there was a field to write it in. `dl --reconcile` is for those:
|
|
926
|
+
|
|
927
|
+
```
|
|
928
|
+
$ dl --reconcile
|
|
929
|
+
devpod workspaces sourced under /home/you/.cache/devlaunch/repos at something that is not a clone:
|
|
930
|
+
|
|
931
|
+
Re-pointing 2:
|
|
932
|
+
- devlaunch-main: .../blooop/devlaunch/main -> .../blooop/devlaunch/devlaunch-main-zovomobo
|
|
933
|
+
- bencher-test1: .../blooop/bencher/test1 -> .../blooop/bencher/bencher-test1-pipagito
|
|
934
|
+
|
|
935
|
+
Each of these needs `dl <workspace> recreate` afterwards: the container
|
|
936
|
+
still has the old source bind-mounted, and no record change moves a mount.
|
|
937
|
+
|
|
938
|
+
Leaving 1, which dl will not guess at:
|
|
939
|
+
- rockerc-main (.../blooop/rockerc/main): no clone of that repository answers to this name
|
|
940
|
+
|
|
941
|
+
Nothing here is deleted. `dl <workspace> rm` is how one goes, if it should.
|
|
942
|
+
```
|
|
943
|
+
|
|
944
|
+
It matches the two sides **by path, never by id** — the id is the thing that
|
|
945
|
+
changed, so it connects nothing, while the source folder devpod kept still names
|
|
946
|
+
the owner and the repository exactly, and its last component still names the
|
|
947
|
+
branch in one of the three ways `dl` has named a clone directory. Where that
|
|
948
|
+
match is not unique it is refused rather than guessed: a clone a live workspace
|
|
949
|
+
already opens — at it, or anywhere under it — is never taken from it, a clone two
|
|
950
|
+
dead records both match is claimed by neither, and a name that two clones answer
|
|
951
|
+
to (the old flattened spelling turned `feature/auth`, `feature auth` and
|
|
952
|
+
`feature:auth` all into `feature-auth`) adopts neither of them. If a live
|
|
953
|
+
workspace's source cannot be followed at all, the whole command stops the way
|
|
954
|
+
`dl --prune` does, because such a workspace could be holding any of the clones on
|
|
955
|
+
offer. **Nothing is ever deleted.** A workspace `dl` cannot match
|
|
956
|
+
is named and left exactly where it is, because whether a workspace is finished
|
|
957
|
+
with is not something `dl` can know, and the two mistakes are not the same size.
|
|
958
|
+
|
|
959
|
+
Run it as often as you like — a repaired workspace is no longer sourced at a
|
|
960
|
+
non-checkout, so a second run finds nothing to do.
|
|
961
|
+
|
|
962
|
+
**A re-pointed workspace still needs rebuilding.** Its container was built with
|
|
963
|
+
the dead path bind-mounted into it, and changing a record does not move a mount.
|
|
964
|
+
`dl <workspace> recreate` is what finishes the repair, and it is the step that
|
|
965
|
+
needs Docker.
|
|
966
|
+
|
|
967
|
+
**Do not point an old `dl` at a reconciled cache.** A `dl` from before the naming
|
|
968
|
+
changed derives the old directory name, does not find it, and treats the launch
|
|
969
|
+
as a cold one: it clones a second directory under the old name, registers a
|
|
970
|
+
second devpod workspace, and rewrites that branch's record with the old naming
|
|
971
|
+
and an empty workspace id — undoing the repair for that one workspace, and
|
|
972
|
+
leaving you two clones of the branch. It is not destructive and the next
|
|
973
|
+
`dl --reconcile` sorts it out, but a machine that runs both builds against one
|
|
974
|
+
cache will keep re-breaking. Upgrade the old one, or give it its own
|
|
975
|
+
`XDG_CACHE_HOME`.
|
|
976
|
+
|
|
615
977
|
### Cleaning up workspaces
|
|
616
978
|
|
|
617
979
|
One workspace per branch means workspaces accumulate, and `--purge` is the wrong
|
|
@@ -694,6 +1056,13 @@ That refusal is the only judgement `dl` makes here, and it is not about finished
|
|
|
694
1056
|
work — it is `dl` declining to destroy the only copy of something, including
|
|
695
1057
|
when it cannot prove there is another copy. Say `--force` if you mean it.
|
|
696
1058
|
|
|
1059
|
+
`--force` changes one more answer: an already-absent workspace counts as
|
|
1060
|
+
deleted, like `rm -f`. Unforced, `rm` reports devpod's refusal to delete a
|
|
1061
|
+
workspace it does not have; forced, the contract is the state afterwards, not
|
|
1062
|
+
that a delete happened — which is what lets the [cold benchmark's per-run
|
|
1063
|
+
reset](#measuring-launch-time) run before the first launch, when there is
|
|
1064
|
+
nothing to remove yet.
|
|
1065
|
+
|
|
697
1066
|
The guard reads `dl`'s metadata record, so the recorded directory is the one it
|
|
698
1067
|
asks about. (The delete does not always remove that same directory: when the
|
|
699
1068
|
recorded path is not on disk it falls back to a derived one. That divergence is
|
|
@@ -745,6 +1114,52 @@ by `git clone` from the bare in `dl`'s own cache:
|
|
|
745
1114
|
difference is a single 270,823,424-byte pack file with one link in the clone and
|
|
746
1115
|
one in the bare, so removing either end frees none of it.
|
|
747
1116
|
|
|
1117
|
+
**That sharing is a promise, not a coincidence, and a test holds it to that.**
|
|
1118
|
+
`git clone <path> <path>` hardlinks pack files by default, and the default is
|
|
1119
|
+
all that was ever keeping it true — a `file://` URL, an intermediate copy, or an
|
|
1120
|
+
explicit `--no-hardlinks` would each forfeit it with nothing failing and no
|
|
1121
|
+
warning printed. Measured on this repo — `du -sc` over the cache and each
|
|
1122
|
+
clone's `.git`, ext4, git 2.55.0 — that is 2400 KB for the cache plus one
|
|
1123
|
+
workspace against 4472 KB unshared, and 196 KB rather than 2268 KB of `.git` for
|
|
1124
|
+
every workspace after the first. So an integration test asserts the pack files are
|
|
1125
|
+
the cache's — same inode, more than one link — and that assertion goes red on
|
|
1126
|
+
all three. No clone flag is used to guard it: `--local` is already the default
|
|
1127
|
+
and does not even reject a `file://` source, and `--shared`/`--reference` were
|
|
1128
|
+
measured to leave a workspace that fails `git fsck` once the cache has fetched
|
|
1129
|
+
and gc'd, for a 2 KB saving.
|
|
1130
|
+
|
|
1131
|
+
Sharing does erode, in one measured way that is a safety property rather than a
|
|
1132
|
+
fault: when the cache repacks, an existing workspace's pack loses its second
|
|
1133
|
+
link and becomes that workspace's own complete copy, still passing `git fsck`.
|
|
1134
|
+
The workspace stops being cheap and never stops being valid — which is the trade
|
|
1135
|
+
`--shared` and `--reference` get wrong, and the reason they are not used.
|
|
1136
|
+
|
|
1137
|
+
**Large files are shared the same way, but nothing about `git clone` does it for
|
|
1138
|
+
you.** git-lfs objects are not git objects: the clone does not carry them at
|
|
1139
|
+
all, so a workspace of an LFS repo used to download the entire payload from the
|
|
1140
|
+
forge and keep a private copy of it in `.git/lfs/objects` — every workspace,
|
|
1141
|
+
every time, on top of the worktree copy. `dl` now makes the bare cache the
|
|
1142
|
+
repo's LFS store as well: the payload is fetched once into `<repo>/.bare/lfs`
|
|
1143
|
+
for the branch being launched, and each workspace materializes out of *that*,
|
|
1144
|
+
which git-lfs does by hardlinking. Measured with git-lfs 3.7.1 on ext4: the
|
|
1145
|
+
workspace's object file is the same `(st_dev, st_ino)` as the cache's, so its
|
|
1146
|
+
store costs nothing, and the materialization succeeds with the remote deleted
|
|
1147
|
+
from disk — the second workspace of an LFS repo touches the network for its
|
|
1148
|
+
large files not at all. What remains per workspace is the worktree copy, which
|
|
1149
|
+
is real bytes and cannot be shared: a container build has to be able to read
|
|
1150
|
+
them. If the cache cannot supply an object — a first launch offline, a payload
|
|
1151
|
+
the branch alone introduces — the old download from `origin` still runs, and a
|
|
1152
|
+
workspace left holding pointer files is retried on the next launch rather than
|
|
1153
|
+
written off.
|
|
1154
|
+
|
|
1155
|
+
Nothing about that is written into the workspace's `.git/config`, and that
|
|
1156
|
+
restraint is load-bearing rather than tidy: `dl` bind-mounts the *clone*
|
|
1157
|
+
directory into the devcontainer and `.bare` is a sibling that is not mounted, so
|
|
1158
|
+
an `lfs.storage` entry or an added remote naming a host path would break every
|
|
1159
|
+
`git checkout` of an LFS repo inside the container while working perfectly on
|
|
1160
|
+
the host. A test asserts the clone keeps exactly one remote, still pointing at
|
|
1161
|
+
the forge, and no `lfs.storage` at all.
|
|
1162
|
+
|
|
748
1163
|
So `dl` counts a file only when every one of its hardlinks lies inside the
|
|
749
1164
|
workspace being measured. Two consequences, both deliberate:
|
|
750
1165
|
|
|
@@ -783,7 +1198,9 @@ repo's own does) is most of that count. That is not a bill a listing should
|
|
|
783
1198
|
present unasked.
|
|
784
1199
|
|
|
785
1200
|
Docker images and named volumes are not counted: `dl` did not create the layer
|
|
786
|
-
store and does not manage volumes. `docker system df` is the tool that knows
|
|
1201
|
+
store and does not manage volumes. `docker system df` is the tool that knows —
|
|
1202
|
+
the same boundary [`--prune` and `--purge` name](#the-disk-neither-command-frees)
|
|
1203
|
+
when they finish.
|
|
787
1204
|
|
|
788
1205
|
## Examples
|
|
789
1206
|
|
|
@@ -805,7 +1222,7 @@ dl blooop/devlaunch stop # Stop workspace
|
|
|
805
1222
|
- **GitHub Shorthand**: Use `owner/repo` instead of full URLs - automatically expands to `github.com/owner/repo`
|
|
806
1223
|
- **Branch Support**: Specify branches with `owner/repo@branch` syntax
|
|
807
1224
|
- **Fast Autocomplete**: Completion cache for ~3ms response time (vs ~700ms without cache)
|
|
808
|
-
- **One Round-Trip Per Question**: every `devpod` call costs ~0.45s, far more than `dl` itself, so a command reads the workspace list at most once — and
|
|
1225
|
+
- **One Round-Trip Per Question**: every `devpod` call costs ~0.45s, far more than `dl` itself, so a command reads the workspace list at most once — and everything a container needs on the way in (naming it, then the tools probe) rides one setup pass, so an interactive `dl <ws>` and a one-shot `dl <ws> -- <cmd>` cost the same trips
|
|
809
1226
|
|
|
810
1227
|
## Measuring launch time
|
|
811
1228
|
|
|
@@ -813,25 +1230,197 @@ Set `DEVLAUNCH_TIMING=1` and a `dl` command ends with one summary on stderr,
|
|
|
813
1230
|
naming each subprocess round trip and the total. Unset (or `0`) records nothing
|
|
814
1231
|
and prints nothing.
|
|
815
1232
|
|
|
1233
|
+
Captured from a real warm launch (the launch's own output elided):
|
|
1234
|
+
|
|
816
1235
|
```bash
|
|
817
|
-
$ DEVLAUNCH_TIMING=1 dl
|
|
818
|
-
|
|
819
|
-
dl-timing: devpod
|
|
820
|
-
dl-timing:
|
|
821
|
-
dl-timing:
|
|
1236
|
+
$ DEVLAUNCH_TIMING=1 dl-next blooop/mcp-devtasks -- true
|
|
1237
|
+
...
|
|
1238
|
+
dl-timing: devpod status 0.454s
|
|
1239
|
+
dl-timing: gh auth token 0.036s
|
|
1240
|
+
dl-timing: devpod ssh 1.952s
|
|
1241
|
+
dl-timing: total 2.444s (in-process, excluding interpreter startup)
|
|
822
1242
|
```
|
|
823
1243
|
|
|
1244
|
+
### The same launch, machine-readable
|
|
1245
|
+
|
|
1246
|
+
`DEVLAUNCH_TIMING=json` swaps that prose for one document on a single
|
|
1247
|
+
`dl-timing-json:` line, so a trend job can read a launch without scraping
|
|
1248
|
+
prose. It decomposes the launch into five **ownership-boundary stages** — one
|
|
1249
|
+
per party that could actually make it faster — with the round trips nested
|
|
1250
|
+
inside the stage that paid for them:
|
|
1251
|
+
|
|
1252
|
+
| stage | what it owns |
|
|
1253
|
+
|---|---|
|
|
1254
|
+
| `handoff` | the gap between the keystroke that resolved to this exec and dl starting (see the stamps below) |
|
|
1255
|
+
| `host-prep` | the host's own git work — the bare clone and its fetches, the lock waits, the LFS probe and, for an LFS repo, the cache's LFS fetch and the workspace's materialization out of it — and the `gh auth token` trip, wherever on the launch it falls |
|
|
1256
|
+
| `devpod-up` | the arm that gets a container running: the existence probe and, when it is not running, the `up` itself. On a warm launch that arm is the probe alone |
|
|
1257
|
+
| `tools` | the probe trip and the conditional lend, including staging the payload tar |
|
|
1258
|
+
| `attach` | the last trip, into the running command |
|
|
1259
|
+
|
|
1260
|
+
Two rules are worth knowing before reading one:
|
|
1261
|
+
|
|
1262
|
+
- **A stage that never ran is absent, not zero.** A warm launch reports no
|
|
1263
|
+
`host-prep` at all, because it did none. A stage that failed is present,
|
|
1264
|
+
timed up to the failure, and marked `failed`.
|
|
1265
|
+
- **A stage totals over its whole arm**, not just over its round trips, so the
|
|
1266
|
+
host-side work between two spawns is attributed rather than lost. Stages
|
|
1267
|
+
never double-count each other: `tools` runs inside the launch `devpod-up`
|
|
1268
|
+
brackets, and those seconds are charged to `tools` alone. The in-process
|
|
1269
|
+
stages therefore add up to the total — measured on a real cold launch below,
|
|
1270
|
+
they came to 20.834s against a 20.834s total.
|
|
1271
|
+
|
|
1272
|
+
`handoff` is the exception to that sum, and the only one: it ends where
|
|
1273
|
+
`total` begins, so it is time the process could not have measured from inside
|
|
1274
|
+
itself. A consumer adding stages up against the total leaves it out.
|
|
1275
|
+
|
|
1276
|
+
Two optional environment variables let whatever launches `dl` — a shell
|
|
1277
|
+
function, an agent front-end — close the loop on the time before dl existed.
|
|
1278
|
+
Both are Unix epoch seconds, which is what `date +%s.%N` prints:
|
|
1279
|
+
|
|
1280
|
+
| variable | meaning |
|
|
1281
|
+
|---|---|
|
|
1282
|
+
| `DEVLAUNCH_HANDOFF_T0` | the keystroke that resolved to this exec. Becomes the `handoff` stage — the only measurement of exec plus interpreter startup there is, since `total` begins after both |
|
|
1283
|
+
| `DEVLAUNCH_PREWARM_FIRED_AT` | when a prewarm (`dl <ws> up`) was fired for this workspace, if one was |
|
|
1284
|
+
|
|
1285
|
+
With the prewarm stamp set, the document also reports what that prewarm was
|
|
1286
|
+
worth: the head start it bought, and which shape the launch then took — `hit`
|
|
1287
|
+
(the workspace was already up), `partial` (this launch queued behind a prewarm
|
|
1288
|
+
still running) or `miss` (this launch ran the `up` itself). dl decides that,
|
|
1289
|
+
not the firer: a prewarm is fired and forgotten, so only the launch that
|
|
1290
|
+
followed can see whether it helped. **A stamp that is missing, unreadable, or
|
|
1291
|
+
ahead of this clock reports nothing** rather than a zero — an absent handoff
|
|
1292
|
+
and an instantaneous one are different facts, and a trend cannot tell them
|
|
1293
|
+
apart once one is written as the other.
|
|
1294
|
+
|
|
1295
|
+
Captured from a real warm launch with both stamps set (one line, wrapped and
|
|
1296
|
+
elided here for reading):
|
|
1297
|
+
|
|
1298
|
+
```bash
|
|
1299
|
+
$ DEVLAUNCH_TIMING=json DEVLAUNCH_HANDOFF_T0=$(date +%s.%N) \
|
|
1300
|
+
DEVLAUNCH_PREWARM_FIRED_AT=... dl-next blooop/mcp-devtasks -- true
|
|
1301
|
+
...
|
|
1302
|
+
dl-timing-json: {"total": 2.210768, "total_epoch": "in-process, excluding interpreter startup",
|
|
1303
|
+
"stages": [{"stage": "handoff", "seconds": 0.130542, "outcome": "ok", "spans": []},
|
|
1304
|
+
{"stage": "host-prep", "seconds": 0.027799, "outcome": "ok",
|
|
1305
|
+
"spans": [{"label": "gh auth token", "seconds": 0.02747}]},
|
|
1306
|
+
{"stage": "devpod-up", "seconds": 0.455188, "outcome": "ok",
|
|
1307
|
+
"spans": [{"label": "devpod status", "seconds": 0.455158}]},
|
|
1308
|
+
{"stage": "attach", "seconds": 1.726961, "outcome": "ok",
|
|
1309
|
+
"spans": [{"label": "devpod ssh", "seconds": 1.72661}]}],
|
|
1310
|
+
"prewarm": {"head_start_seconds": 42.489243, "shape": "hit"}}
|
|
1311
|
+
```
|
|
1312
|
+
|
|
1313
|
+
Stages appear in the order the launch first entered them, and only the ones it
|
|
1314
|
+
reached appear at all — this warm launch built nothing and lent nothing, so
|
|
1315
|
+
there is no `tools` stage and no `devpod up` inside `devpod-up`. That
|
|
1316
|
+
`handoff: 0.131s` is the exec and the interpreter start, which nothing else
|
|
1317
|
+
measures.
|
|
1318
|
+
|
|
1319
|
+
The cold launch of the same repo, same host and session, decomposed as:
|
|
1320
|
+
`host-prep` 2.257s (`git clone --bare` 1.602 + `git fetch` 0.427 + workspace
|
|
1321
|
+
`git clone` 0.065 + LFS probe 0.002 + token 0.034), `devpod-up` 5.848s (`devpod
|
|
1322
|
+
up` 5.113 of it), `tools` 10.224s (probe trip 1.584 + `tools tar` 0.111 +
|
|
1323
|
+
transfer 8.445), `attach` 2.505s — 20.834s of stages against a 20.834s total.
|
|
1324
|
+
|
|
824
1325
|
For before/after numbers, `scripts/bench_launch.py` runs a command N times and
|
|
825
1326
|
reports the median — one command per side of a change:
|
|
826
1327
|
|
|
827
1328
|
```bash
|
|
828
|
-
|
|
1329
|
+
python3 scripts/bench_launch.py -n 5 -- dl-next owner/repo -- true # warm launch
|
|
829
1330
|
```
|
|
830
1331
|
|
|
831
1332
|
(`pixi run bench -n 5 -- ...` in the devcontainer.) It reports no median if any
|
|
832
1333
|
run fails, so a broken launch cannot pass as a fast one. See `bench_launch.py
|
|
833
|
-
--help` for `--before` — the per-run reset that makes a *cold* median cold
|
|
834
|
-
|
|
1334
|
+
--help` for `--before` — the per-run reset that makes a *cold* median cold and
|
|
1335
|
+
whose `rm --force` also succeeds on the first run, when there is nothing to
|
|
1336
|
+
remove yet — and for why its wall clock and `dl-timing: total` are not the
|
|
1337
|
+
same quantity. For scale, on the host the session above was captured on, the
|
|
1338
|
+
warm median over 5 runs was 2.176s. Running the cold recipe exactly as the
|
|
1339
|
+
epilog writes it — `-n 5`, container recreated per run — gave a median of
|
|
1340
|
+
15.899s (runs: 15.9, 20.0, 15.2, 15.7, 17.8). Read that as the cost of
|
|
1341
|
+
recreating a container, not of a first-ever launch: the reset removes the
|
|
1342
|
+
workspace but leaves the docker image layers and the bare clone cache, so
|
|
1343
|
+
every run after the first starts from both. A machine that must also pull or
|
|
1344
|
+
build the image pays more, by an amount this recipe does not measure — but the
|
|
1345
|
+
gap is large: an earlier 3-run median on this same host, reported as its first
|
|
1346
|
+
real launch, was 33.204s.
|
|
1347
|
+
|
|
1348
|
+
Every number in the two paragraphs above was copied into this prose by hand.
|
|
1349
|
+
`--record` is how that stops: it writes the same invocation as one JSON object
|
|
1350
|
+
a trend job can upload without anyone reading it.
|
|
1351
|
+
|
|
1352
|
+
```bash
|
|
1353
|
+
python3 scripts/bench_launch.py -n 5 --record warm.json --shape warm \
|
|
1354
|
+
-- dl-next owner/repo -- true
|
|
1355
|
+
```
|
|
1356
|
+
|
|
1357
|
+
The record holds the command, the run count, each run's wall time and
|
|
1358
|
+
per-stage seconds, and the medians of those — and nothing else, because a CI
|
|
1359
|
+
job stamps its own commit, clock and host better than this script can. Four
|
|
1360
|
+
things about it are load-bearing:
|
|
1361
|
+
|
|
1362
|
+
- **The median is the point, the runs are its evidence.** A trend compares a
|
|
1363
|
+
point against the immediately previous one, so N runs published as N points
|
|
1364
|
+
would read ordinary spread as a regression.
|
|
1365
|
+
- **A stage no run reported is absent, not zero.** A warm launch legitimately
|
|
1366
|
+
has no cold-path stages; a zero would claim the work happened instantly
|
|
1367
|
+
rather than not at all. A stage only *some* runs reported is a median over
|
|
1368
|
+
those runs, carrying a count of how many — a median of two and a median of
|
|
1369
|
+
five are not the same claim.
|
|
1370
|
+
- **Recording asks the launch for its stages** (`DEVLAUNCH_TIMING=json`), so a
|
|
1371
|
+
run that reports no timing document is an error and no record — same
|
|
1372
|
+
discipline as no median over a failed run.
|
|
1373
|
+
- **`--shape` labels the trend line** (`warm`, `cold-recreate`). It is the
|
|
1374
|
+
caller's to say: the same command benches either shape depending on
|
|
1375
|
+
`--before`, and a wrong label is worse in a trend than a missing one.
|
|
1376
|
+
|
|
1377
|
+
### The trend on main
|
|
1378
|
+
|
|
1379
|
+
Every push to `main` runs `.github/workflows/bench.yml`, which benches both
|
|
1380
|
+
shapes on the runner and publishes one point per stage to
|
|
1381
|
+
<https://blooop.github.io/devlaunch/dev/bench/>. It can also be dispatched by
|
|
1382
|
+
hand. Reading it needs nothing but the chart; what follows is for changing it.
|
|
1383
|
+
|
|
1384
|
+
`scripts/bench_points.py` is the step between the two formats — bench records
|
|
1385
|
+
in, one flat array of trend cases out:
|
|
1386
|
+
|
|
1387
|
+
```bash
|
|
1388
|
+
python3 scripts/bench_points.py warm.json cold-recreate.json --out bench.json \
|
|
1389
|
+
--require-stages-on cold-recreate
|
|
1390
|
+
```
|
|
1391
|
+
|
|
1392
|
+
(`pixi run bench-points ...` in the devcontainer.) One case per stage the shape
|
|
1393
|
+
reported, plus that shape's own total: `warm / host-prep`,
|
|
1394
|
+
`cold-recreate / devpod-up`, `warm / total`. Six properties of it are
|
|
1395
|
+
load-bearing:
|
|
1396
|
+
|
|
1397
|
+
- **The published value is the median, and the case name is a key.** The trend
|
|
1398
|
+
compares a point against the immediately previous point and nothing older, so
|
|
1399
|
+
the de-noising has to have happened before publishing. Renaming a stage
|
|
1400
|
+
starts a new, empty series beside a frozen old one.
|
|
1401
|
+
- **The spread rides along as the point's error bar**, and the outside
|
|
1402
|
+
stopwatch (`wall=`) as part of its `extra`. Evidence beside the number,
|
|
1403
|
+
rather than a second trend line for the same launch that disagrees with the
|
|
1404
|
+
first by a constant.
|
|
1405
|
+
- **An absent stage is absent.** A warm launch lends nothing, so there is no
|
|
1406
|
+
`warm / tools` case at all — never a zero, which would claim an instantaneous
|
|
1407
|
+
lend and would drag the line down exactly where a regression should show.
|
|
1408
|
+
- **`--require-stages-on` fails the job rather than the trend.** The way this
|
|
1409
|
+
decomposition is expected to break is an absence, not a wrong number: a stage
|
|
1410
|
+
stops being emitted and the total keeps working. So the cold-recreate shape,
|
|
1411
|
+
where every stage is known to be present, asserts them all, and a run that
|
|
1412
|
+
lost one publishes nothing and goes red. Naming a shape that was not benched
|
|
1413
|
+
fails too — an assertion that covers nothing reads exactly like one that
|
|
1414
|
+
passed.
|
|
1415
|
+
- **A record that is missing or unreadable refuses the same way.** The step
|
|
1416
|
+
that writes the records can exit 0 without having written one, so the absence
|
|
1417
|
+
arrives here — and it prints `bench_points: <reason>` naming the file and
|
|
1418
|
+
writes nothing, like every other refusal, rather than a traceback.
|
|
1419
|
+
- **A regression alerts; it never gates.** The workflow is deliberately not a
|
|
1420
|
+
job in `ci.yml`: a job there would join the CI gate's `needs` by house
|
|
1421
|
+
convention and turn a noisy wall-clock measurement into a merge gate. A point
|
|
1422
|
+
above the threshold leaves a commit comment and a red mark on the chart, and
|
|
1423
|
+
the build stays green.
|
|
835
1424
|
|
|
836
1425
|
## Worktree Backend
|
|
837
1426
|
|
|
@@ -839,7 +1428,22 @@ For git repositories, devlaunch uses an efficient worktree backend by default:
|
|
|
839
1428
|
|
|
840
1429
|
- **Efficient Storage**: Repos are cloned once to `~/.cache/devlaunch/repos/owner/repo/`, then git worktrees are created for each branch
|
|
841
1430
|
- **Shared Git Objects**: All branches share git objects, saving disk space
|
|
842
|
-
- **
|
|
1431
|
+
- **Targeted Fetch**: A launch fetches only the one branch it is launching, so no launch waits on a repo-wide refresh
|
|
1432
|
+
|
|
1433
|
+
### What you get when you push and immediately launch
|
|
1434
|
+
|
|
1435
|
+
- **Attaching to a workspace devpod already knows**: no git at all. The workspace
|
|
1436
|
+
is exactly as you left it; freshness inside it is your own `git pull`.
|
|
1437
|
+
- **A cold launch** (first time this branch is launched on this machine, or a
|
|
1438
|
+
clone devpod has forgotten): one targeted fetch of that branch, every time.
|
|
1439
|
+
Push upstream and immediately `dl` the branch and you get the pushed tip.
|
|
1440
|
+
- **A branch that does not exist yet**: created from the default branch's freshly
|
|
1441
|
+
fetched tip.
|
|
1442
|
+
- **Offline**: a warning, and the launch proceeds from whatever the cache holds.
|
|
1443
|
+
It only fails when there is nothing cached to launch from.
|
|
1444
|
+
- **Everything else** (other branches, tags, prunes) is refreshed by the
|
|
1445
|
+
background updater within the configured interval (default: 1 hour), which
|
|
1446
|
+
never blocks a launch.
|
|
843
1447
|
|
|
844
1448
|
### Container Sharing Mode
|
|
845
1449
|
|
|
@@ -872,7 +1476,7 @@ After running `dl --install`, you get intelligent tab completion:
|
|
|
872
1476
|
The data behind completions lives in `~/.cache/devlaunch/completions.json`, and
|
|
873
1477
|
building it means a `git ls-remote` per known repo — seconds of work. So it is
|
|
874
1478
|
rebuilt in the background at most once an hour (the same interval the worktree
|
|
875
|
-
backend
|
|
1479
|
+
backend's background fetch sweep uses), and at most once per `dl` invocation. Commands
|
|
876
1480
|
that change your workspaces (starting, stopping or deleting one) rebuild it as
|
|
877
1481
|
soon as they finish, regardless of when it was last built. Commands with no use
|
|
878
1482
|
for it — `dl --help`, `dl --version` — do not touch it at all.
|