gdkbox 0.1.18 → 0.1.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +105 -0
- data/lib/gdkbox/box.rb +79 -13
- data/lib/gdkbox/cli.rb +53 -1
- data/lib/gdkbox/completion.rb +2 -2
- data/lib/gdkbox/config.rb +12 -0
- data/lib/gdkbox/docker.rb +19 -1
- data/lib/gdkbox/provisioner.rb +41 -0
- data/lib/gdkbox/store.rb +3 -1
- data/lib/gdkbox/version.rb +1 -1
- data/skills/gdkbox-fleet/SKILL.md +35 -0
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 17efb5fee81f15043aed3c8dbabe3e9259c70a8d8a03dd400b8103231b36a9d6
|
|
4
|
+
data.tar.gz: d926bbae45cbe226fec0530f43e11e12f1ef346450e5801909c251e6f159bf66
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 179859b546c7b76825d22d6e100b39446d96205adaab696982f960409f14041a8135daf67600d3755545613537a011131302c238de44fefd64df5a7980834227
|
|
7
|
+
data.tar.gz: 59f9e916539ca282cc16ec86e8fde2ab8437541aad4c433d4c5a2895f49017cf747880f75e3777e80c39b88e767591c9a8b18d115badfae9fcabf4525bc22f2d
|
data/README.md
CHANGED
|
@@ -91,6 +91,7 @@ Or run it straight from the checkout without installing:
|
|
|
91
91
|
| `gdkbox set-key NAME` | Seed/rotate the Anthropic API key in the box for unattended dispatch. |
|
|
92
92
|
| `gdkbox set-host` | Add the `gdk.local` entry to your `/etc/hosts` so box web UIs resolve (`--remove` to undo). Also runs during `up`. |
|
|
93
93
|
| `gdkbox set-git NAME` | Seed your git identity (`user.name`/`user.email`) into the box so `git commit` works. Also runs during `up`. |
|
|
94
|
+
| `gdkbox set-gcloud NAME` | Copy the host's Google Cloud application-default credentials into the box (Duo / Vertex AI model access; `--file` to override). |
|
|
94
95
|
| `gdkbox set-remote NAME [REMOTE]` | Point the box's GitLab checkout at a different remote (URL or `namespace/project`, e.g. `gitlab-org/gitlab`); default from config.yml. |
|
|
95
96
|
| `gdkbox hydrate NAME` | Backfill the box's treeless GitLab clone so deep rebases/blame/bisect work (`--trees` for a much smaller trees-only fetch). |
|
|
96
97
|
| `gdkbox claim [NAME]` | Claim a box for exclusive use — any free box when NAME is omitted (`--owner ID`, optional `--ttl SECS` lease, `--json`). Advisory lock for orchestrators. |
|
|
@@ -119,6 +120,8 @@ Or run it straight from the checkout without installing:
|
|
|
119
120
|
| `--gitlab-remote REMOTE` | config.yml, else image default | Point the GitLab checkout at this remote (URL or `namespace/project`); the image ships pointing at the community mirror. |
|
|
120
121
|
| `--owner ID` | (unclaimed) | Claim the box for this owner at creation — no window for another orchestrator to grab it. |
|
|
121
122
|
| `--ttl SECS` | no expiry | Lease duration for `--owner`'s claim. |
|
|
123
|
+
| `--docker` | off | Mount the host Docker socket + per-box network so docker-executor CI jobs work. **Grants the box root-equivalent host access** — see "Docker inside a box". |
|
|
124
|
+
| `--publish PORT [PORT...]` | (none) | Publish extra container ports on `127.0.0.1`, same number on both sides (e.g. AI Gateway `5052`, Duo Workflow `50052`). |
|
|
122
125
|
|
|
123
126
|
## Typical workflow
|
|
124
127
|
|
|
@@ -159,6 +162,108 @@ gdkbox hydrate demo # full backfill: several GB, then the clone
|
|
|
159
162
|
# behaves like a normal full clone
|
|
160
163
|
```
|
|
161
164
|
|
|
165
|
+
### Docker inside a box (CI docker executor)
|
|
166
|
+
|
|
167
|
+
Some workloads — notably GitLab Duo agentic flows — finish by running a CI job
|
|
168
|
+
with the **docker executor**, which needs a Docker daemon. Boxes have none by
|
|
169
|
+
default. `gdkbox up NAME --docker` wires it up:
|
|
170
|
+
|
|
171
|
+
- The **host Docker socket** is mounted into the box and the Docker CLI is
|
|
172
|
+
installed there, so the box drives the *host* daemon. Containers the box
|
|
173
|
+
starts are **siblings on the host**, not children of the box.
|
|
174
|
+
- The box joins a per-box network (`gdkbox-NAME`) with the alias `gdk.local`,
|
|
175
|
+
so sibling containers can reach the box's GitLab by hostname. Point the
|
|
176
|
+
box's GitLab Runner at it in `config.toml`:
|
|
177
|
+
|
|
178
|
+
```toml
|
|
179
|
+
[runners.docker]
|
|
180
|
+
network_mode = "gdkbox-NAME" # job containers resolve gdk.local natively
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
**Security:** the socket is root-equivalent control of the **host** — anything
|
|
184
|
+
running in a `--docker` box (including dispatched agents, which skip permission
|
|
185
|
+
prompts) can start privileged containers, mount host paths, and read other
|
|
186
|
+
boxes. Only use `--docker` on boxes whose workloads you trust like your own
|
|
187
|
+
shell, and never combine it with unattended agent tasks you wouldn't run on
|
|
188
|
+
the host directly.
|
|
189
|
+
|
|
190
|
+
### Running Duo agentic flows in a box
|
|
191
|
+
|
|
192
|
+
The main consumer of `--docker` is the GitLab Duo Agent Platform: a flow ends
|
|
193
|
+
by running a docker-executor CI job.
|
|
194
|
+
|
|
195
|
+
<details>
|
|
196
|
+
<summary><strong>Step-by-step setup</strong> (distilled from a real end-to-end
|
|
197
|
+
run; verify commands against the
|
|
198
|
+
<a href="https://gitlab-org.gitlab.io/gitlab-development-kit/howto/ai">GDK AI
|
|
199
|
+
howto</a>)</summary>
|
|
200
|
+
|
|
201
|
+
|
|
202
|
+
|
|
203
|
+
1. **Provision**:
|
|
204
|
+
`gdkbox up duo-1 --docker --publish 5052 50052` then
|
|
205
|
+
`gdkbox set-gcloud duo-1` (Vertex AI credentials; the flow model routes
|
|
206
|
+
through Vertex, project selected by `AIGW_GOOGLE_CLOUD_PLATFORM__PROJECT`).
|
|
207
|
+
2. **AI services** (inside the box): `gdk rake setup_ai_services` clones the
|
|
208
|
+
`gitlab-ai-gateway` monorepo and builds its Python env — **~20 minutes**,
|
|
209
|
+
needs a valid `ANTHROPIC_API_KEY` (a dead key blocks Chat, service boot,
|
|
210
|
+
and flows). Budget memory: GDK + AI stack + flow containers want well over
|
|
211
|
+
10 GB free.
|
|
212
|
+
3. **The env asymmetry that causes most failures**: the AI Gateway reads
|
|
213
|
+
`gitlab-ai-gateway/.env`, but the Duo Workflow Service reads **`env.runit`**
|
|
214
|
+
at the GDK root. `ANTHROPIC_API_KEY` (unquoted),
|
|
215
|
+
`DUO_WORKFLOW_AUTH__ENABLED=false`, the demo JWT keys from `example.env`,
|
|
216
|
+
and `AIGW_GOOGLE_CLOUD_PLATFORM__PROJECT` all go in `env.runit`.
|
|
217
|
+
4. **Agentic WebSocket 403 fix**: `gdk config set gitlab_http_router.enabled
|
|
218
|
+
false && gdk config set nginx.enabled true && gdk reconfigure` — the
|
|
219
|
+
http-router rewrites the upstream Host and workhorse rejects the flow's
|
|
220
|
+
WebSocket upgrade (classic Chat works, flows 403). Also set the acting
|
|
221
|
+
user's `duo_default_namespace_id` to the seeded `gitlab-duo` group.
|
|
222
|
+
5. **Point at the local gateway and seed**:
|
|
223
|
+
`gdk config set gitlab_ai_gateway.environment local`, then from `gitlab/`:
|
|
224
|
+
`DEVELOPMENT_AI_GATEWAY_URL="http://gdk.local:5052" GITLAB_SIMULATE_SAAS=1
|
|
225
|
+
bundle exec rake gitlab:duo:setup` and `... gitlab:duo:onboard_dap`.
|
|
226
|
+
A fresh box has no license, so `gitlab:duo:setup` aborts with
|
|
227
|
+
**"No license found"** — import one first (e.g. copy the `data` column from
|
|
228
|
+
your host GDK's `licenses` table and load it in the box's Rails console
|
|
229
|
+
with `License.new(data: data, cloud: true).save!`).
|
|
230
|
+
6. **Runner**: in a `--docker` box the socket is already at
|
|
231
|
+
`/var/run/docker.sock` (no `docker_host` override needed). Three
|
|
232
|
+
`[runners.docker]` settings in the generated `gitlab-runner-config.toml`
|
|
233
|
+
matter — GDK's template doesn't persist them, so re-apply after any
|
|
234
|
+
`gdk reconfigure`:
|
|
235
|
+
|
|
236
|
+
```toml
|
|
237
|
+
[runners.docker]
|
|
238
|
+
network_mode = "gdkbox-duo-1" # job containers resolve gdk.local natively
|
|
239
|
+
privileged = false # privileged breaks the flow sandbox (below)
|
|
240
|
+
volumes = ["/tmp/gdkbox-docker-cfg/certs.d:/etc/docker/certs.d",
|
|
241
|
+
"/certs/client", "/cache"]
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
Because the box drives the **host** daemon, volume *source* paths must
|
|
245
|
+
exist **on the host**, not in the box — a box-only path fails the job with
|
|
246
|
+
`mounts denied: path not shared from host`. And with `privileged = true`
|
|
247
|
+
the flow's `unshare` sandbox loses network access to `gdk.local`, so the
|
|
248
|
+
job dies with "Workflow could not connect to the Workflow service /
|
|
249
|
+
User: not authenticated"; keep the runner unprivileged.
|
|
250
|
+
7. **Verify**: AI Gateway answers on `http://gdk.local:5052/docs` (published
|
|
251
|
+
by `--publish`), the workflow service binds `:50052`, and triggering the
|
|
252
|
+
`developer` flow from an issue in the seeded `gitlab-duo` project completes
|
|
253
|
+
and opens a Draft MR.
|
|
254
|
+
|
|
255
|
+
> **Note — rate-limited Anthropic keys**: if your key returns `429
|
|
256
|
+
> rate_limit_error` on the default Sonnet models, chat and flows fail even
|
|
257
|
+
> though the setup is correct. Pin Duo to a model your key can use (e.g.
|
|
258
|
+
> Haiku): per namespace, run
|
|
259
|
+
> `Ai::ModelSelection::Namespace::UpdateService.new(feature_setting, group,
|
|
260
|
+
> {offered_model_ref: "claude_haiku_4_5_20251001"}).execute` in the box's
|
|
261
|
+
> Rails console (needs `GITLAB_SIMULATE_SAAS=1`, silently returns `nil`
|
|
262
|
+
> without it), and for the AI Gateway defaults rewrite the model refs in
|
|
263
|
+
> `gitlab-ai-gateway/ai_gateway/model_selection/unit_primitives.yml`.
|
|
264
|
+
|
|
265
|
+
</details>
|
|
266
|
+
|
|
162
267
|
## Orchestrating a fleet of agents
|
|
163
268
|
|
|
164
269
|
The end goal of `gdkbox` is to back an **orchestrator** that runs a pool of
|
data/lib/gdkbox/box.rb
CHANGED
|
@@ -91,7 +91,7 @@ module GDKBox
|
|
|
91
91
|
# box this one is still provisioning.
|
|
92
92
|
def create!(image: nil, ssh_port: nil, web_port: nil, vite_port: nil,
|
|
93
93
|
harness: Harness.default, install_agent: true, api_key: nil,
|
|
94
|
-
claim_owner: nil, claim_ttl: nil)
|
|
94
|
+
claim_owner: nil, claim_ttl: nil, docker_access: false, extra_ports: [])
|
|
95
95
|
raise Error, "Box '#{name}' already exists" if exists?
|
|
96
96
|
|
|
97
97
|
harness = Harness[harness]
|
|
@@ -110,10 +110,28 @@ module GDKBox
|
|
|
110
110
|
# choose ports *and* persist a preliminary record makes each sibling see
|
|
111
111
|
# the others' reservations.
|
|
112
112
|
ssh_port, web_port, vite_port = reserve_ports!(
|
|
113
|
-
cname, ssh_port, web_port, vite_port,
|
|
113
|
+
cname, ssh_port, web_port, vite_port, extra_ports,
|
|
114
114
|
claim_owner: claim_owner, claim_ttl: claim_ttl
|
|
115
115
|
)
|
|
116
116
|
|
|
117
|
+
publish = [
|
|
118
|
+
"127.0.0.1:#{ssh_port}:#{Config::SSH_CONTAINER_PORT}",
|
|
119
|
+
"127.0.0.1:#{web_port}:#{Config::GDK_WEB_CONTAINER_PORT}",
|
|
120
|
+
# Same number on both sides: asset URLs embed this port, so what
|
|
121
|
+
# the browser dials must be where vite listens (see VITE_PORT_BASE).
|
|
122
|
+
"127.0.0.1:#{vite_port}:#{vite_port}"
|
|
123
|
+
]
|
|
124
|
+
# Extra ports publish same-numbered too: in-box services advertise
|
|
125
|
+
# their own port, so what the host dials must match.
|
|
126
|
+
extra_ports.each { |port| publish << "127.0.0.1:#{port}:#{port}" }
|
|
127
|
+
|
|
128
|
+
# Docker access: mount the host socket (containers the box starts are
|
|
129
|
+
# siblings on the host daemon) and put the box on its own network with
|
|
130
|
+
# a gdk.local alias, so those siblings — docker-executor CI jobs — can
|
|
131
|
+
# reach the box's GitLab by hostname.
|
|
132
|
+
net = docker_access ? @config.docker_network(name) : nil
|
|
133
|
+
@docker.ensure_network(net) if net
|
|
134
|
+
|
|
117
135
|
begin
|
|
118
136
|
@docker.run_container(
|
|
119
137
|
name: cname,
|
|
@@ -122,19 +140,17 @@ module GDKBox
|
|
|
122
140
|
# set inside the container, workhorse and the http-router crash-loop
|
|
123
141
|
# on DNS timeouts.
|
|
124
142
|
hostname: Config::GDK_HOSTNAME,
|
|
125
|
-
publish:
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
# the browser dials must be where vite listens (see VITE_PORT_BASE).
|
|
130
|
-
"127.0.0.1:#{vite_port}:#{vite_port}"
|
|
131
|
-
],
|
|
143
|
+
publish: publish,
|
|
144
|
+
volumes: (docker_access ? ["#{Config::DOCKER_SOCKET}:#{Config::DOCKER_SOCKET}"] : []),
|
|
145
|
+
network: net,
|
|
146
|
+
network_alias: (net ? Config::GDK_HOSTNAME : nil),
|
|
132
147
|
labels: { "gdkbox" => "true", "gdkbox.name" => name }
|
|
133
148
|
)
|
|
134
149
|
rescue StandardError
|
|
135
150
|
# The container never started, so release the reserved ports rather
|
|
136
151
|
# than stranding a record that points at nothing.
|
|
137
152
|
@store.delete(name)
|
|
153
|
+
@docker.rm_network(net) if net
|
|
138
154
|
raise
|
|
139
155
|
end
|
|
140
156
|
|
|
@@ -152,8 +168,24 @@ module GDKBox
|
|
|
152
168
|
"harness" => harness.id,
|
|
153
169
|
"created_at" => Time.now.utc.iso8601
|
|
154
170
|
)
|
|
171
|
+
if docker_access
|
|
172
|
+
@data["docker_access"] = true
|
|
173
|
+
@data["docker_network"] = net
|
|
174
|
+
end
|
|
155
175
|
@store.save(@data)
|
|
156
176
|
|
|
177
|
+
# Docker CLI + socket access for the gdk user. Best-effort (needs the
|
|
178
|
+
# network for the static CLI download); the outcome is recorded so the
|
|
179
|
+
# CLI can print a retry hint.
|
|
180
|
+
if docker_access
|
|
181
|
+
@data["docker_cli_installed"] = begin
|
|
182
|
+
provisioner.setup_docker_cli(cname)
|
|
183
|
+
true
|
|
184
|
+
rescue StandardError
|
|
185
|
+
false
|
|
186
|
+
end
|
|
187
|
+
end
|
|
188
|
+
|
|
157
189
|
# Align vite's in-container listen port with the published one so asset
|
|
158
190
|
# URLs work from the host browser. Best-effort — the box is fully
|
|
159
191
|
# usable for SSH/agent work without it — but the outcome is recorded
|
|
@@ -199,7 +231,9 @@ module GDKBox
|
|
|
199
231
|
end
|
|
200
232
|
|
|
201
233
|
def destroy!
|
|
234
|
+
network = data && data["docker_network"]
|
|
202
235
|
@docker.rm(container_name, force: true)
|
|
236
|
+
@docker.rm_network(network) if network
|
|
203
237
|
@store.delete(name)
|
|
204
238
|
end
|
|
205
239
|
|
|
@@ -366,6 +400,27 @@ module GDKBox
|
|
|
366
400
|
result
|
|
367
401
|
end
|
|
368
402
|
|
|
403
|
+
# Copy Google Cloud application-default credentials into the box so Duo /
|
|
404
|
+
# Vertex AI model calls can authenticate. The file holds a refresh token,
|
|
405
|
+
# so copying works where an in-box `gcloud auth` login cannot (no
|
|
406
|
+
# browser). Stored 0600, owned by the GDK user, inside the container only.
|
|
407
|
+
def set_gcloud_credentials!(source_path)
|
|
408
|
+
unless File.file?(source_path)
|
|
409
|
+
raise Error, "No credentials file at #{source_path}. Run " \
|
|
410
|
+
"`gcloud auth application-default login` on the host first, or pass --file."
|
|
411
|
+
end
|
|
412
|
+
|
|
413
|
+
dir = "/home/#{@config.ssh_user}/.config/gcloud"
|
|
414
|
+
dest = "#{dir}/application_default_credentials.json"
|
|
415
|
+
@docker.exec(container_name, 'install -d -m 700 -o "$GDKBOX_USER" -g "$GDKBOX_USER" "$GDKBOX_DIR"',
|
|
416
|
+
user: "root", env: { "GDKBOX_USER" => @config.ssh_user, "GDKBOX_DIR" => dir })
|
|
417
|
+
@docker.cp_into(container_name, source_path, dest)
|
|
418
|
+
@docker.exec(container_name,
|
|
419
|
+
'chown "$GDKBOX_USER:$GDKBOX_USER" "$GDKBOX_DEST" && chmod 600 "$GDKBOX_DEST"',
|
|
420
|
+
user: "root", env: { "GDKBOX_USER" => @config.ssh_user, "GDKBOX_DEST" => dest })
|
|
421
|
+
dest
|
|
422
|
+
end
|
|
423
|
+
|
|
369
424
|
# Seed a git identity into the box so `git commit` works there. Falls back
|
|
370
425
|
# to the host's `git config` when name/email are not given; raises when
|
|
371
426
|
# neither source has anything to seed. Returns the [name, email] seeded.
|
|
@@ -478,6 +533,12 @@ module GDKBox
|
|
|
478
533
|
# support) and whether the in-box alignment step succeeded.
|
|
479
534
|
"vite_port" => vite_port,
|
|
480
535
|
"vite_port_configured" => (data && data["vite_port_configured"]) || false,
|
|
536
|
+
"extra_ports" => (data && data["extra_ports"]) || [],
|
|
537
|
+
# Docker access (up --docker): host socket mounted; docker_network is
|
|
538
|
+
# the per-box network CI job containers should use as network_mode.
|
|
539
|
+
"docker_access" => (data && data["docker_access"]) || false,
|
|
540
|
+
"docker_network" => (data && data["docker_network"]),
|
|
541
|
+
"docker_cli_installed" => (data && data["docker_cli_installed"]) || false,
|
|
481
542
|
# Active claim (nil when free; an expired lease counts as free).
|
|
482
543
|
"claimed_by" => claimed_by,
|
|
483
544
|
"claim_expires_at" => (claimed_by ? data["claim_expires_at"] : nil)
|
|
@@ -518,11 +579,15 @@ module GDKBox
|
|
|
518
579
|
# than the cryptic bind error `docker run` would produce later.
|
|
519
580
|
# A claim_owner is written into this same record, so the box is claimed
|
|
520
581
|
# from the instant it becomes visible to other processes.
|
|
521
|
-
def reserve_ports!(cname, ssh_port, web_port, vite_port,
|
|
522
|
-
|
|
523
|
-
|
|
582
|
+
def reserve_ports!(cname, ssh_port, web_port, vite_port, extra_ports,
|
|
583
|
+
claim_owner: nil, claim_ttl: nil)
|
|
584
|
+
checks = { "--ssh-port" => ssh_port, "--web-port" => web_port,
|
|
585
|
+
"--vite-port" => vite_port }
|
|
586
|
+
extra_ports.each_with_index { |port, i| checks["--publish #{i}"] = port }
|
|
587
|
+
checks.each do |flag, port|
|
|
524
588
|
if port && Ports.bound?(port)
|
|
525
|
-
raise Error,
|
|
589
|
+
raise Error,
|
|
590
|
+
"Port #{port} (#{flag.sub(/ \d+\z/, '')}) is already in use on 127.0.0.1."
|
|
526
591
|
end
|
|
527
592
|
end
|
|
528
593
|
|
|
@@ -536,6 +601,7 @@ module GDKBox
|
|
|
536
601
|
"ssh_port" => ssh_port,
|
|
537
602
|
"web_port" => web_port,
|
|
538
603
|
"vite_port" => vite_port,
|
|
604
|
+
"extra_ports" => extra_ports,
|
|
539
605
|
"agent_installed" => false,
|
|
540
606
|
"api_key_set" => false
|
|
541
607
|
}
|
data/lib/gdkbox/cli.rb
CHANGED
|
@@ -48,6 +48,12 @@ module GDKBox
|
|
|
48
48
|
"orchestrator to grab it; see `gdkbox claim`)"
|
|
49
49
|
option :ttl, type: :numeric,
|
|
50
50
|
desc: "Lease duration in seconds for --owner's claim (default: no expiry)"
|
|
51
|
+
option :docker, type: :boolean, default: false,
|
|
52
|
+
desc: "Mount the host Docker socket so the box can run docker-executor CI jobs " \
|
|
53
|
+
"(grants the box root-equivalent control of the HOST — see README)"
|
|
54
|
+
option :publish, type: :array, default: [],
|
|
55
|
+
desc: "Extra container port(s) to publish on 127.0.0.1, same number on both sides " \
|
|
56
|
+
"(e.g. --publish 5052 50052)"
|
|
51
57
|
def up(name)
|
|
52
58
|
if options[:ttl] && options[:owner].to_s.strip.empty?
|
|
53
59
|
raise Error, "--ttl only makes sense with --owner (it is the claim's lease)."
|
|
@@ -68,7 +74,9 @@ module GDKBox
|
|
|
68
74
|
install_agent: options[:agent],
|
|
69
75
|
api_key: api_key,
|
|
70
76
|
claim_owner: options[:owner],
|
|
71
|
-
claim_ttl: options[:ttl]
|
|
77
|
+
claim_ttl: options[:ttl],
|
|
78
|
+
docker_access: options[:docker],
|
|
79
|
+
extra_ports: parse_extra_ports(options[:publish])
|
|
72
80
|
)
|
|
73
81
|
rewrite_ssh_config
|
|
74
82
|
seed_skills(box)
|
|
@@ -82,6 +90,18 @@ module GDKBox
|
|
|
82
90
|
setup_host
|
|
83
91
|
|
|
84
92
|
say "\nBox '#{name}' is up.", :green
|
|
93
|
+
if options[:docker]
|
|
94
|
+
if box.data["docker_cli_installed"]
|
|
95
|
+
say "\n Docker: host socket mounted; the box drives the HOST daemon " \
|
|
96
|
+
"(containers it starts are siblings).", :yellow
|
|
97
|
+
say " CI docker-executor jobs reach this GitLab via the box's network — " \
|
|
98
|
+
"in the runner's config.toml set:", :cyan
|
|
99
|
+
say " network_mode = \"#{box.data['docker_network']}\"", :cyan
|
|
100
|
+
else
|
|
101
|
+
say "\n Could not install the Docker CLI in the box (network hiccup?). " \
|
|
102
|
+
"Retry: gdkbox rm #{name} && gdkbox up #{name} --docker", :yellow
|
|
103
|
+
end
|
|
104
|
+
end
|
|
85
105
|
unless box.data["vite_port_configured"]
|
|
86
106
|
say "\n Could not align the box's vite port with the published " \
|
|
87
107
|
"#{box.vite_port} — browser asset loading may fail.", :yellow
|
|
@@ -401,6 +421,26 @@ module GDKBox
|
|
|
401
421
|
say "Done. '#{name}' can now work across the full history.", :green
|
|
402
422
|
end
|
|
403
423
|
|
|
424
|
+
desc "set-gcloud NAME", "Copy Google Cloud application-default credentials into the box"
|
|
425
|
+
long_desc <<~DESC
|
|
426
|
+
Duo / Vertex AI model calls from inside a box need Google Cloud
|
|
427
|
+
application-default credentials, and `gcloud auth application-default
|
|
428
|
+
login` cannot run in a box (no browser). This copies the host's ADC
|
|
429
|
+
file — which holds a refresh token, so it works from inside the
|
|
430
|
+
container — into the box at ~/.config/gcloud/, mode 0600, owned by the
|
|
431
|
+
GDK user. Defaults to the host's own ADC file; override with --file.
|
|
432
|
+
DESC
|
|
433
|
+
option :file, type: :string,
|
|
434
|
+
desc: "Credentials file to copy (default: ~/.config/gcloud/application_default_credentials.json)"
|
|
435
|
+
def set_gcloud(name)
|
|
436
|
+
box = load_box!(name)
|
|
437
|
+
source = options[:file] ||
|
|
438
|
+
File.join(Dir.home, ".config", "gcloud", "application_default_credentials.json")
|
|
439
|
+
dest = box.set_gcloud_credentials!(source)
|
|
440
|
+
say "Copied gcloud application-default credentials into '#{name}' (#{dest}).", :green
|
|
441
|
+
end
|
|
442
|
+
map "set-gcloud" => :set_gcloud
|
|
443
|
+
|
|
404
444
|
desc "set-host", "Add (default) or remove the gdk.local entry in /etc/hosts"
|
|
405
445
|
long_desc <<~DESC
|
|
406
446
|
GDK generates URLs and redirects that use the `gdk.local` hostname, so
|
|
@@ -615,6 +655,18 @@ module GDKBox
|
|
|
615
655
|
warn "gdkbox: retry later with: gdkbox set-remote #{box.name} #{remote}"
|
|
616
656
|
end
|
|
617
657
|
|
|
658
|
+
# --publish values as integer container ports, validated.
|
|
659
|
+
def parse_extra_ports(values)
|
|
660
|
+
values.map do |value|
|
|
661
|
+
port = Integer(value, exception: false)
|
|
662
|
+
unless port&.between?(1, 65_535)
|
|
663
|
+
raise Error, "--publish expects port numbers (got '#{value}')."
|
|
664
|
+
end
|
|
665
|
+
|
|
666
|
+
port
|
|
667
|
+
end.uniq
|
|
668
|
+
end
|
|
669
|
+
|
|
618
670
|
# Ensure gdk.local resolves on the host so the box's web UI is reachable
|
|
619
671
|
# (GDK redirects to that hostname). Adding the entry needs sudo, so warn
|
|
620
672
|
# before any password prompt appears. Failures are non-fatal: the box is
|
data/lib/gdkbox/completion.rb
CHANGED
|
@@ -11,8 +11,8 @@ module GDKBox
|
|
|
11
11
|
SHELLS = %w[bash zsh].freeze
|
|
12
12
|
|
|
13
13
|
# Subcommands whose first positional argument is an existing box.
|
|
14
|
-
BOX_COMMANDS = %w[status dispatch ssh code install-agent set-key set-git set-remote
|
|
15
|
-
claim release start stop rm add-skill].freeze
|
|
14
|
+
BOX_COMMANDS = %w[status dispatch ssh code install-agent set-key set-git set-remote
|
|
15
|
+
set-gcloud hydrate claim release start stop rm add-skill].freeze
|
|
16
16
|
|
|
17
17
|
# Marker identifying the rc-file block `install` manages.
|
|
18
18
|
MARKER = "# gdkbox:completion"
|
data/lib/gdkbox/config.rb
CHANGED
|
@@ -29,6 +29,11 @@ module GDKBox
|
|
|
29
29
|
# follow GDK's redirects to it).
|
|
30
30
|
GDK_HOSTNAME = "gdk.local"
|
|
31
31
|
|
|
32
|
+
# The host Docker socket, mounted into a box by `up --docker` so the box
|
|
33
|
+
# can run docker-executor CI jobs (containers it starts are siblings on
|
|
34
|
+
# the host daemon).
|
|
35
|
+
DOCKER_SOCKET = "/var/run/docker.sock"
|
|
36
|
+
|
|
32
37
|
CONTAINER_PREFIX = "gdkbox-"
|
|
33
38
|
HOST_ALIAS_PREFIX = "gdkbox-"
|
|
34
39
|
|
|
@@ -159,6 +164,13 @@ module GDKBox
|
|
|
159
164
|
"#{CONTAINER_PREFIX}#{name}"
|
|
160
165
|
end
|
|
161
166
|
|
|
167
|
+
# Per-box user-defined network for --docker boxes. Per-box (not shared)
|
|
168
|
+
# so each box can hold the gdk.local alias without DNS round-robin
|
|
169
|
+
# between boxes.
|
|
170
|
+
def docker_network(name)
|
|
171
|
+
"#{CONTAINER_PREFIX}#{name}"
|
|
172
|
+
end
|
|
173
|
+
|
|
162
174
|
def ssh_host_alias(name)
|
|
163
175
|
"#{HOST_ALIAS_PREFIX}#{name}"
|
|
164
176
|
end
|
data/lib/gdkbox/docker.rb
CHANGED
|
@@ -196,9 +196,15 @@ module GDKBox
|
|
|
196
196
|
# Docker also maps it to the container's IP in the container's /etc/hosts
|
|
197
197
|
# and re-applies that mapping on every start (a manual edit would be lost:
|
|
198
198
|
# Docker regenerates the file when the container starts).
|
|
199
|
-
def run_container(name:, image:, publish: [], labels: {}, env: {}, args: [], hostname: nil
|
|
199
|
+
def run_container(name:, image:, publish: [], labels: {}, env: {}, args: [], hostname: nil,
|
|
200
|
+
volumes: [], network: nil, network_alias: nil)
|
|
200
201
|
cmd = ["docker", "run", "-d", "--name", name]
|
|
201
202
|
cmd.push("--hostname", hostname) if hostname
|
|
203
|
+
cmd.push("--network", network) if network
|
|
204
|
+
# A network alias gives the box a DNS name *on that network*, so sibling
|
|
205
|
+
# containers (e.g. docker-executor CI jobs) can reach it by hostname.
|
|
206
|
+
cmd.push("--network-alias", network_alias) if network_alias
|
|
207
|
+
volumes.each { |mapping| cmd.push("-v", mapping) }
|
|
202
208
|
labels.each { |key, value| cmd.push("--label", "#{key}=#{value}") }
|
|
203
209
|
env.each { |key, value| cmd.push("-e", "#{key}=#{value}") }
|
|
204
210
|
publish.each { |mapping| cmd.push("-p", mapping) }
|
|
@@ -207,6 +213,18 @@ module GDKBox
|
|
|
207
213
|
@shell.run!(*cmd).stdout.strip
|
|
208
214
|
end
|
|
209
215
|
|
|
216
|
+
# Create a user-defined bridge network if it does not exist (idempotent).
|
|
217
|
+
def ensure_network(name)
|
|
218
|
+
return if @shell.run("docker", "network", "inspect", name).success?
|
|
219
|
+
|
|
220
|
+
@shell.run!("docker", "network", "create", name)
|
|
221
|
+
end
|
|
222
|
+
|
|
223
|
+
# Remove a network; a failure (still in use, already gone) is not fatal.
|
|
224
|
+
def rm_network(name)
|
|
225
|
+
@shell.run("docker", "network", "rm", name)
|
|
226
|
+
end
|
|
227
|
+
|
|
210
228
|
# Run a bash script inside a running container. The script is passed to
|
|
211
229
|
# `bash -lc` as a single argument; values that vary (keys, usernames) are
|
|
212
230
|
# passed through the environment to avoid quoting pitfalls.
|
data/lib/gdkbox/provisioner.rb
CHANGED
|
@@ -205,6 +205,35 @@ module GDKBox
|
|
|
205
205
|
fi
|
|
206
206
|
BASH
|
|
207
207
|
|
|
208
|
+
# Static Docker CLI release installed into --docker boxes. A pinned
|
|
209
|
+
# version keeps provisioning reproducible; the static tarball needs no
|
|
210
|
+
# apt repo and exists for both x86_64 and aarch64.
|
|
211
|
+
DOCKER_CLI_VERSION = "27.5.1"
|
|
212
|
+
|
|
213
|
+
# Give a --docker box a working `docker` command against the mounted host
|
|
214
|
+
# socket: install the (arch-matched) static CLI, then grant the GDK user
|
|
215
|
+
# access to the socket via whatever group owns it — gid 0 on Docker
|
|
216
|
+
# Desktop, the docker group's gid on Linux hosts. Group membership takes
|
|
217
|
+
# effect on new login sessions, which is what dispatch/ssh create.
|
|
218
|
+
DOCKER_CLI_SETUP = <<~'BASH'
|
|
219
|
+
set -e
|
|
220
|
+
if ! command -v docker >/dev/null 2>&1; then
|
|
221
|
+
arch=$(uname -m)
|
|
222
|
+
curl -fsSL "https://download.docker.com/linux/static/stable/${arch}/docker-${GDKBOX_DOCKER_CLI_VERSION}.tgz" \
|
|
223
|
+
| tar -xz -C /tmp docker/docker
|
|
224
|
+
install -m 755 /tmp/docker/docker /usr/local/bin/docker
|
|
225
|
+
rm -rf /tmp/docker
|
|
226
|
+
fi
|
|
227
|
+
sock_gid=$(stat -c %g "$GDKBOX_DOCKER_SOCKET")
|
|
228
|
+
group=$(getent group "$sock_gid" | cut -d: -f1)
|
|
229
|
+
if [ -z "$group" ]; then
|
|
230
|
+
group=gdkbox-docker
|
|
231
|
+
groupadd -g "$sock_gid" "$group"
|
|
232
|
+
fi
|
|
233
|
+
usermod -aG "$group" "$GDKBOX_USER"
|
|
234
|
+
docker --version
|
|
235
|
+
BASH
|
|
236
|
+
|
|
208
237
|
def initialize(docker:, config:)
|
|
209
238
|
@docker = docker
|
|
210
239
|
@config = config
|
|
@@ -242,6 +271,18 @@ module GDKBox
|
|
|
242
271
|
)
|
|
243
272
|
end
|
|
244
273
|
|
|
274
|
+
def setup_docker_cli(container_name)
|
|
275
|
+
@docker.exec(
|
|
276
|
+
container_name, DOCKER_CLI_SETUP,
|
|
277
|
+
user: "root",
|
|
278
|
+
env: {
|
|
279
|
+
"GDKBOX_USER" => @config.ssh_user,
|
|
280
|
+
"GDKBOX_DOCKER_SOCKET" => Config::DOCKER_SOCKET,
|
|
281
|
+
"GDKBOX_DOCKER_CLI_VERSION" => DOCKER_CLI_VERSION
|
|
282
|
+
}
|
|
283
|
+
)
|
|
284
|
+
end
|
|
285
|
+
|
|
245
286
|
def setup_vite_port(container_name, port)
|
|
246
287
|
@docker.exec(
|
|
247
288
|
container_name, VITE_PORT_SETUP,
|
data/lib/gdkbox/store.rb
CHANGED
|
@@ -41,7 +41,9 @@ module GDKBox
|
|
|
41
41
|
|
|
42
42
|
# Every host port already claimed by an existing box.
|
|
43
43
|
def used_ports
|
|
44
|
-
all.flat_map
|
|
44
|
+
all.flat_map do |box|
|
|
45
|
+
[box["ssh_port"], box["web_port"], box["vite_port"], *box["extra_ports"]]
|
|
46
|
+
end.compact
|
|
45
47
|
end
|
|
46
48
|
end
|
|
47
49
|
end
|
data/lib/gdkbox/version.rb
CHANGED
|
@@ -240,6 +240,34 @@ Free a box stranded by a dead orchestrator (check `gdkbox ls` first):
|
|
|
240
240
|
gdkbox release pool-2 --force
|
|
241
241
|
```
|
|
242
242
|
|
|
243
|
+
## Running Duo agentic flows (needs `--docker`)
|
|
244
|
+
|
|
245
|
+
Duo Agent Platform flows finish in a docker-executor CI job, so a flow-capable
|
|
246
|
+
box needs Docker access — this is the sanctioned use of `--docker` (see the
|
|
247
|
+
guardrail below; keep such boxes confined to the flow work). Provision it as
|
|
248
|
+
its own claimed box:
|
|
249
|
+
|
|
250
|
+
```sh
|
|
251
|
+
gdkbox up duo-1 --owner "$OWNER" --ttl 14400 --docker --publish 5052 50052
|
|
252
|
+
gdkbox set-gcloud duo-1 # Vertex AI credentials for the flow model
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
Then, inside the box (dispatch these as tasks or run them over `gdkbox ssh`),
|
|
256
|
+
follow the README's "Running Duo agentic flows in a box" steps — in short:
|
|
257
|
+
`gdk rake setup_ai_services` (~20 min, needs a valid `ANTHROPIC_API_KEY`);
|
|
258
|
+
put the workflow service's env in **`env.runit`** (not `.env` — the two
|
|
259
|
+
services read different files); disable `gitlab_http_router` and enable nginx
|
|
260
|
+
(the router 403s the flow's WebSocket upgrade); set
|
|
261
|
+
`gitlab_ai_gateway.environment local` and run the `gitlab:duo:setup` /
|
|
262
|
+
`onboard_dap` seeds; add `network_mode = "gdkbox-duo-1"` to the generated
|
|
263
|
+
runner config (re-apply after `gdk reconfigure`). Verified end state: the
|
|
264
|
+
`developer` flow triggered from an issue in the seeded `gitlab-duo` project
|
|
265
|
+
completes and opens a Draft MR.
|
|
266
|
+
|
|
267
|
+
Plan for this box being expensive: the AI setup is ~20 minutes and the box
|
|
268
|
+
wants >10 GB free memory once the AI stack and flow containers run — size the
|
|
269
|
+
rest of the pool accordingly, and give the claim a generous `--ttl`.
|
|
270
|
+
|
|
243
271
|
## Guardrails
|
|
244
272
|
|
|
245
273
|
- **Don't exceed the box count the machine can handle** — each GDK box is
|
|
@@ -263,3 +291,10 @@ gdkbox release pool-2 --force
|
|
|
263
291
|
- **Boxes are isolated.** Dispatch runs with `--dangerously-skip-permissions`
|
|
264
292
|
by default because the work happens inside a disposable container; pass
|
|
265
293
|
`--no-yolo` if you need permission prompts honored.
|
|
294
|
+
- **`--docker` boxes are NOT isolated.** `up --docker` mounts the host Docker
|
|
295
|
+
socket, which is root-equivalent control of the host machine — the
|
|
296
|
+
isolation argument above does not hold there. Provision `--docker` when
|
|
297
|
+
the orchestration goal requires docker-executor CI (running Duo agentic
|
|
298
|
+
flows is the canonical case), and confine such boxes to that goal's own
|
|
299
|
+
tasks: never route unrelated, untrusted, or third-party work into a
|
|
300
|
+
`--docker` box, and prefer plain boxes for everything else.
|