rascal 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. checksums.yaml +4 -4
  2. data/.github/workflows/test.yml +4 -4
  3. data/.ruby-version +1 -1
  4. data/AGENTS.md +4 -0
  5. data/CHANGELOG.md +74 -0
  6. data/CONTRIBUTING.md +139 -0
  7. data/Gemfile.common +0 -3
  8. data/Gemfile.lock +1 -47
  9. data/Guardfile +0 -9
  10. data/README.md +679 -76
  11. data/Rakefile +1 -7
  12. data/lib/rascal/cli/base.rb +41 -20
  13. data/lib/rascal/cli/clean.rb +3 -8
  14. data/lib/rascal/cli/environments.rb +1 -3
  15. data/lib/rascal/cli/main.rb +106 -17
  16. data/lib/rascal/cli/run.rb +6 -7
  17. data/lib/rascal/cli/shell.rb +3 -3
  18. data/lib/rascal/cli/status.rb +81 -0
  19. data/lib/rascal/cli/update.rb +8 -8
  20. data/lib/rascal/cli.rb +1 -0
  21. data/lib/rascal/command_script.rb +83 -0
  22. data/lib/rascal/docker/command_containers.rb +101 -0
  23. data/lib/rascal/docker/container.rb +82 -101
  24. data/lib/rascal/docker/image.rb +71 -0
  25. data/lib/rascal/docker/interface.rb +4 -2
  26. data/lib/rascal/docker/network.rb +12 -0
  27. data/lib/rascal/docker/volume.rb +17 -10
  28. data/lib/rascal/docker.rb +11 -4
  29. data/lib/rascal/environment.rb +164 -40
  30. data/lib/rascal/gitlab/config_file.rb +199 -0
  31. data/lib/rascal/gitlab/environment_builder.rb +204 -0
  32. data/lib/rascal/gitlab/jobs.rb +299 -0
  33. data/lib/rascal/gitlab/variables.rb +155 -0
  34. data/lib/rascal/gitlab.rb +16 -0
  35. data/lib/rascal/io_helper.rb +11 -0
  36. data/lib/rascal/service.rb +52 -9
  37. data/lib/rascal/version.rb +1 -1
  38. data/lib/rascal.rb +2 -1
  39. data/rascal.gemspec +3 -2
  40. metadata +15 -8
  41. data/bin/cucumber +0 -29
  42. data/cucumber.yml +0 -1
  43. data/lib/rascal/environments_definition/gitlab.rb +0 -146
  44. data/lib/rascal/environments_definition.rb +0 -23
data/README.md CHANGED
@@ -18,130 +18,733 @@
18
18
 
19
19
  ### Use CI environments locally. Not quite a vagrant.
20
20
 
21
- We use Gitlab-CI to run CI for our projects, using Docker containers.
21
+ rascal gives you a development environment for a project without setting up the project on your
22
+ machine. It reads the project's `.gitlab-ci.yml`, starts the services a CI job needs (like
23
+ PostgreSQL or Redis) in Docker, and runs your commands in the job's Docker image, with your project
24
+ directory mounted.
25
+
26
+ You need nothing but Docker and rascal. Ruby, Node, databases and gems come from the images the
27
+ project already uses in CI, so an old project with Ruby 2.5 works next to a new one with Ruby 4.
28
+
29
+ ```sh
30
+ rascal run rspec -- bundle install # set up
31
+ rascal run rspec -- bundle exec rspec spec/models/user_spec.rb # work
32
+ rascal clean --all --volumes # tear down
33
+ ```
34
+
35
+ rascal is meant for people and for AI agents alike: `rascal run` needs no terminal, keeps the
36
+ command's output separate from its own, exits with the command's status, and explains what is wrong
37
+ when something is missing.
38
+
39
+
40
+ ## Contents
41
+
42
+ - [Installation](#installation)
43
+ - [How it works](#how-it-works)
44
+ - [Setting up a project](#setting-up-a-project)
45
+ - [Working in the environment](#working-in-the-environment)
46
+ - [Common adjustments](#common-adjustments)
47
+ - [Tearing down](#tearing-down)
48
+ - [Commands](#commands)
49
+ - [Configuration](#configuration)
50
+ - [What rascal reads from .gitlab-ci.yml](#what-rascal-reads-from-gitlab-ciyml)
51
+ - [Troubleshooting](#troubleshooting)
52
+ - [Working on rascal](#working-on-rascal)
22
53
 
23
- In certain situations, it can be helpful to set up an environment identical to a CI
24
- node on your local machine. `rascal` allows you to do just that by parsing your CI config
25
- (currently only `.gitlab-ci.yml`), starting required services and bringing up a Docker
26
- container.
27
54
 
28
55
  ## Installation
29
56
 
30
- Install with
57
+ Install rascal with a Ruby of your own, not into the project's bundle. rascal needs neither the
58
+ project's Ruby nor its gems, so one installation works for all projects, old and new. It needs
59
+ Ruby 3.3 or newer.
60
+
61
+ Ruby version managers like mise or rbenv switch to the project's Ruby inside a project directory
62
+ (from `.ruby-version`), and that Ruby usually does not have rascal. So pick one Ruby for rascal, and
63
+ use it to install and to run rascal:
31
64
 
32
- $ gem install rascal
65
+ ```sh
66
+ # With mise
67
+ mise exec ruby@4.0.9 -- gem install rascal
68
+ mise exec ruby@4.0.9 -- rascal run rspec -- bundle install
69
+
70
+ # With rbenv
71
+ RBENV_VERSION=4.0.9 gem install rascal
72
+ RBENV_VERSION=4.0.9 rascal run rspec -- bundle install
33
73
 
74
+ # Without a version manager
75
+ gem install rascal
76
+ rascal run rspec -- bundle install
77
+ ```
34
78
 
35
- ## Caveats
79
+ The examples below just say `rascal`. If rascal is already installed, mise and rbenv name the Ruby
80
+ that has it when you run `rascal` in a project whose Ruby does not. Check the version with
81
+ `rascal --version`. This README describes the latest version.
36
82
 
37
- *This is an early alpha version. Use at your own risk.*
83
+ You also need:
38
84
 
39
- Only the parsing of `.gitlab-ci.yml` is currently supported, and only a subset of the possible syntax
40
- will be properly interpreted.
85
+ - Docker. On macOS, Docker Desktop works.
86
+ - Access to the registries of the project's images. For a private registry, log in first with
87
+ `docker login registry.example.com`.
41
88
 
89
+ The job's image needs `bash`. Most CI images have it.
42
90
 
43
- ## Usage
44
91
 
45
- You need to add some extra information to your `.gitlab-ci.yml`. A working version might look like this:
92
+ ## How it works
46
93
 
94
+ Every top-level job in `.gitlab-ci.yml`, like `rspec`, becomes an environment of the same name.
95
+ `rascal environments` lists them.
96
+
97
+ When you run a command in an environment, rascal:
98
+
99
+ 1. pulls the job's image and the images of its services, if they are missing;
100
+ 2. starts the job's services on a Docker network, reachable under their aliases (like `db` or
101
+ `redis`), unless they are running already. It waits until each service that is still starting
102
+ accepts connections on one of its ports;
103
+ 3. starts a new container from the job's image, with the job's variables, and your project
104
+ directory mounted at `/repo`;
105
+ 4. runs a bash script in it: the `before_shell` commands from the `.rascal` block, your command,
106
+ then the `after_shell` commands;
107
+ 5. gives files in your project that belong to root back to you, since the container usually runs
108
+ as root;
109
+ 6. removes the container, and exits with your command's status.
110
+
111
+ **What stays between commands:** the services keep running, so a database you set up once keeps
112
+ its data. Files in your project directory and in volumes stay too. Everything else in the job's
113
+ container is gone after each command, including its home directory. So settings like
114
+ `bundle config set` must go into the project (`bundle config set --local`) or into variables, see
115
+ [Settings that need to persist](#settings-that-need-to-persist).
116
+
117
+ **Services are reused as they are.** If you change a service's settings, or pull newer images with
118
+ `rascal update`, run `rascal clean <job>` so the next command starts the services anew.
119
+
120
+ rascal does not run the job's `script`, `before_script` or `cache`. These are written for one-off CI
121
+ jobs, so rascal leaves setup to you, or to `before_shell`.
122
+
123
+ **rascal names its Docker resources after the project directory's name**, not its path. Two checkouts
124
+ with the same directory name share their environment, including databases, and `rascal clean` in one
125
+ removes the other's. Give each checkout its own directory name, like `myapp-feature-x`. Prefer clones
126
+ over Git worktrees: Git commands in the container cannot see a worktree's repository.
127
+
128
+
129
+ ## Setting up a project
130
+
131
+ ### 1. Pick the environment
132
+
133
+ ```console
134
+ $ rascal environments
135
+ deploy
136
+ rspec
47
137
  ```
48
- # settings here override job settings
49
- .rascal: # add rascal specific config here
50
- repo_dir: /repo # /repo is the default
51
- variables: # extra env variables
52
- BUNDLE_PATH: /cache/bundle
53
- volumes: # mount these volumes
54
- cache: /cache # we will always mount a /builds volume
138
+
139
+ Pick the job that runs the tests, usually called `rspec`, `test` or similar. Its image has the
140
+ project's language versions and tools, and its services are what the tests need. The list includes
141
+ jobs rascal cannot start, like deploy jobs.
142
+
143
+ ### 2. Look at the environment
144
+
145
+ `rascal status` shows what rascal made of the job's config, without starting or downloading
146
+ anything:
147
+
148
+ ```console
149
+ $ rascal status rspec
150
+ rspec
151
+ Image: registry.example.com/myapp/test:2.0 (not downloaded, the next command downloads it)
152
+ Network: rascal-myapp-rspec (does not exist)
153
+ Services:
154
+ db: postgres:16 (not downloaded), not created, the next command starts it
155
+ redis: redis:7 (not downloaded), not created, the next command starts it
156
+ Volumes:
157
+ /repo: /home/me/myapp
158
+ /builds: volume rascal-myapp-rspec-builds (does not exist)
159
+ Variables:
160
+ BUNDLE_PATH=tmp/cache/bundle
161
+ CI=true
162
+ ...
55
163
  before_shell:
56
- - bundle check # run this when starting a shell
57
- jobs:
58
- rspec: # override settings for a specific job
59
- variables:
60
- BUNDLE_GEMFILE = /repo/Gemfile
61
-
62
- .environment: &environment
63
- image: registry.makandra.de/makandra/ci-images/test-env:2.5
64
- services:
65
- - name: registry.makandra.de/makandra/ci-images/pg:9.5
66
- alias: pg-db
67
- - name: registry.makandra.de/makandra/ci-images/redis:4
68
- alias: redis
69
- before_script:
70
- - ruby -v
71
- - bundle install
72
- - bundle exec rake db:create db:schema:load
73
- variables:
74
- BUNDLE_PATH: ./bundle/vendor
75
- DATABASE_URL: postgresql://pg_user@pg-db/test-db
76
- REDIS_URL: redis://redis
77
- PROMPT: CI env
78
- cache:
79
- paths:
80
- - ./bundle/vendor
164
+ bundle check > /dev/null || bundle install
165
+ ssh_agent: false, services_timeout: 30
166
+ ```
167
+
168
+ Check three things before you go on:
169
+
170
+ - **Problems** keep the environment from starting. Each says what to do, see also
171
+ [Troubleshooting](#troubleshooting).
172
+ - **`before_shell`** runs before each of your commands. The project's `.rascal` block may already
173
+ install dependencies there.
174
+ - **`BUNDLE_PATH`** keeps gems in the project, so they stay installed between commands. Most CI
175
+ configs set it. If it is missing, see [Settings that need to persist](#settings-that-need-to-persist)
176
+ before you install gems.
177
+
178
+ Later, `rascal status` also shows whether each service runs or how it exited, with the command to see
179
+ its logs. With `--json`, it prints the same as JSON.
180
+
181
+ ### 3. Run it once
182
+
183
+ ```console
184
+ $ rascal run rspec -- true
185
+ Downloading image for myapp-rspec
186
+ ...
187
+ Starting container for myapp-rspec_db
188
+ Starting container for myapp-rspec_redis
189
+ ```
190
+
191
+ The first run downloads the images, which can take several minutes. If you run rascal with a
192
+ timeout, give this run a generous one.
81
193
 
194
+ ### 4. Install dependencies and set up the database
82
195
 
83
- # ============= Actual jobs ================
196
+ Look at the job's `before_script` and `script` in `.gitlab-ci.yml` to see how CI does it, and run
197
+ the equivalent steps. For a typical Rails project:
84
198
 
85
- rspec:
86
- <<: *environment
87
- script:
88
- - bundle exec rake knapsack:rspec
89
- parallel: 4
199
+ ```sh
200
+ rascal run rspec -- bundle install
201
+ rascal run rspec -- bin/rails db:create db:schema:load
90
202
  ```
91
203
 
92
- Then, in your project root, run `rascal shell rspec`.
204
+ Leave out steps that only make sense on a CI runner, like ones that use `CI_JOB_TOKEN` to fetch
205
+ private repositories, and fetch private dependencies [over SSH](#private-dependencies-over-ssh)
206
+ instead. Steps like `cp config/database.ci.yml config/database.yml` change your project directory.
207
+ Run them if the file is missing. If it exists, it may be a local file that Git ignores, so compare
208
+ it with the CI version before you replace it.
93
209
 
210
+ You only need the steps for what you want to run. A model spec may need only gems and a database,
211
+ while browser tests may also need Node packages (`yarn install`) and compiled assets.
94
212
 
95
- ### Commands
213
+ **Setup can change files in Git.** For example, `bundle install` may rewrite `Gemfile.lock` if the
214
+ Gemfile treats CI differently (rascal sets `CI` and `GITLAB_CI`, see [Variables](#variables)). Check
215
+ `git status` after setting up, and undo changes you did not intend before you commit anything.
216
+ Some changes come back with every command. If you need to change dependencies, see
217
+ [Changing dependencies](#changing-dependencies).
96
218
 
97
- #### rascal shell <job>
219
+ ### 5. Make it repeatable (optional)
98
220
 
99
- Start a docker container (plus required services) and open an interactive shell.
221
+ Commands in `before_shell` run before every command, so the environment is always ready, even after
222
+ a teardown or when someone else uses it. Add a `.rascal` block to `.gitlab-ci.yml`, which GitLab
223
+ ignores:
100
224
 
101
- Currently requires a "bash" to exist.
225
+ ```yaml
226
+ .rascal:
227
+ before_shell:
228
+ - bundle check > /dev/null || bundle install
229
+ - bin/rails db:create db:migrate
230
+ ```
102
231
 
232
+ All commands run in one bash script, one per line:
103
233
 
104
- #### rascal run <job> -- <command>
234
+ - A failing `before_shell` command does not stop your command, and does not change the exit status.
235
+ - Its output appears with your command's output on every run. Redirect it, like `> /dev/null`
236
+ above, to keep your command's output clean.
237
+ - `cd` or `export` in `before_shell` apply to your command too.
238
+ - It runs before every command, so keep it fast. The example above takes a few seconds, mostly to
239
+ boot Rails.
105
240
 
106
- Run a single command in a docker container (plus required services), then exit
107
- with the command's exit status.
241
+ If you do not want to change the project's CI config, put the same settings into a `.rascal.yml`
242
+ next to `.gitlab-ci.yml`, without the `.rascal:` key, and keep it out of Git:
243
+
244
+ ```sh
245
+ cat > .rascal.yml <<'YAML'
246
+ before_shell:
247
+ - bundle check > /dev/null || bundle install
248
+ YAML
249
+ echo .rascal.yml >> "$(git rev-parse --git-path info/exclude)"
250
+ ```
251
+
252
+ `.rascal.yml` overrides the `.rascal` block: its lists, like `before_shell`, replace the block's,
253
+ and its hashes, like `variables`, are merged into the block's. Wherever this README says "the
254
+ `.rascal` block", `.rascal.yml` works too. In a project you do not maintain, use `.rascal.yml`.
255
+
256
+
257
+ ## Working in the environment
258
+
259
+ Run any command with `rascal run`. Everything after `--` is passed on as arguments, so arguments
260
+ may contain spaces:
108
261
 
109
262
  ```sh
110
263
  rascal run rspec -- bundle exec rspec spec/models/user_spec.rb
264
+ rascal run rspec -- bundle exec rspec spec/models/user_spec.rb:42 --format documentation
265
+ rascal run rspec -- bin/rails runner 'puts User.count'
111
266
  ```
112
267
 
113
- Unlike `rascal shell`, this allocates no TTY and needs no interaction, so it can
114
- be used from scripts, git hooks, editors and AI agents. Everything after `--` is
115
- passed through as a list of arguments, so arguments may contain spaces (`-n 'a
116
- test name'`). To use shell syntax such as `&&` or `cd`, wrap it explicitly:
268
+ To use shell syntax like `&&`, `|` or `cd`, run it through bash:
117
269
 
118
270
  ```sh
119
- rascal run rspec -- bash -c 'cd subproject && bundle exec rspec'
271
+ rascal run rspec -- bash -c 'bin/rails db:migrate && bundle exec rspec spec/models'
120
272
  ```
121
273
 
122
- `before_shell` and `after_shell` still run around the command, but do not affect
123
- the exit status. Note that a failing `before_shell` command (e.g. `bundle check`
124
- in an environment that was never bundled) does not abort the run.
274
+ What to expect from `rascal run`:
275
+
276
+ - **Output:** stdout is your command's stdout (plus the output of `before_shell` and `after_shell`).
277
+ rascal's own messages, like `Starting container for ...`, image downloads and warnings, go to
278
+ stderr.
279
+ - **Input:** stdin is passed to the command when it comes from a file or a pipe, so
280
+ `rascal run rspec -- psql < dump.sql` works, or from a socket that already has input, as Node's
281
+ `child_process` provides it. A terminal, or a socket without input, is not passed on, so a command
282
+ waiting for input, like a forgotten `debugger` call, gets none instead of hanging. A pipe that
283
+ stays open without input still makes such a command wait.
284
+ - **Exit status:** your command's, or 128 plus the signal number if it was killed. If rascal itself
285
+ fails, for example for an unknown job, it exits with 1, like many failing commands, and explains why
286
+ on stderr. If Docker cannot start the container, for example because the image has no bash, the
287
+ status is 125 to 127. 127 is also what bash returns for a command the image does not have.
288
+ - **Stopping:** if you stop rascal, for example with a timeout, it removes the command's container.
289
+ Files the command created in the project may then still belong to root, until the next command
290
+ gives them back.
291
+ - **Environment variables** from your shell are not passed into the container. Use
292
+ `rascal run rspec --env FOO=bar -- bin/rails runner ...`, or the `.rascal` block to set them for
293
+ every command. `--unset-env NAME` removes one, like `CI`.
294
+ - **Parallel commands** in the same environment work, like several test runs at once. Each gets its
295
+ own container, and they share the services, including the test database. Two commands that load
296
+ the schema or run tests at the same time get in each other's way, unless the project's parallel
297
+ test setup gives each its own database.
298
+
299
+ For an interactive shell, like for a Rails console or a debugger, use `rascal shell rspec`. It runs
300
+ bash with `before_shell` and `after_shell` around it, just like a command. It needs a terminal.
301
+ Without one, like as an AI agent, debug with commands instead: run single examples, add output to
302
+ the code, or use `bin/rails runner` to inspect state.
303
+
304
+ Files you edit on your machine are visible in the container immediately. Files the container
305
+ creates, like `Gemfile.lock` changes or installed gems, belong to you afterwards (on Linux; Docker
306
+ Desktop handles this itself).
307
+
308
+ The container runs the job's image, but not the job's `before_script`, so tools that CI starts or
309
+ installs there, like a virtual display for browser tests, are missing unless you start them yourself.
310
+ rascal does not publish ports to your machine, so you cannot open a server running in the container
311
+ in your browser.
312
+
313
+ ### Changing dependencies
314
+
315
+ Update dependencies in the container, like you would on your machine:
316
+
317
+ ```sh
318
+ rascal run rspec -- bundle update rack --conservative
319
+ ```
320
+
321
+ Some Gemfiles leave out gems when `CI` is set, typically private gems that CI cannot fetch. Look
322
+ for `ENV['CI']` or `ENV.key?('CI')` in the Gemfile. Since rascal sets `CI` like GitLab does, every
323
+ Bundler command, even `bundle exec`, then writes a `Gemfile.lock` without these gems: their `GIT`
324
+ block disappears. To change dependencies in such a project, run the change without these variables,
325
+ and with your [SSH agent](#private-dependencies-over-ssh) to fetch the private gems:
326
+
327
+ ```sh
328
+ git checkout HEAD -- Gemfile.lock # start from the committed lockfile (commit changes to keep first)
329
+ rascal run rspec --unset-env CI --unset-env GITLAB_CI --ssh-agent -- bundle update rack --conservative
330
+ git add Gemfile.lock # keep this version
331
+ git diff --cached Gemfile.lock
332
+ ```
333
+
334
+ The diff should only show the changes you intended. Run tests as usual, with `CI` set, since
335
+ projects often check it elsewhere too, like for eager loading. Each Bundler command removes the
336
+ private gems from `Gemfile.lock` again, so before you commit, restore the version you staged with
337
+ `git checkout Gemfile.lock`.
338
+
339
+
340
+ ## Common adjustments
341
+
342
+ Settings for the `.rascal` block or `.rascal.yml` that many projects need. See
343
+ [Configuration](#configuration) for all of them.
344
+
345
+ ### Settings that need to persist
346
+
347
+ Each command runs in a new container, which is gone afterwards with everything outside the project
348
+ directory and volumes: gems installed elsewhere, and settings made with `bundle config set` (without
349
+ `--local`). So set what the project needs on every command as variables in the `.rascal` block:
350
+
351
+ ```yaml
352
+ variables:
353
+ BUNDLE_WITHOUT: deploy # instead of `bundle config set without deploy`
354
+ BUNDLE_PATH: tmp/bundle # if the job does not keep gems in the project
355
+ ```
356
+
357
+ To keep gems out of the project directory, use a volume instead:
358
+
359
+ ```yaml
360
+ volumes:
361
+ gems: /gems
362
+ variables:
363
+ BUNDLE_PATH: /gems
364
+ ```
365
+
366
+ ### Private dependencies over SSH
367
+
368
+ If `bundle install` or `yarn install` fetches private Git repositories over SSH, give the commands
369
+ that need it access to your SSH agent:
370
+
371
+ ```sh
372
+ rascal run rspec --ssh-agent -- bundle install
373
+ ```
374
+
375
+ rascal then mounts your SSH agent and your `~/.ssh/known_hosts` into the container. It does not
376
+ mount `~/.ssh/config`, so host aliases from there do not work. To give the agent to every command,
377
+ set `ssh_agent: true` in the `.rascal` block. `--no-ssh-agent` keeps it from a single command.
378
+
379
+ **While a command runs, any code in the container can use all keys in the agent**, including
380
+ install scripts of dependencies. So give it only to the commands that need it. With Docker Engine on
381
+ Linux, you can also start a separate agent that only holds a key with read access to your Git
382
+ server, and point rascal to it:
383
+
384
+ ```sh
385
+ ssh-agent -a /tmp/git-read-agent.sock > /dev/null
386
+ SSH_AUTH_SOCK=/tmp/git-read-agent.sock ssh-add ~/.ssh/git_read_key
387
+ SSH_AUTH_SOCK=/tmp/git-read-agent.sock rascal run rspec --ssh-agent -- bundle install
388
+ ```
389
+
390
+ Docker Desktop (on macOS, and on Linux) always provides the agent of your desktop session, with all
391
+ its keys, and ignores `SSH_AUTH_SOCK`.
392
+
393
+ **If you are an AI agent,** ask your user before you give a command your SSH agent, unless a
394
+ separate agent with a limited key is available.
395
+
396
+ The container needs an SSH client, and must run as root (or as your user), since the agent's socket
397
+ belongs to you.
398
+
399
+ ### Variables rascal cannot see
400
+
401
+ GitLab sets variables like `CI_REGISTRY_IMAGE` or `CI_COMMIT_SHA`, and projects can define more in
402
+ their CI/CD settings. rascal only knows a few of GitLab's variables (see
403
+ [Variables](#variables)). If a job needs others, define them in the `.rascal` block:
404
+
405
+ ```yaml
406
+ variables:
407
+ CI_REGISTRY_IMAGE: registry.example.com/group/myapp
408
+ ```
409
+
410
+ rascal warns about variables that reference undefined variables, and refuses to start a job whose
411
+ image name does.
412
+
413
+ ### An image's entrypoint that gets in the way
414
+
415
+ rascal runs commands through the image's entrypoint, like CI does. Some images have an entrypoint
416
+ made for another setup, like the project's own Docker development setup, which refuses to run as
417
+ root or expects volumes that only that setup provides. Skip it for rascal in `.rascal.yml`:
418
+
419
+ ```yaml
420
+ jobs:
421
+ rspec:
422
+ image:
423
+ entrypoint: ['']
424
+ ```
425
+
426
+ The job keeps its image: an image without a `name` only changes the parts it gives, while one with a
427
+ `name` replaces the image as a whole. Once the image is downloaded, `rascal status` shows its entrypoint. To see
428
+ what it did, look at it, like `rascal run rspec -- cat /docker-entrypoint.sh`, and do what your
429
+ commands need yourself.
430
+
431
+
432
+ ## Tearing down
433
+
434
+ ```console
435
+ $ rascal clean --all --volumes
436
+ Stopping container for myapp-rspec_db
437
+ Removing container for myapp-rspec_db
438
+ Stopping container for myapp-rspec_redis
439
+ Removing container for myapp-rspec_redis
440
+ Removing network rascal-myapp-rspec
441
+ Removing volume rascal-myapp-rspec-builds
442
+ ```
443
+
444
+ `rascal clean <job>` stops and removes the job's service containers, including their data, and the
445
+ job's Docker network. With `--volumes`, it also removes the job's named volumes: the volume for
446
+ `/builds`, and the volumes from the `.rascal` block. `--all` does this for every environment of the
447
+ project.
448
+
449
+ These stay:
450
+
451
+ - files in your project directory, like gems installed into it, and your `.rascal.yml`;
452
+ - the downloaded images, which other projects may use too. Remove them with `docker image rm`;
453
+ - an empty lock file per job in the temp directory, like `/tmp/rascal-myapp-rspec.lock`.
454
+
455
+ To check that nothing is left, list what Docker has for the project (`myapp` being the name of the
456
+ project directory):
457
+
458
+ ```sh
459
+ docker ps --all --filter name=rascal-myapp-
460
+ docker network ls --filter name=rascal-myapp-
461
+ docker volume ls --filter name=rascal-myapp-
462
+ ```
463
+
464
+
465
+ ## Commands
466
+
467
+ Commands that work on an environment take the job's name. All look for `.gitlab-ci.yml` in the
468
+ current directory.
469
+
470
+ | Command | What it does |
471
+ |---|---|
472
+ | `rascal run <job> -- <command> [args...]` | Runs a command, without a terminal, and exits with its status. |
473
+ | `rascal shell <job>` | Opens an interactive bash. Needs a terminal. |
474
+ | `rascal environments` | Lists the project's jobs, one per line, including ones rascal cannot start. |
475
+ | `rascal status [<job>] [--json]` | Shows what rascal made of the job's config, and what of it exists in Docker (see [Troubleshooting](#troubleshooting)). Without a job, shows all. |
476
+ | `rascal clean <job> [--volumes]` | Stops and removes the job's services and network, and with `--volumes` its volumes. |
477
+ | `rascal clean --all [--volumes]` | The same for all environments of the project. |
478
+ | `rascal update <job>` | Pulls the latest versions of the job's images. Run `rascal clean <job>` afterwards to restart services with them. |
479
+ | `rascal update --all` | The same for all environments. Skips jobs whose images rascal cannot resolve. |
480
+ | `rascal --version` | Prints rascal's version. |
481
+
482
+ Options go before `--`, before or after the job's name.
483
+
484
+ With `--config-file PATH` (or `-c PATH`), rascal uses another `.gitlab-ci.yml`, or the one in another
485
+ directory. The directory of that file counts as the project directory: rascal mounts it, and looks
486
+ for `.rascal.yml` and local includes there.
487
+
488
+ `run` and `shell` take options that change only this command's container. Services keep their
489
+ settings.
490
+
491
+ | Option | What it does |
492
+ |---|---|
493
+ | `--env NAME=VALUE` | Sets a variable. Repeat it for more. |
494
+ | `--unset-env NAME` | Removes a variable, like `CI`. Repeat it for more. |
495
+ | `--ssh-agent` | Gives the container your SSH agent, like `ssh_agent: true` (see [Private dependencies over SSH](#private-dependencies-over-ssh)). |
496
+
497
+ ### Docker resources
498
+
499
+ rascal names everything after the project directory and the job:
500
+
501
+ | Resource | Name |
502
+ |---|---|
503
+ | Network | `rascal-myapp-rspec` |
504
+ | Service container | `rascal-myapp-rspec_db`, after the service's first alias |
505
+ | Container of a command | `rascal-myapp-rspec-run-` and a random suffix, removed after the command |
506
+ | Volume for `/builds` | `rascal-myapp-rspec-builds` |
507
+ | Volume from the `.rascal` block | `rascal-myapp-rspec-cache`, after its name |
508
+
509
+ Only the name of the directory counts, not its path (see [How it works](#how-it-works)).
510
+
511
+
512
+ ## Configuration
513
+
514
+ rascal takes its settings from the `.rascal` block in `.gitlab-ci.yml`, and from a `.rascal.yml`
515
+ next to it. Both use the same keys:
516
+
517
+ ```yaml
518
+ .rascal:
519
+ # Commands to run in the same bash script before and after every command, and around
520
+ # `rascal shell`. Failures do not stop the command.
521
+ before_shell:
522
+ - bundle check > /dev/null || bundle install
523
+ after_shell: []
524
+
525
+ # Variables for all jobs. They override the jobs' variables, and may reference them:
526
+ # RUBYOPT: $RUBYOPT -W:deprecated extends the job's RUBYOPT. Only variables from the
527
+ # CI config count, not those the image sets, so do not extend PATH this way.
528
+ variables:
529
+ BUNDLE_WITHOUT: deploy
530
+
531
+ # Docker volumes as name: path in the container. Each job gets its own, which keep their
532
+ # content between commands, until `rascal clean --volumes`.
533
+ volumes:
534
+ cache: /cache
535
+
536
+ # Where the project directory appears in the container. Default: /repo
537
+ repo_dir: /repo
538
+
539
+ # Give the container access to your SSH agent. Default: false
540
+ ssh_agent: false
541
+
542
+ # How many seconds to wait for each service to accept connections. Default: 30
543
+ services_timeout: 30
544
+
545
+ # Settings for single jobs. Besides the keys above, you can override any key of the job,
546
+ # like `image` or `services`.
547
+ jobs:
548
+ rspec:
549
+ variables:
550
+ PARALLEL_TEST_PROCESSORS: '2'
551
+ deploy:
552
+ hide: true # do not list the job as an environment
553
+ old-name:
554
+ name: new-name # use another name for the environment
555
+ ```
556
+
557
+ Keys in the `.rascal` block replace the job's keys of the same name, for every job. Hashes, like
558
+ `variables`, are merged. Use `jobs:` to change a single job.
559
+
560
+
561
+ ## What rascal reads from .gitlab-ci.yml
562
+
563
+ Every top-level key is a job, except for GitLab's global keywords (`default`, `include`, `stages`,
564
+ `variables`, `workflow`, and the deprecated `image`, `services`, `cache`, `before_script` and
565
+ `after_script`) and keys starting with a dot.
566
+
567
+ For each job, rascal combines its config like GitLab does, then applies rascal's settings on top:
568
+
569
+ 1. `default:` (and the deprecated top-level keys), limited by the job's `inherit: default:`
570
+ 2. the templates in the job's `extends:`, merged in order
571
+ 3. the job itself. Each key from steps 2 and 3 replaces the same key from `default:` as a whole.
572
+ 4. the `.rascal` block, overridden by `.rascal.yml`
573
+ 5. the job's entry under `jobs:` in the `.rascal` block or `.rascal.yml`
574
+
575
+ From the combined config, rascal uses `image` (its name and `entrypoint`), `services` and
576
+ `variables`. It ignores everything else, like `script`, `rules` or `cache`.
577
+
578
+ rascal reads local files from `include:`, including wildcards and nested includes, and resolves
579
+ `!reference` tags. It cannot read includes from other projects, URLs, templates or components, and
580
+ warns about them, since config from them is missing. It cannot evaluate `rules:` on includes either,
581
+ so it includes local files regardless (or skips them if they are missing). A file may start with a
582
+ header document with `spec:`, but rascal does not interpolate `$[[ inputs ]]`.
583
+
584
+ ### Services
585
+
586
+ Services may be given as a plain image name (`- postgres:16`) or with `name:`, `alias:` (one or
587
+ more, separated by commas or spaces), `command:` and `variables:`. Without an `alias`, a service is
588
+ reachable under the host names GitLab derives from its image, like `postgres` for `postgres:16`.
589
+ Your project directory is mounted into the services too.
590
+
591
+ After starting services, rascal waits until each accepts connections on one of the TCP ports its
592
+ image exposes, or on `HEALTHCHECK_TCP_PORT` from the service's own variables, for up to
593
+ `services_timeout` seconds. Then it runs the command anyway. It also waits for services another
594
+ command started less than `services_timeout` seconds ago, so parallel commands work on a fresh
595
+ environment. Services that expose no port are not waited for.
596
+
597
+ ### Variables
598
+
599
+ Variables are resolved like in a GitLab job:
600
+
601
+ - Global `variables:` (limited by the job's `inherit: variables:`) and job variables are merged by
602
+ name. The `.rascal` block and its `jobs:` entries override them, and may extend them by
603
+ referencing their own name.
604
+ - Values, image names and service names may reference variables as `$NAME`, `${NAME}` or `%NAME%`.
605
+ `$$` stands for a literal `$`. A variable defined as `{ value: ..., expand: false }` is used as is.
606
+ - References only see variables from the CI config and the ones rascal predefines, not variables the
607
+ image sets.
608
+ - rascal predefines `CI=true`, `GITLAB_CI=true`, `CI_JOB_NAME`, `CI_BUILDS_DIR` (`/builds`),
609
+ `CI_PROJECT_DIR` (`/repo`, or your `repo_dir`) and `CI_PROJECT_NAME` (the name of your project
610
+ directory). Tools may behave differently when `CI` is set. For example, `pnpm install` refuses to
611
+ change the lockfile. Override them in the `.rascal` block if needed, or remove them with
612
+ `CI: null`, since some tools only check whether `CI` is set.
613
+ - An undefined `$NAME` in a variable's value becomes an empty string, like in GitLab, and rascal
614
+ warns about it. An undefined `%NAME%` stays as it is. An undefined variable in an image name keeps
615
+ rascal from starting the job.
616
+
617
+
618
+ ## Troubleshooting
619
+
620
+ rascal explains most problems itself. These are the messages you may see, and what to do.
621
+
622
+ Run `rascal status <job>` to see what rascal made of the job's config, and what of it runs (see
623
+ [Look at the environment](#2-look-at-the-environment)).
624
+
625
+ #### `Could not find a .gitlab-ci.yml in ...`
626
+
627
+ Run rascal in the project's root directory, or pass `--config-file`.
628
+
629
+ #### `Unknown job ... Jobs: ...`
630
+
631
+ Use one of the listed job names. Jobs starting with a dot are templates, which have no environment.
632
+
633
+ #### `has no image`
634
+
635
+ The job gets its image from somewhere rascal cannot see, often an include from another project.
636
+ Set it in the `.rascal` block: `jobs: <job>: image: ...`.
637
+
638
+ #### `uses the undefined variable`
639
+
640
+ The image of the job or of a service references a variable rascal does not know, often one GitLab
641
+ sets, like `CI_REGISTRY_IMAGE`. Define it under `variables:` in the `.rascal` block, or set the image
642
+ there.
643
+
644
+ #### ``unknown keys in `extends` `` or `could not be found`
645
+
646
+ A template or `!reference` comes from a file rascal cannot read. Set the config the job needs in the
647
+ `.rascal` block, or hide the job with `jobs: <job>: hide: true` if you do not need it.
648
+
649
+ #### `rascal cannot read these includes`
650
+
651
+ The job may lack services or variables from includes of other projects, URLs or templates. If it
652
+ does, set them in the `.rascal` block. Otherwise, ignore the warning.
653
+
654
+ #### ``uses `spec: inputs:` ``
655
+
656
+ rascal does not interpolate `$[[ inputs.name ]]`. Set the value in the `.rascal` block.
657
+
658
+ #### `which is undefined and expanded to an empty string`
659
+
660
+ Harmless if the command does not need the variable. Otherwise define it in the `.rascal` block.
661
+
662
+ #### `Docker may need credentials for ...`
663
+
664
+ Run the `docker login` command from the message.
665
+
666
+ #### `Docker found no image ... in its registry`
667
+
668
+ The image named in the CI config is gone from its registry, often because an old tag was deleted,
669
+ so the project's CI pipeline is probably broken too. Find an existing tag in the registry, like on
670
+ the Container Registry page of the GitLab project, preferably one with the Ruby and Node versions
671
+ the project uses (`.ruby-version`, `.tool-versions`). Set it in the `.rascal` block, like
672
+ `jobs: <job>: image: ...`, or override the job's `services:` there. If the project builds its CI
673
+ image itself, like from a `Dockerfile`, you can also build it under the name from the message.
674
+ `rascal update` cannot update such an image.
675
+
676
+ #### `rascal is not provided by the active ruby` or `rascal: command not found` inside a project
677
+
678
+ A Ruby version manager switched to the project's Ruby, which does not have rascal. Run rascal with
679
+ the Ruby you installed it for, which mise's message names. See [Installation](#installation).
680
+
681
+ #### `Cannot connect to the Docker daemon` or `failed to connect to the docker API`
682
+
683
+ Start Docker.
684
+
685
+ #### Gems or packages are missing after they were installed
686
+
687
+ They were installed outside the project directory, in a container that is gone now. See
688
+ [Settings that need to persist](#settings-that-need-to-persist).
689
+
690
+ #### `does not accept connections ... continuing anyway`
691
+
692
+ The service needs longer to start, or listens on a port its image does not declare. Set
693
+ `services_timeout` in the `.rascal` block, or `HEALTHCHECK_TCP_PORT` in the service's variables.
694
+
695
+ #### `Service ... is not running anymore`
696
+
697
+ The service stopped, usually because of missing settings, like a database password. Its logs say
698
+ why: `rascal status <job>` shows the command, like `docker logs rascal-myapp-rspec_db`. Fix its variables in the `.rascal` block, then run `rascal clean <job>`.
699
+
700
+ #### `not a git repository` from Git in the container
701
+
702
+ The project is a Git worktree, whose `.git` points to the main repository outside the project
703
+ directory, which the container cannot see. Run Git on your machine instead, or use a clone. (In a
704
+ clone, Git works in the container: rascal tells it to trust the project, which belongs to you and
705
+ not to the container's user.)
706
+
707
+ #### `Host key verification failed` or `Permission denied (publickey)` when fetching dependencies
708
+
709
+ Enable `ssh_agent: true` (see [Private dependencies over SSH](#private-dependencies-over-ssh)), and
710
+ make sure your agent holds a key for the Git server (`ssh-add -l`) and that `~/.ssh/known_hosts`
711
+ knows the server.
712
+
713
+ #### The image's entrypoint fails or refuses to run before your command
714
+
715
+ Messages like `Refusing to run as root` come from the image's entrypoint, not from rascal or your
716
+ command. See [An image's entrypoint that gets in the way](#an-images-entrypoint-that-gets-in-the-way).
125
717
 
126
- Currently requires a "bash" to exist.
718
+ #### Tests pass, but an `after(:suite)` hook of Knapsack fails
127
719
 
720
+ Many projects split their tests across CI machines with Knapsack, and turn it on when `CI` is set,
721
+ which rascal sets like GitLab. Knapsack then expects files that CI provides, like
722
+ `knapsack_rspec_report.json`, and fails the run after all tests passed. Look at how the project turns
723
+ on Knapsack, usually in `spec/spec_helper.rb`. If only `CI` turns it on, run the tests with
724
+ `--unset-env CI` (keeping in mind that other code may check `CI` too), or give Knapsack the file the
725
+ message names, outside Git.
128
726
 
129
- #### rascal environments
727
+ #### `"bash": executable file not found`
130
728
 
131
- Print the names of all available (non-hidden) environments, one per line, so that
132
- scripts can discover them without parsing an error message.
729
+ The job's image has no bash, so rascal cannot use it. Use another job, or set another image for the
730
+ job in the `.rascal` block.
133
731
 
732
+ #### The database or another service is in a bad state
134
733
 
135
- #### rascal clean <job> | --all [--volumes]
734
+ Run `rascal clean <job>` to remove the services and their data. The next command starts them
735
+ fresh.
136
736
 
137
- Stop and remove all created containers, services, networks for either the given or all jobs.
737
+ #### Leftovers from a project directory you deleted
138
738
 
139
- If `--volumes` is given, also remove all cached volumes.
739
+ Find them with `docker ps --all --filter name=rascal-`, `docker network ls --filter name=rascal-`
740
+ and `docker volume ls --filter name=rascal-`. Remove them with `docker rm --force`,
741
+ `docker network rm` and `docker volume rm`.
140
742
 
141
743
 
142
- #### rascal update <job> | --all
744
+ ## Working on rascal
143
745
 
144
- Update all images for the given job, or all jobs.
746
+ See [CONTRIBUTING.md](CONTRIBUTING.md) for how rascal is built, how to run its tests and where
747
+ changes go.
145
748
 
146
749
 
147
750
  ## License