gitlab-qa 14.18.0 → 14.19.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/docs/architecture.md +2 -2
- data/docs/developer/style_guide.md +1 -1
- data/docs/release_process.md +10 -9
- data/docs/run_qa_against_gdk.md +43 -39
- data/docs/running_against_remote_grid.md +2 -2
- data/docs/{specifics_for_macos_m1_m2.md → troubleshooting.md} +8 -5
- data/lib/gitlab/qa/component/ai_gateway.rb +2 -1
- data/lib/gitlab/qa/version.rb +1 -1
- metadata +3 -5
- data/docs/environment_setup/users.md +0 -9
- data/docs/waits.md +0 -24
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 3d5b82d626da25ea72196cade9499553040e8fb3dadc5b1afbbceef06ba1abb1
|
4
|
+
data.tar.gz: 3a16cb5354eebd63a097a1f64269d4ad5cc6b4995069fb0ad30e38fd6b418758
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 3fef225828d82dafce07623c0fda78ce2b40676c20415e0246133e6ca5a71111f0a9d0b8353ee10ac845613dab94d0d2d0e499826dcf9f29d36d9725e3b17ddf
|
7
|
+
data.tar.gz: e80f04f59c0863114455bf2e8daf7806e9f3fd1d4d21c3900de4d500678c55075f022ae09d265e15fa444c18ceb7ea4c03071c10438f9d403e145db6e7aa28b4
|
data/Gemfile.lock
CHANGED
data/docs/architecture.md
CHANGED
@@ -1,8 +1,8 @@
|
|
1
1
|
# GitLab QA architecture
|
2
2
|
|
3
|
-
|
3
|
+
![GitLab QA architecture diagram](https://docs.google.com/drawings/d/e/2PACX-1vTBlBcIBFFnQOd9xGQ7-2rij96mP2-ajNd9sQILhSI2D7x-q_8oMuO_GFG8-BWRoQZkVHWdQZTdXF8c/pub?w=855&h=2426)
|
4
4
|
|
5
|
-
_[
|
5
|
+
_[Edit diagram (for GitLab team members only)](https://docs.google.com/drawings/d/12mTCF7BU-xaxxS9MmIjkSYnf31RN4wgkkHcXTmS0NoA/edit)_
|
6
6
|
|
7
7
|
----
|
8
8
|
|
@@ -26,4 +26,4 @@ We have a `lefthook.yml` checked in but it is ignored until Lefthook is installe
|
|
26
26
|
bundle exec lefthook run pre-push
|
27
27
|
```
|
28
28
|
|
29
|
-
For a detailed guide on left hook configuration see https://github.com/evilmartians/lefthook/blob/master/docs/configuration.md
|
29
|
+
For a detailed guide on left hook configuration see <https://github.com/evilmartians/lefthook/blob/master/docs/configuration.md>
|
data/docs/release_process.md
CHANGED
@@ -4,13 +4,13 @@
|
|
4
4
|
|
5
5
|
We follow [Semantic Versioning](https://semver.org). In short, this means that the new version should reflect the types of changes that are about to be released.
|
6
6
|
|
7
|
-
|
8
|
-
|
9
|
-
MAJOR.MINOR.PATCH
|
10
|
-
|
11
|
-
- MAJOR version when you make incompatible API changes,
|
12
|
-
- MINOR version when you add functionality in a
|
13
|
-
- PATCH version when you make
|
7
|
+
> _Summary from semver.org_
|
8
|
+
>
|
9
|
+
> MAJOR.MINOR.PATCH
|
10
|
+
>
|
11
|
+
> - MAJOR version when you make incompatible API changes,
|
12
|
+
> - MINOR version when you add functionality in a backward compatible manner
|
13
|
+
> - PATCH version when you make backward compatible bug fixes.
|
14
14
|
|
15
15
|
## When we release
|
16
16
|
|
@@ -27,7 +27,7 @@ when we make a change - no matter the size of the change.
|
|
27
27
|
- Merge the merge request.
|
28
28
|
- The new version should automatically be tagged and pushed to Rubygems by the `gem-release` job in the merge commit pipeline.
|
29
29
|
- Update the release notes for the newly created tag (https://gitlab.com/gitlab-org/gitlab-qa/-/tags):
|
30
|
-
|
30
|
+
- **Release notes**: Copy the release notes from the merge request.
|
31
31
|
|
32
32
|
Note: The `gem-release` job uses:
|
33
33
|
|
@@ -43,7 +43,8 @@ Please note that `gitlab-qa` version in `gitlab-org/gitlab` project will be upda
|
|
43
43
|
The flowchart below outlines a high-level overview of the `gitlab-qa` release process, along with post-release maintenance items required
|
44
44
|
to update the `gitlab-qa` version for our pipelines.
|
45
45
|
|
46
|
-
|
46
|
+
### Key
|
47
|
+
|
47
48
|
- Rectangles represent manual actions
|
48
49
|
- Hexagons represent automated actions
|
49
50
|
|
data/docs/run_qa_against_gdk.md
CHANGED
@@ -5,6 +5,7 @@ make a few changes to your `gdk/gitlab/config/gitlab.yml` file.
|
|
5
5
|
|
6
6
|
1. Retrieve your current local IP with `ifconfig`, e.g. `192.168.0.12`.
|
7
7
|
2. In your gdk directory, create file `gdk.yml`. Within `gdk.yml` add:
|
8
|
+
|
8
9
|
```yaml
|
9
10
|
---
|
10
11
|
gdk:
|
@@ -16,6 +17,7 @@ make a few changes to your `gdk/gitlab/config/gitlab.yml` file.
|
|
16
17
|
webpack:
|
17
18
|
host: 192.168.0.12
|
18
19
|
```
|
20
|
+
|
19
21
|
3. Run `gdk reconfigure`
|
20
22
|
- This should update `localhost` in your `gitlab/config/gitlab.yml`.
|
21
23
|
and `gdk/Procfile` as well as `gdk/openssh/sshd_config` to the provided IP address.
|
@@ -27,7 +29,7 @@ make a few changes to your `gdk/gitlab/config/gitlab.yml` file.
|
|
27
29
|
- If reconfigure is successful, you should now be able to browse to `http://192.168.0.12:3000` instead of `http://localhost:3000`
|
28
30
|
7. Run the QA scenario as follows:
|
29
31
|
|
30
|
-
```
|
32
|
+
```shell
|
31
33
|
$ gitlab-qa Test::Instance::Any CE http://192.168.0.12:3000 -- qa/specs/features/browser_ui/1_manage/login/log_in_spec.rb
|
32
34
|
|
33
35
|
# Or if you want to run your local `gitlab-qa/exe/gitlab-qa`:
|
@@ -55,37 +57,37 @@ GitLab QA expects the default initial password to be used in tests; see all defa
|
|
55
57
|
If you have changed your root password, you must set the `GITLAB_INITIAL_ROOT_PASSWORD` environment
|
56
58
|
variable.
|
57
59
|
|
58
|
-
```
|
59
|
-
export GITLAB_INITIAL_ROOT_PASSWORD="<GDK root password>"
|
60
|
-
```
|
60
|
+
```shell
|
61
|
+
export GITLAB_INITIAL_ROOT_PASSWORD="<GDK root password>"
|
62
|
+
```
|
61
63
|
|
62
64
|
**Note**: If you encounter the following error, you can resolve it by unsetting the Docker host environment variable using the command `unset DOCKER_HOST`:
|
63
65
|
|
64
|
-
```
|
65
|
-
ERROR: error during connect: get "http://docker.local:2375/v1.24/images/json": dial tcp: lookup docker.local: no such host
|
66
|
-
```
|
66
|
+
```shell
|
67
|
+
ERROR: error during connect: get "http://docker.local:2375/v1.24/images/json": dial tcp: lookup docker.local: no such host
|
68
|
+
```
|
67
69
|
|
68
|
-
|
70
|
+
## Running EE tests
|
69
71
|
|
70
72
|
When running EE tests you'll need to have a license available. GitLab engineers can [request a license](https://about.gitlab.com/handbook/developer-onboarding/#working-on-gitlab-ee).
|
71
73
|
|
72
74
|
Once you have the license file you can export it as an environment variable and then `gitlab-qa` can use it. If you do so it will be installed automatically by the QA framework.
|
73
75
|
|
74
|
-
```
|
75
|
-
export EE_LICENSE=$(cat /path/to/gitlab_license)
|
76
|
-
```
|
76
|
+
```shell
|
77
|
+
export EE_LICENSE=$(cat /path/to/gitlab_license)
|
78
|
+
```
|
77
79
|
|
78
80
|
## Run Geo QA tests against your Geo GDK setup
|
79
81
|
|
80
82
|
Run from the `gdk-ee/gitlab/qa` directory with GDK primary and secondary running:
|
81
83
|
|
82
|
-
```
|
83
|
-
# Run in the background
|
84
|
-
$ bundle exec bin/qa QA::EE::Scenario::Test::Geo --primary-address http://localhost:3001 --secondary-address http://localhost:3002 --primary-name primary --secondary-name secondary --without-setup
|
84
|
+
```shell
|
85
|
+
# Run in the background
|
86
|
+
$ bundle exec bin/qa QA::EE::Scenario::Test::Geo --primary-address http://localhost:3001 --secondary-address http://localhost:3002 --primary-name primary --secondary-name secondary --without-setup
|
85
87
|
|
86
|
-
# Run in visible Chrome browser
|
87
|
-
$ WEBDRIVER_HEADLESS=0 bundle exec bin/qa QA::EE::Scenario::Test::Geo --primary-address http://localhost:3001 --secondary-address http://localhost:3002 --primary-name primary --secondary-name secondary --without-setup
|
88
|
-
```
|
88
|
+
# Run in visible Chrome browser
|
89
|
+
$ WEBDRIVER_HEADLESS=0 bundle exec bin/qa QA::EE::Scenario::Test::Geo --primary-address http://localhost:3001 --secondary-address http://localhost:3002 --primary-name primary --secondary-name secondary --without-setup
|
90
|
+
```
|
89
91
|
|
90
92
|
### QA Tool support on macOS
|
91
93
|
|
@@ -149,42 +151,44 @@ development machine that maps the VHOST to `localhost:port`, changing to the por
|
|
149
151
|
You first need to map the hostnames to the local ip:
|
150
152
|
|
151
153
|
Edit the `/etc/hosts` file
|
152
|
-
|
153
|
-
|
154
|
-
|
154
|
+
|
155
|
+
```plaintext
|
156
|
+
127.0.0.1 gitlab-primary.geo gitlab-secondary.geo
|
157
|
+
```
|
158
|
+
|
155
159
|
We are going to use [caddyserver](https://caddyserver.com/) in this example. You can install caddy with `brew install caddy`.
|
156
160
|
|
157
161
|
After booting-up your containers, list the assigned ports:
|
158
162
|
|
159
|
-
```
|
160
|
-
$ docker ps
|
163
|
+
```shell
|
164
|
+
$ docker ps
|
161
165
|
|
162
|
-
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
|
163
|
-
d28cc97870b4 gitlab/gitlab-ee:nightly "/assets/wrapper" 1 second ago Up Less than a second (health: starting) 22/tcp, 443/tcp, 0.0.0.0:32775->80/tcp gitlab-secondary
|
164
|
-
41f86bb951c5 gitlab/gitlab-ee:nightly "/assets/wrapper" 2 minutes ago Up 2 minutes (healthy) 22/tcp, 443/tcp, 0.0.0.0:32774->80/tcp gitlab-primary
|
165
|
-
```
|
166
|
+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
|
167
|
+
d28cc97870b4 gitlab/gitlab-ee:nightly "/assets/wrapper" 1 second ago Up Less than a second (health: starting) 22/tcp, 443/tcp, 0.0.0.0:32775->80/tcp gitlab-secondary
|
168
|
+
41f86bb951c5 gitlab/gitlab-ee:nightly "/assets/wrapper" 2 minutes ago Up 2 minutes (healthy) 22/tcp, 443/tcp, 0.0.0.0:32774->80/tcp gitlab-primary
|
169
|
+
```
|
166
170
|
|
167
171
|
Create a Caddyfile, mapping the VHOSTs with the `localhost:port` combination:
|
168
172
|
|
169
|
-
```
|
170
|
-
http://gitlab-primary.geo {
|
171
|
-
|
172
|
-
|
173
|
+
```plaintext
|
174
|
+
http://gitlab-primary.geo {
|
175
|
+
proxy / localhost:32774 {
|
176
|
+
transparent
|
177
|
+
}
|
173
178
|
}
|
174
|
-
}
|
175
179
|
|
176
|
-
http://gitlab-secondary.geo {
|
177
|
-
|
178
|
-
|
180
|
+
http://gitlab-secondary.geo {
|
181
|
+
proxy / localhost:32775 {
|
182
|
+
transparent
|
183
|
+
}
|
179
184
|
}
|
180
|
-
|
181
|
-
```
|
185
|
+
```
|
182
186
|
|
183
187
|
And run caddy:
|
184
188
|
|
185
|
-
```
|
186
|
-
caddy -conf Caddyfile
|
187
|
-
```
|
189
|
+
```shell
|
190
|
+
caddy -conf Caddyfile
|
191
|
+
```
|
188
192
|
|
189
193
|
You should be able to use your navigator and point it to `http://gitlab-primary.geo` or `http://gitlab-secondary.geo`.
|
190
194
|
|
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
The QA tests have the ability to be run against a local or remote grid.
|
4
4
|
|
5
|
-
I.e, if you have a Selenium server set up at http://localhost:4444 or if you have a SauceLabs / BrowserStack account.
|
5
|
+
I.e, if you have a Selenium server set up at <http://localhost:4444> or if you have a SauceLabs / BrowserStack account.
|
6
6
|
|
7
7
|
## Variables
|
8
8
|
|
@@ -70,10 +70,10 @@ To start the tunnel, copy the run command in **Sauce Labs > Tunnels** and run it
|
|
70
70
|
|
71
71
|
It is highly recommended to use `GITLAB_QA_ACCESS_TOKEN` to speed up tests and reduce flakiness.
|
72
72
|
|
73
|
-
|
74
73
|
### Run a test in a desktop browser
|
75
74
|
|
76
75
|
While tunnel is running, to test against a local instance in a desktop browser, run:
|
76
|
+
|
77
77
|
```shell
|
78
78
|
$ QA_BROWSER="safari" \
|
79
79
|
QA_REMOTE_GRID="ondemand.saucelabs.com:80" \
|
@@ -1,10 +1,12 @@
|
|
1
|
-
#
|
1
|
+
# Troubleshooting
|
2
2
|
|
3
|
-
|
3
|
+
## Mac OS with ARM processors & Docker Desktop
|
4
|
+
|
5
|
+
You might encounter the following errors when running tests locally, using Gitlab QA, on Mac OS with `ARM` processors. These errors usually stem from using Docker images that are based on Linux/AMD64 platforms.
|
4
6
|
|
5
7
|
- `MADV_DONTNEED` does not work (memset will be used instead)
|
6
8
|
|
7
|
-
```
|
9
|
+
```shell
|
8
10
|
<jemalloc>: MADV_DONTNEED does not work (memset will be used instead)
|
9
11
|
<jemalloc>: (This is the expected behaviour if you are running under QEMU)
|
10
12
|
bundler: failed to load command: bin/qa (bin/qa)
|
@@ -16,14 +18,14 @@ You might encounter the following errors when running tests locally, using Gitla
|
|
16
18
|
|
17
19
|
- `QA::Support::Repeater::WaitExceededError`
|
18
20
|
|
19
|
-
```
|
21
|
+
```shell
|
20
22
|
QA::Support::Repeater::WaitExceededError:
|
21
23
|
Page did not fully load. This could be due to an unending async request or loading icon.
|
22
24
|
```
|
23
25
|
|
24
26
|
- `Selenium::WebDriver::Error::UnknownError`
|
25
27
|
|
26
|
-
```
|
28
|
+
```shell
|
27
29
|
Selenium::WebDriver::Error::UnknownError:
|
28
30
|
unknown error: session deleted because of page crash
|
29
31
|
from tab crashed
|
@@ -33,6 +35,7 @@ You might encounter the following errors when running tests locally, using Gitla
|
|
33
35
|
To resolve these issues:
|
34
36
|
|
35
37
|
1. Do not use `/dev/shm` shared memory. Set `CHROME_DISABLE_DEV_SHM` environment variable to `true`.
|
38
|
+
|
36
39
|
```shell
|
37
40
|
$ export CHROME_DISABLE_DEV_SHM=true
|
38
41
|
# Disable Chrome shared memory
|
@@ -104,7 +104,8 @@ module Gitlab
|
|
104
104
|
'AIGW_LOGGING__LEVEL' => 'debug',
|
105
105
|
'AIGW_LOGGING__TO_FILE' => "..#{LOG_DIR}/modelgateway_debug.log",
|
106
106
|
'AIGW_SELF_SIGNED_JWT__SIGNING_KEY' => TEST_SIGNING_KEY,
|
107
|
-
'AIGW_SELF_SIGNED_JWT__VALIDATION_KEY' => TEST_VALIDATION_KEY
|
107
|
+
'AIGW_SELF_SIGNED_JWT__VALIDATION_KEY' => TEST_VALIDATION_KEY,
|
108
|
+
'CLOUD_CONNECTOR_SERVICE_NAME' => 'gitlab-ai-gateway'
|
108
109
|
}
|
109
110
|
end
|
110
111
|
end
|
data/lib/gitlab/qa/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: gitlab-qa
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 14.
|
4
|
+
version: 14.19.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- GitLab Quality
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2024-10-
|
11
|
+
date: 2024-10-24 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: climate_control
|
@@ -365,15 +365,13 @@ files:
|
|
365
365
|
- docs/developer/http_mocking.md
|
366
366
|
- docs/developer/ssl.md
|
367
367
|
- docs/developer/style_guide.md
|
368
|
-
- docs/environment_setup/users.md
|
369
368
|
- docs/how_it_works.md
|
370
369
|
- docs/omnibus_configurations/license_mode.md
|
371
370
|
- docs/release_process.md
|
372
371
|
- docs/run_qa_against_gdk.md
|
373
372
|
- docs/running_against_remote_grid.md
|
374
|
-
- docs/specifics_for_macos_m1_m2.md
|
375
373
|
- docs/trainings.md
|
376
|
-
- docs/
|
374
|
+
- docs/troubleshooting.md
|
377
375
|
- docs/what_tests_can_be_run.md
|
378
376
|
- exe/gitlab-qa
|
379
377
|
- fixtures/cvs/vulnerabilities_template.erb
|
@@ -1,9 +0,0 @@
|
|
1
|
-
# Environment Setup | Users
|
2
|
-
|
3
|
-
Most tests ([orchestrated and instance-level](../docs/what_tests_can_be_run.md#the-two-types-of-qa-tests)) [run as the default `root` user](https://gitlab.com/gitlab-org/gitlab/-/blob/3f247bf0d6d24b479b7f169f9c14068968d9672a/qa/qa/runtime/user.rb#L16). However, some require additional users. If the test framework has administrator access to the test environment it can create users itself. If administrator access is not available you will need to create users before being able to run certain tests.
|
4
|
-
|
5
|
-
## Disable email verification
|
6
|
-
|
7
|
-
If [email verification](https://docs.gitlab.com/ee/security/email_verification.html) is enabled on the test environment (via the `require_email_verification` feature flag), a user cannot log in under certain conditions (e.g., logging in the first time from a new location) unless they enter a verification code that is sent to their email address.
|
8
|
-
|
9
|
-
To disable email verification you can disable the `require_email_verification` feature flag, which will disable email verification for all users on the instance. Alternatively, you can skip verification for individual users by enabling the `skip_require_email_verification` feature flag for that user.
|
data/docs/waits.md
DELETED
@@ -1,24 +0,0 @@
|
|
1
|
-
Waits
|
2
|
-
---
|
3
|
-
|
4
|
-
All Capybara Node Finders utilize a waiting mechanism.
|
5
|
-
|
6
|
-
Per the [Capybara API](https://www.rubydoc.info/github/jnicklas/capybara/Capybara/Node/Finders:find) -
|
7
|
-
|
8
|
-
> If the driver is capable of executing JavaScript, `find` will wait for a set amount of time and continuously retry finding the element until either the element is found or the time expires. The length of time find will wait is controlled through `Capybara.default_max_wait_time` and defaults to `2` seconds. `find` takes the same options as all.
|
9
|
-
|
10
|
-
Ideally the [GitLab QA Framework](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa) should implement its own explicit waiting to avoid hard sleeps but currently that is [not the case](https://gitlab.com/gitlab-org/gitlab-qa/issues/280).
|
11
|
-
|
12
|
-
## Hard Sleeps
|
13
|
-
|
14
|
-
**[qa/qa/page/base.rb](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/qa/qa/page/base.rb#L16)**
|
15
|
-
|
16
|
-
```
|
17
|
-
def wait(max: 60, time: 0.1, reload: true)
|
18
|
-
...
|
19
|
-
end
|
20
|
-
```
|
21
|
-
|
22
|
-
- `max` : Specifies the max amount of *seconds* to wait until the block given is satisfied
|
23
|
-
- `time` : The interval/poll time to sleep *in seconds*. If this time reaches `max`, the wait returns `false`
|
24
|
-
- `reload` : If the wait is not satiated, the test will sleep then reload the page if `:reload` is set to `true`
|