gitlab-qa 5.4.1 → 5.4.2
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +12 -11
- data/docs/running_specific_orchestrated_tests.md +11 -0
- data/lib/gitlab/qa/report/results_in_issues.rb +1 -1
- data/lib/gitlab/qa/version.rb +1 -1
- metadata +2 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 6ce58eec53ccc54effca3fc1b94070e6ec6f930c13e11aef8dafb600c59feb59
|
4
|
+
data.tar.gz: 97437cf5d0bed65931c57af0dc459d6097f37f3787ba8498f9784d4c45158d2a
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5ecabb59dd4285a6a0923a680390e6ba7f87eeb85b58827279ebb06870ea32695218ded6a8b78bdaec1d6c4a522d5ed60d888859b6c0dd45c56aa9611a782350
|
7
|
+
data.tar.gz: 46970e6a919d479d104dc664f249b7aa03543e4e95499e7e7912381e02cb383085ea837b61b1e3dfb74eeb6cf50c12ee9ee46e4a5b25a7e4448427fe1bd02622
|
data/README.md
CHANGED
@@ -12,10 +12,10 @@
|
|
12
12
|
GitLab QA framework and Capybara to setup and perform individual end-to-end
|
13
13
|
tests against a live GitLab instance.
|
14
14
|
Located at [`gitlab-org/gitlab-ce@qa/qa/specs/features/`][instance-level-scenarios].
|
15
|
-
- **GitLab QA orchestrator** (this project): An orchestration tool that enables
|
15
|
+
- **GitLab QA orchestrator** (this project): An orchestration tool that enables
|
16
16
|
running various QA test suites in a simple manner.
|
17
17
|
- **GitLab QA orchestrated scenarios**: Scenarios where containers are started,
|
18
|
-
configured, and execute instance-level scenarios against a running GitLab
|
18
|
+
configured, and execute instance-level scenarios against a running GitLab
|
19
19
|
instance.
|
20
20
|
Located at [`gitlab-org/gitlab-qa@lib/gitlab/qa/scenario/test/`][orchestrated-scenarios].
|
21
21
|
|
@@ -39,7 +39,7 @@ merge request, even before merging code into the `master` branch.
|
|
39
39
|
### We can run tests against any instance of GitLab
|
40
40
|
|
41
41
|
GitLab QA is a click-driven, black-box testing tool. We also use it to run
|
42
|
-
tests against the staging environment, and we strive to make it useful for our
|
42
|
+
tests against the staging environment, and we strive to make it useful for our
|
43
43
|
users as well.
|
44
44
|
|
45
45
|
### GitLab QA tests running in the CI/CD environment
|
@@ -60,8 +60,8 @@ To make it easier to reproduce test failures locally we have published the
|
|
60
60
|
`gitlab-qa` gem [on rubygems.org](https://rubygems.org/gems/gitlab-qa) and we
|
61
61
|
are using exactly the same approach to run tests in the CI/CD environment.
|
62
62
|
|
63
|
-
It means that using the `gitlab-qa` CLI tool, which orchestrates the test
|
64
|
-
environment and runs the GitLab QA test suite, is a reproducible way of running
|
63
|
+
It means that using the `gitlab-qa` CLI tool, which orchestrates the test
|
64
|
+
environment and runs the GitLab QA test suite, is a reproducible way of running
|
65
65
|
tests locally and in the CI/CD environment.
|
66
66
|
|
67
67
|
It also means that we cannot have custom code in `.gitlab-ci.yml` to, for
|
@@ -70,16 +70,16 @@ example, start new containers / services.
|
|
70
70
|
### Test the installation / deployment process too
|
71
71
|
|
72
72
|
We distribute GitLab in a package (like a Debian package or a Docker image) so
|
73
|
-
we want to test the installation process to ensure that our package is not
|
73
|
+
we want to test the installation process to ensure that our package is not
|
74
74
|
broken.
|
75
75
|
|
76
76
|
But we are also working on making GitLab be a cloud native product. This means
|
77
|
-
that, for example, using Helm becomes yet another installation / deployment
|
77
|
+
that, for example, using Helm becomes yet another installation / deployment
|
78
78
|
process that we want to test with GitLab QA.
|
79
79
|
|
80
80
|
Considering our goal of being able to test all changes in merge requests, it is
|
81
81
|
especially important to be able to test our Kubernetes deployments, as that is
|
82
|
-
essential to scaling our test environments to efficiently handle a large number
|
82
|
+
essential to scaling our test environments to efficiently handle a large number
|
83
83
|
of tests.
|
84
84
|
|
85
85
|
## Documentation
|
@@ -88,14 +88,15 @@ of tests.
|
|
88
88
|
- [How it works](docs/how_it_works.md)
|
89
89
|
- [Release process](docs/release_process.md)
|
90
90
|
- [Run QA tests against your GDK setup](docs/run_qa_against_gdk.md)
|
91
|
+
- [Running specific orchestrated tests](docs/running_specific_orchestrated_tests.md)
|
91
92
|
- [Trainings](docs/trainings.md)
|
92
93
|
- [Waits](docs/waits.md)
|
93
94
|
- [What tests can be run?](docs/what_tests_can_be_run.md)
|
94
95
|
|
95
96
|
## How do we use it
|
96
97
|
|
97
|
-
Currently, we execute the test suite against GitLab Docker images created by
|
98
|
-
Omnibus nightly via a pipeline in the [nightly](https://gitlab.com/gitlab-org/quality/nightly)
|
98
|
+
Currently, we execute the test suite against GitLab Docker images created by
|
99
|
+
Omnibus nightly via a pipeline in the [nightly](https://gitlab.com/gitlab-org/quality/nightly)
|
99
100
|
project.
|
100
101
|
|
101
102
|
We also execute the test suite nightly against our [staging environment](https://staging.gitlab.com)
|
@@ -125,7 +126,7 @@ Note: The GitLab QA tool requires that [Docker](https://docs.docker.com/install/
|
|
125
126
|
Scenarios (test cases) and scripts to run them are located in the
|
126
127
|
[CE](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa) and
|
127
128
|
[EE](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/qa)
|
128
|
-
repositories under the `qa/` directory, so please also check the documentation
|
129
|
+
repositories under the `qa/` directory, so please also check the documentation
|
129
130
|
there.
|
130
131
|
|
131
132
|
## Contributing
|
@@ -0,0 +1,11 @@
|
|
1
|
+
# Running specific orchestrated tests
|
2
|
+
|
3
|
+
## Maven artifact spec
|
4
|
+
|
5
|
+
The [maven repository spec](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/specs/features/ee/browser_ui/5_package/maven_repository_spec.rb) creates a Maven artifact and links it to a GitLab project. The artifact is created within a [Maven docker image](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/service/docker_run/maven.rb#L8).
|
6
|
+
|
7
|
+
Using `gitlab-qa` to run these tests reduces the liklihood of network errors between the maven container and GitLab instance.
|
8
|
+
|
9
|
+
To run this with `gitlab-qa` you can use the `Test::Instance::Image` that is needed for your test. For example:
|
10
|
+
|
11
|
+
`gitlab-qa Test::Instance::Image registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:2cb9a85d2beadd51b926eaddb05005403bee0013 -- --tag orchestrated qa/specs/features/ee/browser_ui/5_package/maven_repository_spec.rb` - runs the test against a specific Omnibus GitLab image that was built through `package-and-qa`.
|
@@ -234,7 +234,7 @@ module Gitlab
|
|
234
234
|
# because the other pipelines will be monitored by the author of the MR that triggered them.
|
235
235
|
# So we assume that we're reporting a master pipeline if the project name is 'gitlab-qa'.
|
236
236
|
|
237
|
-
Runtime::Env.ci_project_name
|
237
|
+
Runtime::Env.ci_project_name.to_s.start_with?('gitlab-qa') ? 'master' : Runtime::Env.ci_project_name
|
238
238
|
end
|
239
239
|
end
|
240
240
|
end
|
data/lib/gitlab/qa/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: gitlab-qa
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 5.4.
|
4
|
+
version: 5.4.2
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Grzegorz Bizon
|
@@ -225,6 +225,7 @@ files:
|
|
225
225
|
- docs/release_process.md
|
226
226
|
- docs/run_qa_against_gdk.md
|
227
227
|
- docs/running_against_remote_grid.md
|
228
|
+
- docs/running_specific_orchestrated_tests.md
|
228
229
|
- docs/trainings.md
|
229
230
|
- docs/waits.md
|
230
231
|
- docs/what_tests_can_be_run.md
|