RubyGems - gitlab-qa - Versions diffs - 10.4.0 → 10.5.0 - Mend

gitlab-qa 10.4.0 → 10.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (14) hide show

checksums.yaml +4 -4
data/.gitlab/merge_request_templates/Release.md +1 -1
data/Gemfile.lock +1 -1
data/README.md +1 -2
data/docs/running_against_remote_grid.md +39 -2
data/lib/gitlab/qa/component/gitaly_cluster.rb +0 -1
data/lib/gitlab/qa/component/selenoid.rb +8 -3
data/lib/gitlab/qa/runtime/env.rb +21 -1
data/lib/gitlab/qa/scenario/test/instance/airgapped.rb +0 -2
data/lib/gitlab/qa/scenario/test/integration/gitaly_cluster.rb +0 -2
data/lib/gitlab/qa/scenario/test/integration/mtls.rb +20 -6
data/lib/gitlab/qa/scenario/test/integration/praefect.rb +16 -10
data/lib/gitlab/qa/version.rb +1 -1
metadata +2 -2

checksums.yaml CHANGED Viewed

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fe25be14520a4bf78ee88e0d5bd7e36534d79b32c5734b92b8337736c659b6b2
-  data.tar.gz: c0064bfce6ed145f0e7464ac72f8ca8ddc2c2ae467a956f361645e8443f7546a
+  metadata.gz: 0e892bc0c445fc7981ca1e1d511411818d92b577e041d0cf465813793a05462d
+  data.tar.gz: aeb7f3acdd748d80a47946fb26614f9fe5a9d1462935f2b19dae9269bd55afd1
 SHA512:
-  metadata.gz: f3df8e960239e908aa0f86b7f6dcac023ce6996aadd3a4e49108cbf6f58876de94946f3d475b0828578ccb04309ca241ea8f8eeb3fa44b760624f4cb33c29b2f
-  data.tar.gz: 0af92a24612916090ae9a71aebef586b3261a6f99c5cd93f287199881f1f959e897bc68811f8fdb86fc607539bb4998839b6420233d300b1ceca6a3005fa7573
+  metadata.gz: 582219201b2786ebc62d042a7a280c152d4337b941c840d6e56bef371ffab400421c7fe793ac29cddc063441300e0f7a18414eba15a3951e09c21c208530bbb6
+  data.tar.gz: fe82a271991a20a99bbf212241e98ba8821a960d23b72f789d386dd85bd895255c50eaa9a8356ece428c3d32f467b19e8b396c9a719c58039548ab4db33d8e4a

data/.gitlab/merge_request_templates/Release.md CHANGED Viewed

@@ -34,7 +34,7 @@ with the latest commit from https://gitlab.com/gitlab-org/gitlab-qa/commits/mast
 - Checklist after gitlab-qa version has been released by the post merge pipeline:
   - [ ] In the pipeline-common project, [update the GITLAB_QA_VERSION](https://gitlab.com/gitlab-org/quality/pipeline-common/-/blob/master/ci/base.gitlab-ci.yml) and [create a release](https://gitlab.com/gitlab-org/quality/pipeline-common#release-process).
-  - [ ] In the GitLab project, update the ref for pipeline-common for [package-and-test](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/package-and-test/main.gitlab-ci.yml) and [review-apps](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/review-apps/qa.gitlab-ci.yml).
+  - [ ] In the GitLab project, update the ref for pipeline-common in [qa-common](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/qa-common/main.gitlab-ci.yml) and [review-apps](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/review-apps/qa.gitlab-ci.yml).
   - [ ] Unless already done by [renovate-gitlab-bot](https://gitlab.com/dashboard/merge_requests?scope=all&state=opened&author_username=gitlab-dependency-update-bot&label_name[]=Quality), or if you need it sooner, in the GitLab project, [update the gitlab-qa gem version](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/Gemfile) and [`Gemfile.lock`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/Gemfile.lock) (for an example, see: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/117054).
 /label ~Quality ~"type::maintenance" ~"maintenance::dependency"

data/Gemfile.lock CHANGED Viewed

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    gitlab-qa (10.4.0)
+    gitlab-qa (10.5.0)
       activesupport (~> 6.1)
       gitlab (~> 4.18.0)
       http (~> 5.0)

data/README.md CHANGED Viewed

@@ -96,8 +96,7 @@ of tests.
 ## How do we use it
 Currently, we execute the test suite against GitLab Docker images created by
-Omnibus nightly via a pipeline in the [nightly](https://gitlab.com/gitlab-org/quality/nightly)
-project.
+Omnibus nightly via a [`gitlab-org/gitlab` nightly schedule pipeline](https://gitlab.com/gitlab-org/gitlab/-/pipeline_schedules).
 We also execute the test suite nightly against our [staging environment](https://staging.gitlab.com)
 via a pipeline in the [staging project](https://gitlab.com/gitlab-org/quality/staging).

data/docs/running_against_remote_grid.md CHANGED Viewed

@@ -8,19 +8,56 @@ I.e, if you have a Selenium server set up at http://localhost:4444 or if you hav
 | Variable                  | Description                                                    | Default  | Example(s)                     |
 |---------------------------|----------------------------------------------------------------|----------|--------------------------------|
-| QA_BROWSER                | Browser to run against                                         | "chrome" | "chrome" "firefox" "safari"    |
-| QA_BROWSER_VERSION        | Version of browser to run against                              | "latest" | "latest" "111.0" "mobile-111.0"|
+| QA_BROWSER                | Browser to run against                                         | "chrome" | "chrome" "edge"                |
+| QA_BROWSER_VERSION        | Version of browser to run against                              | "111.0"  | "latest" "111.0" "mobile-111.0"|
 | QA_REMOTE_GRID_PROTOCOL   | Protocol to use                                                | "http"   | "http" "https"                 |
 | QA_REMOTE_GRID            | Remote grid to run tests against                               |          | "localhost:3000" "provider:80" "selenoid:4444" |
 | QA_LAYOUT                 | Used with Selenoid. Tells test nav to expect collapsed menus. "phone" expects collapsed top and left nav bars, "tablet" expects collapsed left nav bar only. |          | "phone", "tablet"              |
 | SELENOID_DIRECTORY        | Used with Selenoid. Directory to save videos to                | "<host_artifacts_dir>/selenoid" |        |
 | USE_SELENOID              | Used with Selenoid. Sets up selenoid containers.               | false    | false, true                    |
 | QA_RECORD_VIDEO           | Used with Selenoid. Triggers video recording.                  | false    | false, true                    |
+| QA_BROWSER_IMAGE          | Used with Selenoid. Sets the browser image to use for video recording. | "selenoid/chrome" | "selenoid/chrome", "registry.gitlab.com/gitlab-org/gitlab-qa/selenoid-chrome-gitlab" |
+| QA_VIDEO_RECORDER_IMAGE   | Used with Selenoid. Sets the video recorder image to use for video recording. | "presidenten/selenoid-manual-video-recorder" | "presidenten/selenoid-manual-video-recorder" |
+| QA_VIDEO_RECORDER_VERSION | Used with Selenoid. Sets the video recorder image version to use for video recording. | "latest" | "latest"                     |
+| QA_RECORD_VIDEO           | Used with Selenoid. Triggers video recording.                  | false    | false, true                    |
 | QA_REMOTE_GRID_USERNAME   | Used with Sauce Labs. Username to specify in the remote grid. "USERNAME@provider:80" |          | "gitlab-sl"                    |
 | QA_REMOTE_GRID_ACCESS_KEY | Used with Sauce Labs. Key/Token paired with `QA_REMOTE_GRID_USERNAME` |          |                                |
 | QA_REMOTE_TUNNEL_ID       | Used with Sauce Labs. Name of the remote tunnel to use         | "gitlab-sl_tunnel_id" |                                |
 | QA_REMOTE_MOBILE_DEVICE_NAME | Used with Sauce Labs. Name of mobile device to test against. `QA_BROWSER` must be set to `safari` for iOS devices and `chrome` for Android devices. |          | "iPhone 12 Simulator"          |
+## Testing with Selenoid
+Running directly against an environment like staging is not recommended because test videos can expose credentials. Therefore, it is best practice to not run against live environments.
+Available browsers are defined in [browsers.json](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/fixtures/selenoid/browsers.json)
+Failure videos are available as job artifacts in the selenoid directory.
+### Add failure video recording to a pipeline for a non-live environment
+Set these environment variables:
+USE_SELENOID=true
+QA_RECORD_VIDEO=true
+QA_REMOTE_GRID="selenoid:4444"
+#### To test with an Edge browser instead of Chrome
+Include these environment variables:
+QA_BROWSER="edge"
+QA_BROWSER_IMAGE="selenoid/edge"
+#### To test with a Chrome mobile device browser
+Include these environment variables:
+QA_LAYOUT="phone"
+QA_BROWSER_IMAGE="registry.gitlab.com/gitlab-org/gitlab-qa/selenoid-chrome-gitlab"
+QA_BROWSER_VERSION=mobile-111.0
+For now we only have version 111.0 available for chrome mobile browser testing.
 ## Testing with Sauce Labs
 Running directly against an environment like staging is not recommended because test logs expose credentials. Therefore, it is best practice and the default to use a tunnel.

data/lib/gitlab/qa/component/gitaly_cluster.rb CHANGED Viewed

@@ -135,7 +135,6 @@ module Gitlab
             postgresql['enable'] = false;
             redis['enable'] = false;
             nginx['enable'] = false;
-            grafana['enable'] = false;
             puma['enable'] = false;
             sidekiq['enable'] = false;
             gitlab_workhorse['enable'] = false;

data/lib/gitlab/qa/component/selenoid.rb CHANGED Viewed

@@ -27,7 +27,12 @@ module Gitlab
         def start
           pull_images
-          docker.run(image: image, tag: tag, args: ['-container-network', network]) do |command|
+          docker.run(image: image, tag: tag, args:
+                      ['-video-recorder-image',
+                       QA::Runtime::Env.video_recorder_image,
+                       '-container-network',
+                       network]
+                    ) do |command|
             set_command_args(command)
             set_volumes(command)
           end
@@ -55,8 +60,8 @@ module Gitlab
         end
         def pull_images
-          docker.pull(image: "selenoid/chrome", tag: Runtime::Env.browser_version) if Runtime::Env.browser == 'chrome' && !Runtime::Env.mobile_layout?
-          docker.pull(image: "selenoid/video-recorder", tag: "latest-release") if Runtime::Env.record_video
+          docker.pull(image: QA::Runtime::Env.browser_image, tag: Runtime::Env.browser_version)
+          docker.pull(image: QA::Runtime::Env.video_recorder_image, tag: QA::Runtime::Env.video_recorder_version)
         end
         # Set custom run command arguments

data/lib/gitlab/qa/runtime/env.rb CHANGED Viewed

@@ -25,6 +25,9 @@ module Gitlab
           'QA_BROWSER_VERSION' => :browser_version,
           'QA_RECORD_VIDEO' => :record_video,
           'QA_LAYOUT' => :layout,
+          'QA_VIDEO_RECORDER_IMAGE' => :video_recorder_image,
+          'QA_VIDEO_RECORDER_VERSION' => :video_recorder_version,
+          'QA_BROWSER_IMAGE' => :browser_image,
           'QA_ADDITIONAL_REPOSITORY_STORAGE' => :qa_additional_repository_storage,
           'QA_PRAEFECT_REPOSITORY_STORAGE' => :qa_praefect_repository_storage,
           'QA_GITALY_NON_CLUSTER_STORAGE' => :qa_gitaly_non_cluster_storage,
@@ -154,7 +157,8 @@ module Gitlab
           'RELEASE_REGISTRY_USERNAME' => :release_registry_username,
           'RELEASE_REGISTRY_PASSWORD' => :release_registry_password,
           'SELENOID_DIRECTORY' => :selenoid_directory,
-          'USE_SELENOID' => :use_selenoid
+          'USE_SELENOID' => :use_selenoid,
+          'SCHEDULE_TYPE' => :schedule_type
         }.freeze
         ENV_VARIABLES.each do |env_name, method_name|
@@ -367,6 +371,22 @@ module Gitlab
           env_var_value_if_defined('QA_LAYOUT')&.match?(/tablet|phone/i)
         end
+        def video_recorder_image
+          env_var_value_if_defined('QA_VIDEO_RECORDER_IMAGE') || 'presidenten/selenoid-manual-video-recorder'
+        end
+        def video_recorder_version
+          env_var_value_if_defined('QA_VIDEO_RECORDER_VERSION') || 'latest'
+        end
+        def browser_image
+          env_var_value_if_defined('QA_BROWSER_IMAGE') || 'selenoid/chrome'
+        end
+        def browser_version
+          env_var_value_if_defined('QA_BROWSER_VERSION') || '111.0'
+        end
         def qa_run_type
           return env_var_value_if_defined('QA_RUN_TYPE') if env_var_value_valid?('QA_RUN_TYPE')

data/lib/gitlab/qa/scenario/test/instance/airgapped.rb CHANGED Viewed

@@ -130,8 +130,6 @@ module Gitlab
                     ]
                   }
                 ];
-                grafana['disable_login_form'] = false;
-                grafana['admin_password'] = 'GRAFANA_ADMIN_PASSWORD';
               OMNIBUS
             end
           end

data/lib/gitlab/qa/scenario/test/integration/gitaly_cluster.rb CHANGED Viewed

@@ -79,8 +79,6 @@ module Gitlab
                     ]
                   }
                 ];
-                grafana['disable_login_form'] = false;
-                grafana['admin_password'] = 'GRAFANA_ADMIN_PASSWORD';
               OMNIBUS
             end
           end

data/lib/gitlab/qa/scenario/test/integration/mtls.rb CHANGED Viewed

@@ -87,17 +87,32 @@ module Gitlab
             def gitaly_omnibus
               <<~OMNIBUS
-                gitaly['tls_listen_addr'] = '0.0.0.0:9999';
-                gitaly['certificate_path'] = '/etc/gitlab/ssl/gitaly.test.crt';
-                gitaly['key_path'] = '/etc/gitlab/ssl/gitaly.test.key';
+                gitaly['configuration'] = {
+                  auth: {
+                    token: 'abc123secret',
+                  },
+                  tls_listen_addr: '0.0.0.0:9999',
+                  tls: {
+                    certificate_path: '/etc/gitlab/ssl/gitaly.test.crt',
+                    key_path: '/etc/gitlab/ssl/gitaly.test.key',
+                  },
+                  storage: [
+                    {
+                      name: 'default',
+                      path: '/var/opt/gitlab/git-data/repositories',
+                    },
+                    {
+                      name: 'storage1',
+                      path: '/mnt/gitlab/git-data/repositories',
+                    },
+                  ],
+                };
                 postgresql['enable'] = false;
                 redis['enable'] = false;
                 nginx['enable'] = false;
                 puma['enable'] = false;
                 sidekiq['enable'] = false;
                 gitlab_workhorse['enable'] = false;
-                grafana['enable'] = false;
                 gitlab_exporter['enable'] = false;
                 alertmanager['enable'] = false;
                 prometheus['enable'] = false;
@@ -105,7 +120,6 @@ module Gitlab
                 gitlab_rails['rake_cache_clear'] = false;
                 gitlab_rails['auto_migrate'] = false;
-                gitaly['auth_token'] = 'abc123secret';
                 gitlab_shell['secret_token'] = 'shellsecret';
                 gitlab_rails['internal_api_url'] = 'https://#{@gitlab_name}.#{@network}';

data/lib/gitlab/qa/scenario/test/integration/praefect.rb CHANGED Viewed

@@ -30,14 +30,22 @@ module Gitlab
                   }
                 });
                 gitaly['enable'] = true;
-                gitaly['listen_addr'] = '0.0.0.0:8075';
-                gitaly['auth_token'] = 'secret-token';
-                gitaly['storage'] = [
-                  {
-                    'name' => 'gitaly',
-                    'path' => '/var/opt/gitlab/git-data/repositories'
-                  }
-                ];
+                gitaly['configuration'] = {
+                  auth: {
+                    token: 'secret-token',
+                  },
+                  listen_addr: '0.0.0.0:8075',
+                  tls: {
+                    certificate_path: '/etc/gitlab/ssl/gitaly.test.crt',
+                    key_path: '/etc/gitlab/ssl/gitaly.test.key',
+                  },
+                  storage: [
+                    {
+                      name: 'gitaly',
+                      path: '/var/opt/gitlab/git-data/repositories',
+                    },
+                  ],
+                };
                 gitlab_rails['gitaly_token'] = 'secret-token';
                 gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN';
                 prometheus['scrape_configs'] = [
@@ -60,8 +68,6 @@ module Gitlab
                     ]
                   }
                 ];
-                grafana['disable_login_form'] = false;
-                grafana['admin_password'] = 'GRAFANA_ADMIN_PASSWORD';
               OMNIBUS
             end
           end

data/lib/gitlab/qa/version.rb CHANGED Viewed

@@ -2,6 +2,6 @@
 module Gitlab
   module QA
-    VERSION = '10.4.0'
+    VERSION = '10.5.0'
   end
 end

metadata CHANGED Viewed

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: gitlab-qa
 version: !ruby/object:Gem::Version
-  version: 10.4.0
+  version: 10.5.0
 platform: ruby
 authors:
 - GitLab Quality
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2023-05-16 00:00:00.000000000 Z
+date: 2023-05-25 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: climate_control