gitlab-qa 13.0.0 → 13.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/Gemfile.lock +1 -1
- data/docs/what_tests_can_be_run.md +24 -1
- data/lib/gitlab/qa/component/ai_gateway.rb +26 -0
- data/lib/gitlab/qa/component/gitlab.rb +7 -0
- data/lib/gitlab/qa/runtime/env.rb +8 -0
- data/lib/gitlab/qa/scenario/test/integration/ai_gateway.rb +90 -0
- data/lib/gitlab/qa/version.rb +1 -1
- data/support/setup/code_suggestions_setup.rb +59 -0
- metadata +5 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 16a2d1d19d2421861d4698ef692926f2fa15cc9f4081e39d1ca7e4da46a8ba7c
|
4
|
+
data.tar.gz: 0c65e2931b680e7737df751f702943e40851ac207afcb3be7a01765ec2b47291
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 81c553e7373d1fc790429dd8e4e9c1538e7c2a6afc382b7b79170c0edb6b8b83c4f533a60746defc313c846ab92fb4505f84899f3eede1f21bcdcdc586e7b90c
|
7
|
+
data.tar.gz: 492ad5e00e24b345ab1ac724dcf059c9520d992df141a03b413b674262720138a724ed44d633b9d9a926f40471aa44744994b8dcf38eee8753d03728ad3c715e
|
data/Gemfile.lock
CHANGED
@@ -58,7 +58,8 @@ All environment variables used by GitLab QA should be defined in [`lib/gitlab/qa
|
|
58
58
|
| `GITLAB_QA_ACCESS_TOKEN`|- | A valid personal access token with the `api` scope. This allows tests to use the API without having to create a new personal access token first. It is also used to check what version [deployed environments](https://gitlab.com/gitlab-org/quality/pipeline-common/-/blob/master/ci/dot-com/README.md#projects) are currently running. Existing tokens for each environment can be found in the shared 1Password vault. |No|
|
59
59
|
| `GITLAB_QA_ADMIN_ACCESS_TOKEN` |- | A valid personal access token with the `api` scope from a user with admin access. Used for API access as an admin during tests. | No|
|
60
60
|
| `GITLAB_QA_CONTAINER_REGISTRY_ACCESS_TOKEN` | - | A valid personal access token with the `read_registry` scope. Used to [access the container registry on `registry.gitlab.com` when tests run in a CI job that _is not_ triggered via another pipeline](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/364addb83e7b136ff0f9d8719ca9553d290aa9ab/lib/gitlab/qa/release.rb#L152). For example, if you manually run a [new Staging pipeline](https://ops.gitlab.net/gitlab-org/quality/staging/-/pipelines/new), this token will be used. | No |
|
61
|
-
| `EE_LICENSE` |- | Enterprise Edition license. | No|
|
61
|
+
| `EE_LICENSE` |- | Enterprise Edition license. For Staging license to be applied successfully requires `CUSTOMER_PORTAL_URL` set. | No|
|
62
|
+
| `GITLAB_LICENSE_MODE` |- | If set to `test` gitlab-qa will then set customers portal address to Staging address. | No|
|
62
63
|
| `QA_EE_ACTIVATION_CODE` |- | Cloud activation code to enable Enterprise Edition features. | No|
|
63
64
|
| `QA_ARTIFACTS_DIR` |`/tmp/gitlab-qa`| Path to a directory where artifacts (logs and screenshots) for failing tests will be saved. | No|
|
64
65
|
| `DOCKER_HOST` |`http://localhost`| Docker host to run tests against. | No|
|
@@ -1173,6 +1174,28 @@ need to be updated, it can be useful to proxy all requests to real `GitHub` inst
|
|
1173
1174
|
|
1174
1175
|
Testing import [by direct transfer](https://docs.gitlab.com/ee/user/group/import/#migrate-groups-by-direct-transfer-recommended) is done by spinning up 2 omnibus installations - `import-target` and `import-source`. In order for tests to work, application setting `allow_local_requests_from_web_hooks_and_services` must be enabled in target instance. This is automatically done by test process if environment variable `QA_ALLOW_LOCAL_REQUESTS` is set to `true`.
|
1175
1176
|
|
1177
|
+
### `Test::Integration::AiGateway EE|<full image address>`
|
1178
|
+
|
1179
|
+
This scenario spins up a GitLab Omnibus instance integrated with a test AI Gateway using [fake models](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#faking-out-ai-models). It will run specs tagged with the `:ai_gateway` tag in order to help verify that cloud licensing and authentication for AI features work as expected for self-managed instances.
|
1180
|
+
|
1181
|
+
**Required environment variables:**
|
1182
|
+
|
1183
|
+
- `QA_EE_ACTIVATION_CODE`: An activation code for a Staging-generated Premium or Ultimate cloud license with a purchased AI add-on. This can be found in the GitLab QA 1Password vault.
|
1184
|
+
- `GITLAB_LICENSE_MODE`: Set to `test` in order to configure GitLab Omnibus with the correct license mode and `CUSTOMER_PORTAL_URL`.
|
1185
|
+
|
1186
|
+
**Optional environment variables:**
|
1187
|
+
|
1188
|
+
- `QA_USE_SYNC_SERVICE_TOKEN_WORKER`: When true, will enable the `use_sync_service_token_worker` feature flag during code suggestions setup to use the `SyncServiceTokenWorker` to fetch a code suggestions JWT during activation instead of the `SyncSeatLinkWorker`. See rollout issue https://gitlab.com/gitlab-org/gitlab/-/issues/431608 for details.
|
1189
|
+
|
1190
|
+
Example:
|
1191
|
+
|
1192
|
+
```shell
|
1193
|
+
$ export QA_EE_ACTIVATION_CODE="<value from 1Password>"
|
1194
|
+
$ export GITLAB_LICENSE_MODE="test"
|
1195
|
+
|
1196
|
+
$ gitlab-qa Test::Integration::AiGateway EE
|
1197
|
+
```
|
1198
|
+
|
1176
1199
|
----
|
1177
1200
|
|
1178
1201
|
[Back to README.md](../README.md)
|
@@ -0,0 +1,26 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Gitlab
|
4
|
+
module QA
|
5
|
+
module Component
|
6
|
+
class AiGateway < Base
|
7
|
+
DOCKER_IMAGE = 'registry.gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/model-gateway'
|
8
|
+
DOCKER_IMAGE_TAG = 'latest'
|
9
|
+
|
10
|
+
def name
|
11
|
+
@name ||= 'ai-gateway'
|
12
|
+
end
|
13
|
+
|
14
|
+
def configure_environment(gitlab_hostname:)
|
15
|
+
@environment = {
|
16
|
+
'GITLAB_URL' => "http://#{gitlab_hostname}",
|
17
|
+
'GITLAB_API_URL' => "http://#{gitlab_hostname}/api/v4",
|
18
|
+
'CUSTOMER_PORTAL_BASE_URL' => Runtime::Env.customer_portal_url,
|
19
|
+
'USE_FAKE_MODELS' => true,
|
20
|
+
'USE_LOCAL_CACHE' => true
|
21
|
+
}
|
22
|
+
end
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
26
|
+
end
|
@@ -83,6 +83,13 @@ module Gitlab
|
|
83
83
|
@environment['ELASTIC_URL'] = url
|
84
84
|
end
|
85
85
|
|
86
|
+
def set_ee_activation_code
|
87
|
+
raise 'QA_EE_ACTIVATION_CODE is required!' if Runtime::Env.ee_activation_code.to_s.strip.empty?
|
88
|
+
|
89
|
+
@omnibus_gitlab_rails_env['QA_EE_ACTIVATION_CODE'] = Runtime::Env.ee_activation_code
|
90
|
+
secrets << Runtime::Env.ee_activation_code
|
91
|
+
end
|
92
|
+
|
86
93
|
def release=(release)
|
87
94
|
@release = QA::Release.new(release)
|
88
95
|
end
|
@@ -428,6 +428,14 @@ module Gitlab
|
|
428
428
|
env_var_value_if_defined('CUSTOMER_PORTAL_URL') || 'https://customers.staging.gitlab.com'
|
429
429
|
end
|
430
430
|
|
431
|
+
def ee_activation_code
|
432
|
+
env_var_value_if_defined('QA_EE_ACTIVATION_CODE')
|
433
|
+
end
|
434
|
+
|
435
|
+
def use_sync_service_token_worker?
|
436
|
+
enabled?(env_var_value_if_defined('QA_USE_SYNC_SERVICE_TOKEN_WORKER'), default: false)
|
437
|
+
end
|
438
|
+
|
431
439
|
def geo_staging_url
|
432
440
|
env_var_value_if_defined('GEO_STAGING_URL') || 'https://geo.staging.gitlab.com'
|
433
441
|
end
|
@@ -0,0 +1,90 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Gitlab
|
4
|
+
module QA
|
5
|
+
module Scenario
|
6
|
+
module Test
|
7
|
+
module Integration
|
8
|
+
class AiGateway < Scenario::Template
|
9
|
+
SETUP_SRC_PATH = File.expand_path('../../../../../../support/setup', __dir__)
|
10
|
+
SETUP_DEST_PATH = '/tmp/setup-scripts'
|
11
|
+
|
12
|
+
def initialize
|
13
|
+
@network = 'test'
|
14
|
+
@ai_gateway_name = 'ai-gateway'
|
15
|
+
@ai_gateway_hostname = "#{@ai_gateway_name}.#{@network}"
|
16
|
+
@ai_gateway_port = 5000
|
17
|
+
end
|
18
|
+
|
19
|
+
def perform(release, *rspec_args)
|
20
|
+
Component::Gitlab.perform do |gitlab|
|
21
|
+
setup_gitlab(gitlab, release)
|
22
|
+
|
23
|
+
Component::AiGateway.perform do |ai_gateway|
|
24
|
+
setup_ai_gateway(ai_gateway, gitlab_hostname: gitlab.hostname)
|
25
|
+
|
26
|
+
ai_gateway.instance do
|
27
|
+
gitlab.instance do
|
28
|
+
setup_code_suggestions(gitlab)
|
29
|
+
run_specs(gitlab, *rspec_args)
|
30
|
+
end
|
31
|
+
end
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
private
|
37
|
+
|
38
|
+
def setup_gitlab(gitlab, release)
|
39
|
+
gitlab.release = QA::Release.new(release)
|
40
|
+
gitlab.name = 'gitlab'
|
41
|
+
gitlab.network = @network
|
42
|
+
|
43
|
+
gitlab.omnibus_gitlab_rails_env['CODE_SUGGESTIONS_BASE_URL'] = "http://#{@ai_gateway_hostname}:#{@ai_gateway_port}"
|
44
|
+
|
45
|
+
# Determines whether to use the SyncServiceTokenWorker or SyncSeatLinkWorker
|
46
|
+
# for generating a code suggestions access token during license activation
|
47
|
+
gitlab.omnibus_gitlab_rails_env['QA_USE_SYNC_SERVICE_TOKEN_WORKER'] = Runtime::Env.use_sync_service_token_worker?
|
48
|
+
|
49
|
+
gitlab.set_ee_activation_code
|
50
|
+
end
|
51
|
+
|
52
|
+
def setup_ai_gateway(ai_gateway, gitlab_hostname:)
|
53
|
+
ai_gateway.name = @ai_gateway_name
|
54
|
+
ai_gateway.network = @network
|
55
|
+
ai_gateway.ports = [@ai_gateway_port]
|
56
|
+
|
57
|
+
ai_gateway.configure_environment(gitlab_hostname: gitlab_hostname)
|
58
|
+
end
|
59
|
+
|
60
|
+
def setup_code_suggestions(gitlab)
|
61
|
+
Runtime::Logger.info('Setting up Code Suggestions on GitLab instance')
|
62
|
+
|
63
|
+
gitlab.docker.copy(gitlab.name, SETUP_SRC_PATH, SETUP_DEST_PATH)
|
64
|
+
|
65
|
+
gitlab.docker.exec(
|
66
|
+
gitlab.name,
|
67
|
+
"gitlab-rails runner #{SETUP_DEST_PATH}/code_suggestions_setup.rb",
|
68
|
+
mask_secrets: gitlab.secrets
|
69
|
+
)
|
70
|
+
end
|
71
|
+
|
72
|
+
def run_specs(gitlab, *rspec_args)
|
73
|
+
Runtime::Logger.info('Running AI Gateway specs!')
|
74
|
+
|
75
|
+
rspec_args << "--" unless rspec_args.include?('--')
|
76
|
+
rspec_args << %w[--tag ai_gateway]
|
77
|
+
|
78
|
+
Component::Specs.perform do |specs|
|
79
|
+
specs.suite = 'Test::Instance::All'
|
80
|
+
specs.release = gitlab.release
|
81
|
+
specs.network = gitlab.network
|
82
|
+
specs.args = [gitlab.address, *rspec_args]
|
83
|
+
end
|
84
|
+
end
|
85
|
+
end
|
86
|
+
end
|
87
|
+
end
|
88
|
+
end
|
89
|
+
end
|
90
|
+
end
|
data/lib/gitlab/qa/version.rb
CHANGED
@@ -0,0 +1,59 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
class CodeSuggestionsSetup
|
4
|
+
class << self
|
5
|
+
def configure!
|
6
|
+
if ENV.fetch('QA_USE_SYNC_SERVICE_TOKEN_WORKER', false)
|
7
|
+
puts 'Enabling use_sync_service_token_worker feature flag...'
|
8
|
+
Feature.enable(:use_sync_service_token_worker)
|
9
|
+
end
|
10
|
+
|
11
|
+
puts 'Enabling code_suggestions_tokens_api feature flag...'
|
12
|
+
Feature.enable(:code_suggestions_tokens_api)
|
13
|
+
|
14
|
+
activate_cloud_license
|
15
|
+
|
16
|
+
# Due to the various async Sidekiq processes involved, we wait to verify
|
17
|
+
# that the code suggestions access token has been generated before proceeding
|
18
|
+
verify_code_suggestions_access_token
|
19
|
+
|
20
|
+
puts 'Enabling application setting instance_level_code_suggestions_enabled...'
|
21
|
+
ApplicationSetting.last.update(instance_level_code_suggestions_enabled: true)
|
22
|
+
end
|
23
|
+
|
24
|
+
private
|
25
|
+
|
26
|
+
def activate_cloud_license
|
27
|
+
puts 'Activating cloud license...'
|
28
|
+
activation_result = ::GitlabSubscriptions::ActivateService.new.execute(ENV.fetch('QA_EE_ACTIVATION_CODE', nil))
|
29
|
+
|
30
|
+
if activation_result[:success]
|
31
|
+
puts 'Cloud license activation successful'
|
32
|
+
else
|
33
|
+
puts 'Cloud license activation failed!'
|
34
|
+
puts Array(result[:errors]).join(' ')
|
35
|
+
exit 1
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
def verify_code_suggestions_access_token
|
40
|
+
puts 'Waiting for code suggestions access token to be available...'
|
41
|
+
|
42
|
+
max_attempts = 3
|
43
|
+
attempts = 0
|
44
|
+
|
45
|
+
until (tokens = ::Ai::ServiceAccessToken.active.count)&.positive? || attempts == max_attempts
|
46
|
+
puts 'Attempting to verify access token exists...'
|
47
|
+
attempts += 1
|
48
|
+
sleep 30
|
49
|
+
end
|
50
|
+
|
51
|
+
return if tokens&.positive?
|
52
|
+
|
53
|
+
puts "Failed to create code suggestions access token after #{max_attempts} attempts"
|
54
|
+
exit 1
|
55
|
+
end
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
59
|
+
CodeSuggestionsSetup.configure!
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: gitlab-qa
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 13.
|
4
|
+
version: 13.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- GitLab Quality
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-12-
|
11
|
+
date: 2023-12-14 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: climate_control
|
@@ -383,6 +383,7 @@ files:
|
|
383
383
|
- gitlab-qa.gemspec
|
384
384
|
- lefthook.yml
|
385
385
|
- lib/gitlab/qa.rb
|
386
|
+
- lib/gitlab/qa/component/ai_gateway.rb
|
386
387
|
- lib/gitlab/qa/component/alpine.rb
|
387
388
|
- lib/gitlab/qa/component/base.rb
|
388
389
|
- lib/gitlab/qa/component/chaos.rb
|
@@ -444,6 +445,7 @@ files:
|
|
444
445
|
- lib/gitlab/qa/scenario/test/instance/staging_geo.rb
|
445
446
|
- lib/gitlab/qa/scenario/test/instance/staging_ref.rb
|
446
447
|
- lib/gitlab/qa/scenario/test/instance/staging_ref_geo.rb
|
448
|
+
- lib/gitlab/qa/scenario/test/integration/ai_gateway.rb
|
447
449
|
- lib/gitlab/qa/scenario/test/integration/chaos.rb
|
448
450
|
- lib/gitlab/qa/scenario/test/integration/client_ssl.rb
|
449
451
|
- lib/gitlab/qa/scenario/test/integration/elasticsearch.rb
|
@@ -496,6 +498,7 @@ files:
|
|
496
498
|
- support/manifests/suggested_reviewer/pubsub.yaml
|
497
499
|
- support/manifests/suggested_reviewer/recommender-bot.yaml
|
498
500
|
- support/manifests/suggested_reviewer/recommender.yaml
|
501
|
+
- support/setup/code_suggestions_setup.rb
|
499
502
|
- tls_certificates/authority/ca.crt
|
500
503
|
- tls_certificates/authority/ca.key
|
501
504
|
- tls_certificates/authority/ca.pem
|