govuk_sidekiq 6.0.0 → 7.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0b29995a70caf772608c804cddc8f38fedc0d79c5a02157a8c9d1c46576403fc
4
- data.tar.gz: 660d8a260f5a175c4bea49e978eb2d302fad1d0a35672e2740770908e134f789
3
+ metadata.gz: 07e39d37942f15f173005674d60471d56fbe3db4d59d35e67ba5c0e1a33155df
4
+ data.tar.gz: 3a68c0f33812e2558cb5374b9c56533d2573732702e15eb0e5bc38ef588b63ef
5
5
  SHA512:
6
- metadata.gz: 2c2ef5066358f76e4c6b4452bb452d498f8c4e7fe38363b570488457cbe3d8c687aec4fac3fee9c5a6208e9e8f57015b7e8ee6fc277246e7e1626b6dcd25c5f2
7
- data.tar.gz: 9982cee079fc37d903726c2c6ef259d48884802fa410b071a07ae16d32f1169d0f5a7bba1d4484fac0836dfa9fbbf062f0afb8606e223f232f56851390423981
6
+ metadata.gz: 1de3bd1211158830997a52e81f3ce1086732d66dc6534c6b4faf08709bd5b16d10555fcf4a571e17abb01b7cc68dea594c5ff07d348f6cea8ad927c1118311f6
7
+ data.tar.gz: 833611f37c06d5cd8cf621f0115650915aec9e69cf1dd37e3e48d6f66ea3347cf2600eefbe270a4d7e7e3dc783e35909c19d78d78aa13f873117bc4fa6c07dad
data/README.md CHANGED
@@ -13,9 +13,7 @@ What does `govuk_sidekiq` do for you?
13
13
  This means that for each request a unique ID (`govuk_request_id`) will be passed on to downstream applications.
14
14
  [Read more about request tracing][req-tracing].
15
15
  3. Makes sure that we use JSON logging, so that Sidekiq logs will end up
16
- properly in Kibana.
17
- 4. Sends activity stats to Statsd, so that you can make pretty graphs of activity
18
- in Grafana or Graphite. See the [Rummager dashboards for an example](https://grafana.publishing.service.gov.uk/dashboard/file/rummager_queues.json).
16
+ properly in Logit and searchable through Kibana.
19
17
 
20
18
  [req-tracing]: https://docs.publishing.service.gov.uk/manual/setting-up-request-tracing.html
21
19
 
@@ -39,46 +37,20 @@ gem "govuk_sidekiq"
39
37
  This file also allows you to configure queues with priority.
40
38
  [See the Sidekiq wiki for available options](https://github.com/mperham/sidekiq/wiki/Advanced-Options).
41
39
 
42
- ### 3. Add a Procfile
40
+ ### 3. Configure environment variables in EKS
43
41
 
44
- This is what puppet uses to create the process.
42
+ For each environment ([integration](https://github.com/alphagov/govuk-helm-charts/blob/main/charts/app-config/values-integration.yaml), [staging](https://github.com/alphagov/govuk-helm-charts/blob/main/charts/app-config/values-staging.yaml) and [production](https://github.com/alphagov/govuk-helm-charts/blob/main/charts/app-config/values-production.yaml)), add a `REDIS_URL` environment variable for your application with a value of `redis://shared-redis-govuk.eks.{environment}.govuk-internal.digital`.
45
43
 
46
- ```sh
47
- # Procfile
48
- worker: bundle exec sidekiq -C ./config/sidekiq.yml
49
- ```
50
-
51
- ### 4. Configure puppet
52
-
53
- - Set a `REDIS_URL` environment variable. `GOVUK_APP_NAME` should also be
54
- set, but this is already done by the default `govuk::app::config`.
55
-
56
- Apply redis variables for your app in [the default config](https://github.com/alphagov/govuk-puppet/blob/main/hieradata_aws/common.yaml). For example:
57
-
58
- ```
59
- govuk::apps::your_app::redis_host: "%{hiera('sidekiq_host')}"
60
- govuk::apps::your_app::redis_port: "%{hiera('sidekiq_port')}"
61
- ```
62
- - Make sure puppet creates and starts the Procfile worker.
44
+ Additionally, set the value of `workerEnabled` to `true` for your application. This will [result in a `worker` process](https://github.com/alphagov/govuk-helm-charts/blob/8b008832b5e8f62f2f489d3b030be21945d2b08b/charts/generic-govuk-app/values.yaml#L16-L21) running alongside the web application. The queue length and max delay can be monitored using the [Sidekiq Grafana dashboard](https://grafana.eks.production.govuk.digital/d/sidekiq-queues), once the Sidekiq worker is initialised.
63
45
 
64
- There's no step-by-step guide for this, but [you can copy the config from collections-publisher](https://github.com/alphagov/govuk-puppet/blob/main/modules/govuk/manifests/apps/collections_publisher.pp).
65
-
66
- ### 5. Configure deployment scripts
67
-
68
- Make sure you restart the worker after deploying by adding a hook to the [capistrano scripts in govuk-app-deployment](https://github.com/alphagov/govuk-app-deployment). Otherwise the worker will keep running old code.
46
+ There's no step-by-step guide for this, but [you can copy the changes made when Sidekiq was added to the release application](https://github.com/alphagov/govuk-helm-charts/pull/1117/files). You may also want to [resize resource requests](https://github.com/alphagov/govuk-helm-charts/pull/1121/files) for the app depending on the predicted request rate.
69
47
 
70
48
  ```ruby
71
49
  # your-application/config/deploy.rb
72
50
  after "deploy:restart", "deploy:restart_procfile_worker"
73
51
  ```
74
52
 
75
- ### 6. Add app to sidekiq-monitoring
76
-
77
- See the dev docs for a step-by-step guide: [Add sidekiq-monitoring to your application][monitoring]
78
-
79
- [monitoring]: https://docs.publishing.service.gov.uk/manual/setting-up-new-sidekiq-monitoring-app.html
80
-
81
- ### 8. Create some jobs
53
+ ### 4. Create some jobs
82
54
 
83
55
  You can [use normal Sidekiq jobs](https://github.com/mperham/sidekiq/wiki/Getting-Started):
84
56
 
@@ -1,4 +1,5 @@
1
1
  require "gds_api/govuk_headers"
2
+ require "sidekiq"
2
3
 
3
4
  module GovukSidekiq
4
5
  module APIHeaders
@@ -16,6 +17,7 @@ module GovukSidekiq
16
17
  else
17
18
  job["args"] << header_arguments
18
19
  end
20
+ Sidekiq::Context.add("govuk_request_id", job["args"].last["request_id"])
19
21
 
20
22
  yield
21
23
  end
@@ -45,6 +47,7 @@ module GovukSidekiq
45
47
  authenticated_user = last_arg["authenticated_user"]
46
48
  GdsApi::GovukHeaders.set_header(:govuk_request_id, request_id)
47
49
  GdsApi::GovukHeaders.set_header(:x_govuk_authenticated_user, authenticated_user)
50
+ Sidekiq::Context.add("govuk_request_id", request_id)
48
51
  end
49
52
 
50
53
  yield
@@ -0,0 +1,19 @@
1
+ require "sidekiq"
2
+ require "govuk_sidekiq/api_headers"
3
+
4
+ module GovukSidekiq
5
+ class GovukJsonFormatter < Sidekiq::Logger::Formatters::Base
6
+ def call(severity, time, _, message)
7
+ hash = {
8
+ "@timestamp": time.utc.iso8601(3),
9
+ pid: ::Process.pid,
10
+ tid: tid,
11
+ level: severity,
12
+ message: message,
13
+ tags: %w[sidekiq],
14
+ }
15
+ ctx.each { |key, value| hash[key] = value unless hash[key] }
16
+ Sidekiq.dump_json(hash) << "\n"
17
+ end
18
+ end
19
+ end
@@ -1,7 +1,6 @@
1
1
  require "sidekiq"
2
- require "sidekiq-statsd"
3
2
  require "govuk_sidekiq/api_headers"
4
- require "govuk_app_config/govuk_statsd"
3
+ require "govuk_sidekiq/govuk_json_formatter"
5
4
 
6
5
  module GovukSidekiq
7
6
  module SidekiqInitializer
@@ -12,8 +11,6 @@ module GovukSidekiq
12
11
  )
13
12
 
14
13
  Sidekiq.configure_server do |config|
15
- config.log_formatter = Sidekiq::Logger::Formatters::JSON.new if ENV["GOVUK_SIDEKIQ_JSON_LOGGING"]
16
-
17
14
  # $real_stdout is defined by govuk_app_config and is used to point to
18
15
  # STDOUT as that redirects $stdout to actually be $stderr.
19
16
  # When govuk_app_config does this we need to use $real_stdout to output logs to STDOUT.
@@ -22,11 +19,11 @@ module GovukSidekiq
22
19
  # rubocop:disable Style/GlobalVars
23
20
  config.logger = Sidekiq::Logger.new($real_stdout) if defined?($real_stdout)
24
21
  # rubocop:enable Style/GlobalVars
22
+ config.log_formatter = GovukSidekiq::GovukJsonFormatter.new if ENV["GOVUK_SIDEKIQ_JSON_LOGGING"]
25
23
 
26
24
  config.redis = redis_config
27
25
 
28
26
  config.server_middleware do |chain|
29
- chain.add Sidekiq::Statsd::ServerMiddleware, statsd: GovukStatsd, env: nil, prefix: "workers"
30
27
  chain.add GovukSidekiq::APIHeaders::ServerMiddleware
31
28
  end
32
29
  end
@@ -1,3 +1,3 @@
1
1
  module GovukSidekiq
2
- VERSION = "6.0.0".freeze
2
+ VERSION = "7.1.0".freeze
3
3
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: govuk_sidekiq
3
3
  version: !ruby/object:Gem::Version
4
- version: 6.0.0
4
+ version: 7.1.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - GOV.UK Dev
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2023-01-05 00:00:00.000000000 Z
11
+ date: 2023-06-20 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: gds-api-adapters
@@ -66,20 +66,6 @@ dependencies:
66
66
  - - "~>"
67
67
  - !ruby/object:Gem::Version
68
68
  version: '6'
69
- - !ruby/object:Gem::Dependency
70
- name: sidekiq-statsd
71
- requirement: !ruby/object:Gem::Requirement
72
- requirements:
73
- - - ">="
74
- - !ruby/object:Gem::Version
75
- version: '2.1'
76
- type: :runtime
77
- prerelease: false
78
- version_requirements: !ruby/object:Gem::Requirement
79
- requirements:
80
- - - ">="
81
- - !ruby/object:Gem::Version
82
- version: '2.1'
83
69
  - !ruby/object:Gem::Dependency
84
70
  name: climate_control
85
71
  requirement: !ruby/object:Gem::Requirement
@@ -142,14 +128,14 @@ dependencies:
142
128
  requirements:
143
129
  - - '='
144
130
  - !ruby/object:Gem::Version
145
- version: 4.9.0
131
+ version: 4.10.0
146
132
  type: :development
147
133
  prerelease: false
148
134
  version_requirements: !ruby/object:Gem::Requirement
149
135
  requirements:
150
136
  - - '='
151
137
  - !ruby/object:Gem::Version
152
- version: 4.9.0
138
+ version: 4.10.0
153
139
  description:
154
140
  email:
155
141
  - govuk-dev@digital.cabinet-office.gov.uk
@@ -161,6 +147,7 @@ files:
161
147
  - README.md
162
148
  - lib/govuk_sidekiq.rb
163
149
  - lib/govuk_sidekiq/api_headers.rb
150
+ - lib/govuk_sidekiq/govuk_json_formatter.rb
164
151
  - lib/govuk_sidekiq/railtie.rb
165
152
  - lib/govuk_sidekiq/sidekiq_initializer.rb
166
153
  - lib/govuk_sidekiq/testing.rb
@@ -177,14 +164,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
177
164
  requirements:
178
165
  - - ">="
179
166
  - !ruby/object:Gem::Version
180
- version: '2.7'
167
+ version: '3.0'
181
168
  required_rubygems_version: !ruby/object:Gem::Requirement
182
169
  requirements:
183
170
  - - ">="
184
171
  - !ruby/object:Gem::Version
185
172
  version: '0'
186
173
  requirements: []
187
- rubygems_version: 3.4.2
174
+ rubygems_version: 3.4.14
188
175
  signing_key:
189
176
  specification_version: 4
190
177
  summary: Provides standard setup and behaviour for Sidekiq in GOV.UK applications.