bidi2pdf-rails 0.1.6 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +30 -2
- data/README.md +111 -2
- data/lib/bidi2pdf_rails/chrome_sweeping.rb +132 -0
- data/lib/bidi2pdf_rails/chromedriver_manager_singleton.rb +17 -4
- data/lib/bidi2pdf_rails/config.rb +48 -0
- data/lib/bidi2pdf_rails/railtie.rb +4 -0
- data/lib/bidi2pdf_rails/services/pdf_browser_session.rb +18 -10
- data/lib/bidi2pdf_rails/tasks/sessions.rake +30 -0
- data/lib/bidi2pdf_rails/test_helpers/settings_helper.rb +29 -5
- data/lib/bidi2pdf_rails/version.rb +1 -1
- data/lib/bidi2pdf_rails.rb +14 -0
- data/spec/acceptance/user_can_connect_to_an_external_webdriver_spec.rb +1 -1
- data/spec/acceptance/user_can_have_leaked_chrome_sessions_swept_spec.rb +88 -0
- data/spec/dummy/log/test.log +3434 -0
- data/spec/unit/bidi2pdf_rails/chrome_sweeping_spec.rb +164 -0
- data/spec/unit/bidi2pdf_rails/chromedriver_manager_singleton_spec.rb +61 -0
- metadata +9 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 74c1f65510817caf8e3b01167055b489bbfd70c018818865e3063eccab4eba01
|
|
4
|
+
data.tar.gz: 01d9b3124d923686e935cf5a6ce92424496a229e091148f7d828c5927b5c9256
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 60f0e9270cef332f9086de1bedde5e9462bb2789900ea3dc4c7017ca873f591084619ff277ab88fa2688b9f28cf2d1706f651c5e19e13abd3cc4b5dc4403e0d3
|
|
7
|
+
data.tar.gz: 8dfb6bdf5650cb0d4c4e92786a880b896a4cdfd3ca4825b74a4943d4048530a854d02442c82cbec33c6f8336042c8e98e93258b6b2296bd253707da75cf080d8
|
data/CHANGELOG.md
CHANGED
|
@@ -8,10 +8,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
-
[unreleased]: https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.
|
|
11
|
+
[unreleased]: https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.7..HEAD
|
|
12
12
|
|
|
13
13
|
<!-- generated by git-cliff end -->
|
|
14
14
|
|
|
15
|
+
## [0.1.7] - 2026-09-30
|
|
16
|
+
|
|
17
|
+
### 🐛 Fixed
|
|
18
|
+
|
|
19
|
+
- Match only the stable example.com sentence by @dieter-medium
|
|
20
|
+
in [#27](https://github.com/dieter-medium/bidi2pdf-rails/pull/27)
|
|
21
|
+
- Keep sweeper leases and retries consistent by @dieter-medium
|
|
22
|
+
in [#27](https://github.com/dieter-medium/bidi2pdf-rails/pull/27)
|
|
23
|
+
- Keep the first original in the settings helpers by @dieter-medium
|
|
24
|
+
in [#27](https://github.com/dieter-medium/bidi2pdf-rails/pull/27)
|
|
25
|
+
- Document and address blank PDFs from Chrome's network checks by @dieter-medium
|
|
26
|
+
in [#25](https://github.com/dieter-medium/bidi2pdf-rails/pull/25)
|
|
27
|
+
|
|
28
|
+
### 📝 Docs
|
|
29
|
+
|
|
30
|
+
- Name bidi2pdf's lease_ttl option by @dieter-medium
|
|
31
|
+
in [#27](https://github.com/dieter-medium/bidi2pdf-rails/pull/27)
|
|
32
|
+
|
|
33
|
+
### 🚀 Added
|
|
34
|
+
|
|
35
|
+
- Sweep leaked Chrome sessions on the remote browser by @dieter-medium
|
|
36
|
+
in [#27](https://github.com/dieter-medium/bidi2pdf-rails/pull/27)
|
|
37
|
+
|
|
38
|
+
[0.1.7]: https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.6..v0.1.7
|
|
39
|
+
|
|
15
40
|
## [0.1.6] - 2026-09-21
|
|
16
41
|
|
|
17
42
|
### 🎨 Refactored
|
|
@@ -140,7 +165,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
140
165
|
|
|
141
166
|
### 🔄 Released
|
|
142
167
|
|
|
143
|
-
- [unreleased](https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.
|
|
168
|
+
- [unreleased](https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.7..HEAD)
|
|
169
|
+
- [0.1.7](https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.6..v0.1.7)
|
|
170
|
+
- [0.1.6](https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.5..v0.1.6)
|
|
171
|
+
- [0.1.5](https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.4..v0.1.5)
|
|
144
172
|
- [0.1.4](https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.3..v0.1.4)
|
|
145
173
|
- [0.1.3](https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.2..v0.1.3)
|
|
146
174
|
- [0.1.2](https://github.com/dieter-medium/bidi2pdf-rails/compare/v0.1.1..v0.1.2)
|
data/README.md
CHANGED
|
@@ -216,6 +216,56 @@ Implementing these steps helps avoid interlock-induced deadlocks when generating
|
|
|
216
216
|
|
|
217
217
|
---
|
|
218
218
|
|
|
219
|
+
## 🕳️ Blank PDFs: Chrome's Local Network Access Check
|
|
220
|
+
|
|
221
|
+
**Symptom.** `render pdf:` of a Rails view returns a PDF with no styling and, with a JS-driven layout
|
|
222
|
+
(Paged.js, Stimulus, ...), no content at all. Nothing raises. In the log every asset request ends in
|
|
223
|
+
`state="error"`, the wait for the page runs into its timeout, and - the telltale part - **your app
|
|
224
|
+
never logs a single `GET /assets/...` for that render**.
|
|
225
|
+
|
|
226
|
+
**Cause.** A rendered view is handed to Chrome as a `data:` URL. Chrome treats that document as a
|
|
227
|
+
*public* origin, while its stylesheets and scripts point back at your app - which in development, CI
|
|
228
|
+
and most Docker setups is a *private* address (`localhost`, `host.docker.internal`, a Compose service
|
|
229
|
+
name). Recent Chrome versions (confirmed with Chrome 153) block a public document from requesting
|
|
230
|
+
private-network resources, before the request ever leaves the browser. That is why CORS settings
|
|
231
|
+
make no difference here: there is no request for your app to answer.
|
|
232
|
+
|
|
233
|
+
**Fix.** Opt out of that one check where your asset host is private:
|
|
234
|
+
|
|
235
|
+
```ruby
|
|
236
|
+
# config/initializers/bidi2pdf_rails.rb
|
|
237
|
+
chrome_args = Bidi2pdf::Bidi::Session::DEFAULT_CHROME_ARGS.dup
|
|
238
|
+
|
|
239
|
+
unless Rails.env.production?
|
|
240
|
+
chrome_args.map! do |arg|
|
|
241
|
+
arg.start_with?("--disable-features=") ? "#{arg},LocalNetworkAccessChecks" : arg
|
|
242
|
+
end
|
|
243
|
+
end
|
|
244
|
+
|
|
245
|
+
config.general_options.chrome_session_args = chrome_args
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
- **Merge, don't append.** `DEFAULT_CHROME_ARGS` already carries a `--disable-features=` list, and
|
|
249
|
+
Chrome only honors the *last* occurrence of that switch - a second one silently drops bidi2pdf's
|
|
250
|
+
own defaults.
|
|
251
|
+
- `BlockInsecurePrivateNetworkRequests` alone does **not** help; `LocalNetworkAccessChecks` is the
|
|
252
|
+
feature that matters.
|
|
253
|
+
- **Leave the check on in production** when your assets come from a public `https://` asset host -
|
|
254
|
+
nothing is blocked there, and it is a real browser protection.
|
|
255
|
+
- Rendering a remote `url:` that is served from the same private host as its assets is not
|
|
256
|
+
affected - document and assets then share one address space.
|
|
257
|
+
- With the session warmer enabled, restart the app after changing Chrome arguments - warm sessions
|
|
258
|
+
were started with the old ones.
|
|
259
|
+
|
|
260
|
+
The second half of the same problem is the name itself: the asset host has to be one the *browser*
|
|
261
|
+
can resolve back to your app (`pdf_settings.asset_host`, or your own `config.asset_host`), which for
|
|
262
|
+
a remote or containerized Chrome is rarely the URL you typed into your own browser.
|
|
263
|
+
|
|
264
|
+
> This repo's own dummy app never runs into the check because it starts Chrome with
|
|
265
|
+
> `--disable-web-security` - fine for a test fixture, not something to copy into an application.
|
|
266
|
+
|
|
267
|
+
---
|
|
268
|
+
|
|
219
269
|
## 🧪 Acceptance Examples
|
|
220
270
|
|
|
221
271
|
This repo includes **real integration tests** that serve as usage documentation:
|
|
@@ -230,6 +280,7 @@ This repo includes **real integration tests** that serve as usage documentation:
|
|
|
230
280
|
- [Rendering with a pre-warmed session pool (
|
|
231
281
|
`Bidi2pdf::SessionWarmer`)](spec/acceptance/user_can_render_with_a_warm_session_spec.rb)
|
|
232
282
|
- [Using the session warmer against a remote chromedriver](spec/acceptance/user_can_render_with_a_warm_session_using_remote_chromedriver_spec.rb)
|
|
283
|
+
- [Closing Chrome sessions left behind on a remote chromedriver](spec/acceptance/user_can_have_leaked_chrome_sessions_swept_spec.rb)
|
|
233
284
|
|
|
234
285
|
---
|
|
235
286
|
|
|
@@ -245,7 +296,7 @@ bin/rails generate bidi2pdf_rails:initializer
|
|
|
245
296
|
|
|
246
297
|
Or explore [Bidi2pdfRails::Config::CONFIG_OPTIONS](lib/bidi2pdf_rails/config.rb) in the source.
|
|
247
298
|
|
|
248
|
-
Requires **bidi2pdf >= 0.1.
|
|
299
|
+
Requires **bidi2pdf >= 0.1.20**. Settings that came with it:
|
|
249
300
|
|
|
250
301
|
| Setting | Default | Description |
|
|
251
302
|
|----------------------------------------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
@@ -264,12 +315,70 @@ when the warmer is on.
|
|
|
264
315
|
|
|
265
316
|
**Session warmer settings are boot-time immutable.** `Bidi2pdf::SessionWarmer` is configured once, on
|
|
266
317
|
the first `ChromedriverManagerSingleton.initialize_manager` call. A later change to
|
|
267
|
-
`session_warmer_settings`, `general_options.headless`/`chrome_session_args`, or
|
|
318
|
+
`session_warmer_settings`, `sweeper_settings`, `general_options.headless`/`chrome_session_args`, or
|
|
268
319
|
`render_remote_settings.browser_url` is logged as a warning and otherwise ignored until you call
|
|
269
320
|
`ChromedriverManagerSingleton.shutdown` followed by `.initialize_manager` again to re-apply it.
|
|
270
321
|
Outside a server process (specs, a console) both calls return early unless you pass `force: true`;
|
|
271
322
|
`initialize_manager force: true` on its own also works - it stops whatever is running first.
|
|
272
323
|
|
|
324
|
+
### 🧹 Leaked Chrome sessions (remote browser)
|
|
325
|
+
|
|
326
|
+
A remote chromedriver keeps every session - a whole Chrome - until someone deletes it, and a worker
|
|
327
|
+
that is killed or crashes leaves its sessions open until the container runs out of room for new
|
|
328
|
+
ones. With a remote browser, every session a render opens is **leased** in a small registry file:
|
|
329
|
+
the rendering process renews the lease while the session is open, so no sweeper in any process
|
|
330
|
+
closes it - not another Puma worker's, not a job worker's. When a process dies, its leases run out
|
|
331
|
+
and its sessions become leftovers.
|
|
332
|
+
|
|
333
|
+
`sweeper_settings.enabled` puts a `Bidi2pdf::ChromeSweeper` in every process:
|
|
334
|
+
|
|
335
|
+
- a render that fails for lack of resources - chromedriver refusing a session, Chrome dying or not
|
|
336
|
+
answering - is **retried once after a last-resort sweep**, which closes every session nobody holds
|
|
337
|
+
that is older than `min_age` (`retry_on_failure`); a page error (navigation, script) is not retried;
|
|
338
|
+
- with an `interval`, it sweeps in the background by its rules: sessions older than `orphan_age`,
|
|
339
|
+
unresponsive ones, and the oldest ones over `max_sessions`;
|
|
340
|
+
- the warmer, when enabled, leases its sessions and sweeps before retrying a refused session.
|
|
341
|
+
|
|
342
|
+
```ruby
|
|
343
|
+
config.render_remote_settings.browser_url = "http://remote-chrome:3000/session"
|
|
344
|
+
config.sweeper_settings.enabled = true
|
|
345
|
+
config.sweeper_settings.scope = "all" # the chromedriver belongs to this app
|
|
346
|
+
config.sweeper_settings.max_sessions = "auto" # floor(pids_limit * 0.8 / 110)
|
|
347
|
+
config.sweeper_settings.pids_limit = 1024 # the chromedriver container's pids limit
|
|
348
|
+
config.sweeper_settings.interval = 60
|
|
349
|
+
config.sweeper_settings.registry_dir = "/shared/bidi2pdf" # web and job containers must share it
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
| Setting | Default | Description |
|
|
353
|
+
|-----------------------------------------|--------------|----------------------------------------------------------------------------------------------------------------------------------------------|
|
|
354
|
+
| `sweeper_settings.enabled` | `false` | Run a sweeper against `render_remote_settings.browser_url`, and retry a render that failed for lack of resources. |
|
|
355
|
+
| `sweeper_settings.scope` | `"recorded"` | `"recorded"`: only sessions bidi2pdf recorded. `"all"`: every session on the chromedriver - only for a chromedriver your application owns. |
|
|
356
|
+
| `sweeper_settings.orphan_age` | `600` | Close sessions nobody holds that are older than this many seconds; `nil` turns the rule off. |
|
|
357
|
+
| `sweeper_settings.min_age` | `60` | Never close a session younger than this. |
|
|
358
|
+
| `sweeper_settings.unresponsive_checks` | `2` | Close a session nobody holds after this many failed checks in a row; `nil` turns the rule off. |
|
|
359
|
+
| `sweeper_settings.max_sessions` | `nil` | Close the oldest sessions nobody holds while more exist; `"auto"` derives it from `pids_limit`. |
|
|
360
|
+
| `sweeper_settings.pids_limit` | `nil` | The chromedriver container's pids limit, for `max_sessions = "auto"`. |
|
|
361
|
+
| `sweeper_settings.interval` | `nil` | Seconds between background sweeps in every process; `nil` sweeps only when a render fails or on demand. |
|
|
362
|
+
| `sweeper_settings.registry_dir` | `Dir.tmpdir` | Where the registry lives. Processes only see each other's leases when they share it. |
|
|
363
|
+
| `sweeper_settings.retry_on_failure` | `true` | Retry a render once after a last-resort sweep; with the warmer, also a session it was refused. Off: no retry anywhere. |
|
|
364
|
+
|
|
365
|
+
On demand, e.g. when the application suspects a leak, or from a shell during an incident:
|
|
366
|
+
|
|
367
|
+
```ruby
|
|
368
|
+
Bidi2pdfRails.chrome_sessions # ids, ages, live or not - no page content
|
|
369
|
+
Bidi2pdfRails.sweep_sessions! # one sweep by the configured rules
|
|
370
|
+
Bidi2pdfRails.sweep_sessions!(pressure: true, dry_run: true) # what a last-resort sweep would close
|
|
371
|
+
```
|
|
372
|
+
|
|
373
|
+
```bash
|
|
374
|
+
bin/rails bidi2pdf_rails:sessions
|
|
375
|
+
bin/rails bidi2pdf_rails:sweep # PRESSURE=1, DRY_RUN=1, CHECK_INTERVAL=10
|
|
376
|
+
```
|
|
377
|
+
|
|
378
|
+
Both work whether or not `sweeper_settings.enabled` is set. Sessions other tools opened carry no
|
|
379
|
+
lease: with `scope = "all"`, only `min_age` protects them. See bidi2pdf's README ("Leaked Chrome
|
|
380
|
+
sessions") for how sessions are aged and checked.
|
|
381
|
+
|
|
273
382
|
**Rails 8.1.3.1 and `json` 3.** `json` 3 accepts options as keywords only, while Rails 8.1.3.1 still
|
|
274
383
|
passes `JSON.parse` a positional options hash. With both in one bundle, Active Storage attachments
|
|
275
384
|
and signed cookies fail with `ArgumentError: wrong number of arguments (given 2, expected 1)` - this
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Bidi2pdfRails
|
|
4
|
+
# Connects bidi2pdf's leased session registry and Bidi2pdf::ChromeSweeper to the Rails settings
|
|
5
|
+
# (sweeper_settings) - remote browser only, there is no shared chromedriver otherwise.
|
|
6
|
+
#
|
|
7
|
+
# Every remote session a render opens is leased in the registry (#registry), sweeper or not: a
|
|
8
|
+
# sweeper in another process - another Puma worker, a job worker - then knows it is in use. With
|
|
9
|
+
# sweeper_settings.enabled, each process gets one sweeper (#sweeper): it sweeps every
|
|
10
|
+
# sweeper_settings.interval seconds when set, and #with_retry makes it the last resort for a
|
|
11
|
+
# render that failed for lack of resources.
|
|
12
|
+
module ChromeSweeping
|
|
13
|
+
# sweeper_settings keys passed to Bidi2pdf::ChromeSweeper as they are.
|
|
14
|
+
# Bidi2pdf::ChromeSweeper's lease_ttl option is deliberately not passed: every lease carries the
|
|
15
|
+
# TTL its owner's heartbeat promises, and that option only covers entries written by bidi2pdf
|
|
16
|
+
# 0.1.18.
|
|
17
|
+
PASSED_THROUGH = %i[orphan_age min_age unresponsive_checks pids_limit registry_dir].freeze
|
|
18
|
+
|
|
19
|
+
# With the warmer, a refused session was already swept for and retried by the warmer itself -
|
|
20
|
+
# retrying the whole render on top would stack a second pair of attempts.
|
|
21
|
+
RETRYABLE_WITH_WARMER = ->(error) { !error.is_a?(Bidi2pdf::SessionNotStartedError) && Bidi2pdf::ChromeSweeper::RETRYABLE.call(error) }
|
|
22
|
+
|
|
23
|
+
class << self
|
|
24
|
+
def enabled?
|
|
25
|
+
Bidi2pdfRails.use_remote_browser? && settings.enabled_value
|
|
26
|
+
end
|
|
27
|
+
|
|
28
|
+
# The registry remote sessions are leased in; nil without a remote browser.
|
|
29
|
+
def registry
|
|
30
|
+
return unless Bidi2pdfRails.use_remote_browser?
|
|
31
|
+
|
|
32
|
+
Bidi2pdf::SessionRegistry.new(browser_url, dir: settings.registry_dir_value)
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# Bidi2pdf::ChromeSweeper settings from sweeper_settings (without the background interval).
|
|
36
|
+
def options
|
|
37
|
+
PASSED_THROUGH.to_h { |key| [key, settings.public_send("#{key}_value")] }
|
|
38
|
+
.merge(scope: settings.scope_value.to_s.to_sym, max_sessions: max_sessions)
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
# The sweeper settings for Bidi2pdf::SessionWarmer - nil unless it should retry a refused
|
|
42
|
+
# session, which is all its own sweeper does (background sweeps belong to #sweeper).
|
|
43
|
+
def warmer_sweeper_options
|
|
44
|
+
options if enabled? && settings.retry_on_failure_value
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
# This process's sweeper, nil when sweeping is off. Built on first use, never before - that is
|
|
48
|
+
# a render or a sweep in a worker, not the Puma master that forks it, whose background thread
|
|
49
|
+
# would not survive the fork. A forked process builds its own.
|
|
50
|
+
def sweeper
|
|
51
|
+
return unless enabled?
|
|
52
|
+
|
|
53
|
+
@mutex.synchronize do
|
|
54
|
+
reset_after_fork
|
|
55
|
+
@sweeper ||= build_sweeper
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
# Runs a render; when it fails for lack of resources, sweeps under pressure and runs it once
|
|
60
|
+
# more (Bidi2pdf::ChromeSweeper#with_retry). Without sweeping, or with retry_on_failure off,
|
|
61
|
+
# just runs it.
|
|
62
|
+
def with_retry(&)
|
|
63
|
+
current = settings.retry_on_failure_value ? sweeper : nil
|
|
64
|
+
return yield unless current
|
|
65
|
+
|
|
66
|
+
retry_on = ChromedriverManagerSingleton.session_warmer_active? ? RETRYABLE_WITH_WARMER : Bidi2pdf::ChromeSweeper::RETRYABLE
|
|
67
|
+
current.with_retry(retry_on: retry_on, &)
|
|
68
|
+
end
|
|
69
|
+
|
|
70
|
+
# One sweep now, with the configured rules - whether or not sweeper_settings.enabled.
|
|
71
|
+
#
|
|
72
|
+
# @param pressure [Boolean] the last resort: close everything nobody holds past min_age.
|
|
73
|
+
# @param dry_run [Boolean] report what would be closed, close nothing.
|
|
74
|
+
# @param check_interval [Numeric, nil] check twice this many seconds apart first, so the
|
|
75
|
+
# unresponsive rule applies too.
|
|
76
|
+
# @return [Bidi2pdf::ChromeSweeper::Result]
|
|
77
|
+
def sweep!(pressure: false, dry_run: false, check_interval: nil)
|
|
78
|
+
one_shot(dry_run: dry_run).sweep!(reason: :manual, pressure: pressure, check_interval: check_interval)
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
# Every session on the remote chromedriver in the configured scope; live ones are marked.
|
|
82
|
+
#
|
|
83
|
+
# @return [Array<Bidi2pdf::ChromeSweeper::Inspector::SessionInfo>]
|
|
84
|
+
def sessions
|
|
85
|
+
one_shot.sessions
|
|
86
|
+
end
|
|
87
|
+
|
|
88
|
+
# Stops the background sweeps of this process.
|
|
89
|
+
def stop
|
|
90
|
+
current = @mutex.synchronize { @sweeper.tap { @sweeper = nil } }
|
|
91
|
+
current&.stop
|
|
92
|
+
end
|
|
93
|
+
|
|
94
|
+
private
|
|
95
|
+
|
|
96
|
+
def settings
|
|
97
|
+
Bidi2pdfRails.config.sweeper_settings
|
|
98
|
+
end
|
|
99
|
+
|
|
100
|
+
def browser_url
|
|
101
|
+
Bidi2pdfRails.config.render_remote_settings.browser_url_value
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
def max_sessions
|
|
105
|
+
value = settings.max_sessions_value
|
|
106
|
+
value.to_s == "auto" ? :auto : value&.to_i
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
def build_sweeper
|
|
110
|
+
Bidi2pdf::ChromeSweeper.new(browser_url, interval: settings.interval_value, **options).tap do |built|
|
|
111
|
+
built.start if built.interval
|
|
112
|
+
end
|
|
113
|
+
end
|
|
114
|
+
|
|
115
|
+
def one_shot(dry_run: false)
|
|
116
|
+
raise ArgumentError, "sweeping needs render_remote_settings.browser_url" unless Bidi2pdfRails.use_remote_browser?
|
|
117
|
+
|
|
118
|
+
Bidi2pdf::ChromeSweeper.new(browser_url, dry_run: dry_run, **options)
|
|
119
|
+
end
|
|
120
|
+
|
|
121
|
+
# A forked child starts without the parent's sweeper - its thread did not survive the fork.
|
|
122
|
+
def reset_after_fork
|
|
123
|
+
return if @pid == Process.pid
|
|
124
|
+
|
|
125
|
+
@pid = Process.pid
|
|
126
|
+
@sweeper = nil
|
|
127
|
+
end
|
|
128
|
+
end
|
|
129
|
+
|
|
130
|
+
@mutex = Mutex.new
|
|
131
|
+
end
|
|
132
|
+
end
|
|
@@ -66,10 +66,12 @@ module Bidi2pdfRails
|
|
|
66
66
|
def session
|
|
67
67
|
Thread.current.bidi2pdf_rails_session ||= begin
|
|
68
68
|
if Bidi2pdfRails.use_remote_browser?
|
|
69
|
+
# Leased while open, so no other process's sweeper takes it.
|
|
69
70
|
session = Bidi2pdf::Bidi::Session.new(
|
|
70
71
|
session_url: Bidi2pdfRails.config.render_remote_settings.browser_url_value,
|
|
71
72
|
headless: Bidi2pdfRails.config.general_options.headless_value,
|
|
72
|
-
chrome_args: Bidi2pdfRails.config.general_options.chrome_session_args_value
|
|
73
|
+
chrome_args: Bidi2pdfRails.config.general_options.chrome_session_args_value,
|
|
74
|
+
registry: ChromeSweeping.registry
|
|
73
75
|
)
|
|
74
76
|
else
|
|
75
77
|
session = @manager.session
|
|
@@ -115,6 +117,8 @@ module Bidi2pdfRails
|
|
|
115
117
|
# session_warmer_enabled? says right now - a setting flipped in between must not leave the
|
|
116
118
|
# thing that is really running alive. Callers hold @mutex.
|
|
117
119
|
def teardown
|
|
120
|
+
ChromeSweeping.stop
|
|
121
|
+
|
|
118
122
|
case @active_mode
|
|
119
123
|
when :warmer
|
|
120
124
|
Bidi2pdfRails.logger.info "Shutting down Bidi2pdf::SessionWarmer"
|
|
@@ -154,7 +158,14 @@ module Bidi2pdfRails
|
|
|
154
158
|
c.max_idle_age = @session_warmer_settings_snapshot[:max_idle_age]
|
|
155
159
|
c.headless = @session_warmer_settings_snapshot[:headless]
|
|
156
160
|
c.chrome_args = @session_warmer_settings_snapshot[:chrome_args]
|
|
157
|
-
|
|
161
|
+
next unless Bidi2pdfRails.use_remote_browser?
|
|
162
|
+
|
|
163
|
+
c.remote_browser_url = @session_warmer_settings_snapshot[:remote_browser_url]
|
|
164
|
+
c.registry_dir = @session_warmer_settings_snapshot[:registry_dir]
|
|
165
|
+
# The warmer leases its sessions in remote mode on its own. Its sweeper exists only to sweep
|
|
166
|
+
# under pressure and retry when a new session is refused - so only with
|
|
167
|
+
# sweeper_settings.retry_on_failure; background sweeps belong to ChromeSweeping#sweeper.
|
|
168
|
+
c.sweeper = @session_warmer_settings_snapshot[:sweeper]
|
|
158
169
|
end
|
|
159
170
|
end
|
|
160
171
|
|
|
@@ -171,7 +182,9 @@ module Bidi2pdfRails
|
|
|
171
182
|
max_idle_age: Bidi2pdfRails.config.session_warmer_settings.max_idle_age_value,
|
|
172
183
|
headless: Bidi2pdfRails.config.general_options.headless_value,
|
|
173
184
|
chrome_args: Bidi2pdfRails.config.general_options.chrome_session_args_value,
|
|
174
|
-
remote_browser_url: Bidi2pdfRails.use_remote_browser? ? Bidi2pdfRails.config.render_remote_settings.browser_url_value : nil
|
|
185
|
+
remote_browser_url: Bidi2pdfRails.use_remote_browser? ? Bidi2pdfRails.config.render_remote_settings.browser_url_value : nil,
|
|
186
|
+
registry_dir: Bidi2pdfRails.config.sweeper_settings.registry_dir_value,
|
|
187
|
+
sweeper: ChromeSweeping.warmer_sweeper_options
|
|
175
188
|
}
|
|
176
189
|
end
|
|
177
190
|
|
|
@@ -181,7 +194,7 @@ module Bidi2pdfRails
|
|
|
181
194
|
|
|
182
195
|
Bidi2pdfRails.logger.warn(
|
|
183
196
|
"Bidi2pdf::SessionWarmer is already configured; session_warmer_settings/general_options/" \
|
|
184
|
-
"render_remote_settings changes are boot-time immutable and were ignored. Call " \
|
|
197
|
+
"render_remote_settings/sweeper_settings changes are boot-time immutable and were ignored. Call " \
|
|
185
198
|
"ChromedriverManagerSingleton.shutdown then .initialize_manager to apply them."
|
|
186
199
|
)
|
|
187
200
|
end
|
|
@@ -66,6 +66,54 @@ module Bidi2pdfRails
|
|
|
66
66
|
]
|
|
67
67
|
},
|
|
68
68
|
|
|
69
|
+
sweeper_settings: {
|
|
70
|
+
name: "Chrome Sweeper Settings (remote browser only)",
|
|
71
|
+
ask: "Close Chrome sessions left behind on the remote browser, and retry a render that failed for lack of resources? (y/n)",
|
|
72
|
+
options: [
|
|
73
|
+
{
|
|
74
|
+
name: :enabled,
|
|
75
|
+
desc: "Run a Bidi2pdf::ChromeSweeper against render_remote_settings.browser_url: sessions of dead " \
|
|
76
|
+
"processes are closed, and a render that failed for lack of resources is retried once after a " \
|
|
77
|
+
"last-resort sweep. Sessions a running process holds (every worker's renders and warm spares) " \
|
|
78
|
+
"are never touched",
|
|
79
|
+
default: false,
|
|
80
|
+
ask: true,
|
|
81
|
+
color: :green
|
|
82
|
+
},
|
|
83
|
+
{
|
|
84
|
+
name: :scope,
|
|
85
|
+
desc: "recorded: only sessions bidi2pdf recorded in the registry; all: every session on the " \
|
|
86
|
+
"chromedriver - only for a chromedriver your application owns",
|
|
87
|
+
default: "recorded",
|
|
88
|
+
limited_to: %w[recorded all],
|
|
89
|
+
ask: true,
|
|
90
|
+
color: :yellow
|
|
91
|
+
},
|
|
92
|
+
{ name: :orphan_age, desc: "Close sessions nobody holds that are older than this many seconds; nil turns the rule off", default: 600, ask: true, color: :yellow },
|
|
93
|
+
{ name: :min_age, desc: "Never close a session younger than this many seconds", default: 60, ask: true, color: :yellow },
|
|
94
|
+
{ name: :unresponsive_checks, desc: "Close a session nobody holds after this many failed checks in a row; nil turns the rule off", default: 2, ask: false, color: :yellow },
|
|
95
|
+
{
|
|
96
|
+
name: :max_sessions,
|
|
97
|
+
desc: "Close the oldest sessions nobody holds while more than this many exist; \"auto\" derives it " \
|
|
98
|
+
"from pids_limit, nil turns the limit off",
|
|
99
|
+
default: nil,
|
|
100
|
+
ask: true,
|
|
101
|
+
color: :yellow
|
|
102
|
+
},
|
|
103
|
+
{ name: :pids_limit, desc: "The chromedriver container's pids limit, for max_sessions = \"auto\"", default: nil, ask: true, color: :yellow },
|
|
104
|
+
{ name: :interval, desc: "Seconds between background sweeps in every process; nil sweeps only when a render fails or on demand", default: nil, ask: true, color: :yellow },
|
|
105
|
+
{
|
|
106
|
+
name: :registry_dir,
|
|
107
|
+
desc: "Directory of the session registry; every process that renders against the same chromedriver " \
|
|
108
|
+
"(web and job containers) must share it, nil means Dir.tmpdir",
|
|
109
|
+
default: nil,
|
|
110
|
+
ask: false,
|
|
111
|
+
color: :yellow
|
|
112
|
+
},
|
|
113
|
+
{ name: :retry_on_failure, desc: "Retry a render once after a last-resort sweep when it failed for lack of resources", default: true, ask: true, color: :green }
|
|
114
|
+
]
|
|
115
|
+
},
|
|
116
|
+
|
|
69
117
|
proxy_settings: {
|
|
70
118
|
name: "Proxy Settings",
|
|
71
119
|
ask: "Use a proxy server? (y/n)",
|
|
@@ -39,6 +39,10 @@ module Bidi2pdfRails
|
|
|
39
39
|
#
|
|
40
40
|
#
|
|
41
41
|
class Railtie < ::Rails::Railtie
|
|
42
|
+
rake_tasks do
|
|
43
|
+
load File.expand_path("tasks/sessions.rake", __dir__)
|
|
44
|
+
end
|
|
45
|
+
|
|
42
46
|
initializer "bidi2pdf_rails.add_mime_type" do
|
|
43
47
|
Mime::Type.register "application/pdf", :pdf unless Mime::Type.lookup_by_extension(:pdf)
|
|
44
48
|
end
|
|
@@ -3,7 +3,16 @@
|
|
|
3
3
|
module Bidi2pdfRails
|
|
4
4
|
module Services
|
|
5
5
|
module PdfBrowserSession
|
|
6
|
+
# With sweeper_settings.enabled, a render that fails for lack of resources (chromedriver
|
|
7
|
+
# refused the session, Chrome died or stopped answering) is run once more after a last-resort
|
|
8
|
+
# sweep - see ChromeSweeping.with_retry. Each run opens and cleans up its own tab and session.
|
|
6
9
|
def run_browser_session
|
|
10
|
+
ChromeSweeping.with_retry { run_browser_session_once }
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
private
|
|
14
|
+
|
|
15
|
+
def run_browser_session_once
|
|
7
16
|
future = Concurrent::Promises.future do
|
|
8
17
|
Rails.application.executor.wrap do
|
|
9
18
|
if ChromedriverManagerSingleton.session_warmer_active?
|
|
@@ -17,28 +26,27 @@ module Bidi2pdfRails
|
|
|
17
26
|
future.value!
|
|
18
27
|
end
|
|
19
28
|
|
|
20
|
-
private
|
|
21
|
-
|
|
22
29
|
# Bidi2pdf::SessionWarmer#with_tab already checks out, closes the tab/window/user_context, and
|
|
23
30
|
# retires the underlying Chrome session on its own - nothing left for this method to clean up.
|
|
24
31
|
def run_with_session_warmer
|
|
25
32
|
Bidi2pdf::SessionWarmer.with_tab { |tab| print_and_notify(tab) }
|
|
26
33
|
end
|
|
27
34
|
|
|
35
|
+
# The session is covered by the ensure too: when its browser cannot be created, it is still
|
|
36
|
+
# closed - a half-started session left in the pooled thread's slot would be reused broken, by
|
|
37
|
+
# the retry among others.
|
|
28
38
|
def run_with_chromedriver_manager_singleton
|
|
29
39
|
browser = ChromedriverManagerSingleton.session.browser
|
|
30
40
|
context = browser.create_user_context
|
|
31
41
|
window = context.create_browser_window
|
|
32
42
|
tab = window.create_browser_tab
|
|
33
43
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
ChromedriverManagerSingleton.session_close
|
|
41
|
-
end
|
|
44
|
+
print_and_notify(tab)
|
|
45
|
+
ensure
|
|
46
|
+
tab&.close
|
|
47
|
+
window&.close
|
|
48
|
+
context&.close
|
|
49
|
+
ChromedriverManagerSingleton.session_close
|
|
42
50
|
end
|
|
43
51
|
|
|
44
52
|
def print_and_notify(tab)
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
namespace :bidi2pdf_rails do
|
|
4
|
+
desc "List the Chrome sessions the remote browser holds (id, age, live or not - no page content)"
|
|
5
|
+
task sessions: :environment do
|
|
6
|
+
sessions = Bidi2pdfRails.chrome_sessions
|
|
7
|
+
puts "no sessions" if sessions.empty?
|
|
8
|
+
sessions.each do |info|
|
|
9
|
+
state = if info.live then "live"
|
|
10
|
+
elsif info.responsive then "responsive"
|
|
11
|
+
else "unresponsive"
|
|
12
|
+
end
|
|
13
|
+
puts format("%-34<id>s %8<age>s %<state>s", id: info.id, age: info.age ? "#{info.age.round}s" : "?", state: state)
|
|
14
|
+
end
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
desc "Close leaked Chrome sessions on the remote browser (PRESSURE=1: last resort, DRY_RUN=1: report only, " \
|
|
18
|
+
"CHECK_INTERVAL=10: check twice first)"
|
|
19
|
+
task sweep: :environment do
|
|
20
|
+
check_interval = ENV["CHECK_INTERVAL"].presence&.to_f
|
|
21
|
+
result = Bidi2pdfRails.sweep_sessions!(pressure: ENV["PRESSURE"] == "1", dry_run: ENV["DRY_RUN"] == "1",
|
|
22
|
+
check_interval: check_interval)
|
|
23
|
+
verb = result.dry_run ? "would close" : "closed"
|
|
24
|
+
result.closed.each { |closed| puts "#{verb} #{closed.id} (#{closed.why}, #{closed.age}s old)" }
|
|
25
|
+
puts "#{verb} #{result.closed.size} of #{result.sessions} session(s)"
|
|
26
|
+
puts "limit #{result.limit} still exceeded" if result.limit_exceeded
|
|
27
|
+
result.errors.each { |error| warn "error: #{error}" }
|
|
28
|
+
exit(1) if result.errors.any? || result.limit_exceeded
|
|
29
|
+
end
|
|
30
|
+
end
|
|
@@ -8,7 +8,7 @@ module Bidi2pdfRails
|
|
|
8
8
|
# @param [Symbol] key the setting key to override
|
|
9
9
|
# @param [Object] value the new value to set
|
|
10
10
|
def with_render_setting(key, value)
|
|
11
|
-
overridden_render_settings[key] = Bidi2pdfRails.config.render_remote_settings.public_send(key)
|
|
11
|
+
overridden_render_settings[key] = Bidi2pdfRails.config.render_remote_settings.public_send(key) unless overridden_render_settings.key?(key)
|
|
12
12
|
Bidi2pdfRails.config.render_remote_settings.public_send("#{key}=", value)
|
|
13
13
|
end
|
|
14
14
|
|
|
@@ -17,7 +17,7 @@ module Bidi2pdfRails
|
|
|
17
17
|
# @param [Symbol] key the setting key to override
|
|
18
18
|
# @param [Object] value the new value to set
|
|
19
19
|
def with_pdf_settings(key, value)
|
|
20
|
-
overridden_pdf_settings[key] = Bidi2pdfRails.config.pdf_settings.public_send(key)
|
|
20
|
+
overridden_pdf_settings[key] = Bidi2pdfRails.config.pdf_settings.public_send(key) unless overridden_pdf_settings.key?(key)
|
|
21
21
|
Bidi2pdfRails.config.pdf_settings.public_send("#{key}=", value)
|
|
22
22
|
end
|
|
23
23
|
|
|
@@ -26,7 +26,7 @@ module Bidi2pdfRails
|
|
|
26
26
|
# @param [Symbol] key the setting key to override
|
|
27
27
|
# @param [Object] value the new value to set
|
|
28
28
|
def with_lifecycle_settings(key, value)
|
|
29
|
-
overridden_lifecycle_settings[key] = Bidi2pdfRails.config.lifecycle_settings.public_send(key)
|
|
29
|
+
overridden_lifecycle_settings[key] = Bidi2pdfRails.config.lifecycle_settings.public_send(key) unless overridden_lifecycle_settings.key?(key)
|
|
30
30
|
Bidi2pdfRails.config.lifecycle_settings.public_send("#{key}=", value)
|
|
31
31
|
end
|
|
32
32
|
|
|
@@ -35,7 +35,7 @@ module Bidi2pdfRails
|
|
|
35
35
|
# @param [Symbol] key the setting key to override
|
|
36
36
|
# @param [Object] value the new value to set
|
|
37
37
|
def with_chromedriver_settings(key, value)
|
|
38
|
-
overridden_chromedriver_settings[key] = Bidi2pdfRails.config.chromedriver_settings.public_send(key)
|
|
38
|
+
overridden_chromedriver_settings[key] = Bidi2pdfRails.config.chromedriver_settings.public_send(key) unless overridden_chromedriver_settings.key?(key)
|
|
39
39
|
Bidi2pdfRails.config.chromedriver_settings.public_send("#{key}=", value)
|
|
40
40
|
end
|
|
41
41
|
|
|
@@ -44,7 +44,7 @@ module Bidi2pdfRails
|
|
|
44
44
|
# @param [Symbol] key the setting key to override
|
|
45
45
|
# @param [Object] value the new value to set
|
|
46
46
|
def with_proxy_settings(key, value)
|
|
47
|
-
overridden_proxy_settings[key] = Bidi2pdfRails.config.proxy_settings.public_send(key)
|
|
47
|
+
overridden_proxy_settings[key] = Bidi2pdfRails.config.proxy_settings.public_send(key) unless overridden_proxy_settings.key?(key)
|
|
48
48
|
Bidi2pdfRails.config.proxy_settings.public_send("#{key}=", value)
|
|
49
49
|
end
|
|
50
50
|
|
|
@@ -89,6 +89,29 @@ module Bidi2pdfRails
|
|
|
89
89
|
@__overridden_proxy_settings ||= {}
|
|
90
90
|
end
|
|
91
91
|
|
|
92
|
+
# Temporarily overrides a sweeper setting with a new value; the first override of a key keeps
|
|
93
|
+
# the original for #reset_sweeper_settings.
|
|
94
|
+
# @param [Symbol] key the setting key to override
|
|
95
|
+
# @param [Object] value the new value to set
|
|
96
|
+
def with_sweeper_settings(key, value)
|
|
97
|
+
overridden_sweeper_settings[key] = Bidi2pdfRails.config.sweeper_settings.public_send(key) unless overridden_sweeper_settings.key?(key)
|
|
98
|
+
Bidi2pdfRails.config.sweeper_settings.public_send("#{key}=", value)
|
|
99
|
+
end
|
|
100
|
+
|
|
101
|
+
# @return [Hash] the overridden sweeper settings with their original values
|
|
102
|
+
def overridden_sweeper_settings
|
|
103
|
+
@__overridden_sweeper_settings ||= {}
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
# Resets every overridden sweeper setting to its original value.
|
|
107
|
+
def reset_sweeper_settings
|
|
108
|
+
overridden_sweeper_settings.each do |key, original_value|
|
|
109
|
+
Bidi2pdfRails.config.sweeper_settings.public_send("#{key}=", original_value)
|
|
110
|
+
end
|
|
111
|
+
|
|
112
|
+
@__overridden_sweeper_settings = {}
|
|
113
|
+
end
|
|
114
|
+
|
|
92
115
|
# Retrieves the hash of overridden session warmer settings.
|
|
93
116
|
# @return [Hash] a hash of overridden session warmer settings
|
|
94
117
|
def overridden_session_warmer_settings
|
|
@@ -168,6 +191,7 @@ module Bidi2pdfRails
|
|
|
168
191
|
reset_chromedriver_settings if respond_to?(:reset_chromedriver_settings)
|
|
169
192
|
reset_proxy_settings if respond_to?(:reset_proxy_settings)
|
|
170
193
|
reset_session_warmer_settings if respond_to?(:reset_session_warmer_settings)
|
|
194
|
+
reset_sweeper_settings if respond_to?(:reset_sweeper_settings)
|
|
171
195
|
end
|
|
172
196
|
end
|
|
173
197
|
end
|