bidi2pdf 0.1.16 → 0.1.17
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +26 -2
- data/README.md +37 -0
- data/docker/Dockerfile +7 -0
- data/docker/Dockerfile.chromedriver +7 -0
- data/docker/Dockerfile.slim +7 -0
- data/lib/bidi2pdf/bidi/buffered_web_socket_client.rb +9 -0
- data/lib/bidi2pdf/bidi/session.rb +5 -1
- data/lib/bidi2pdf/session_registry.rb +97 -0
- data/lib/bidi2pdf/session_sweeper.rb +64 -0
- data/lib/bidi2pdf/session_warmer.rb +51 -1
- data/lib/bidi2pdf/version.rb +1 -1
- data/lib/bidi2pdf.rb +2 -0
- data/sig/bidi2pdf/bidi/buffered_web_socket_client.rbs +7 -0
- data/sig/bidi2pdf/bidi/session.rbs +6 -0
- data/sig/bidi2pdf/session_registry.rbs +44 -0
- data/sig/bidi2pdf/session_sweeper.rbs +36 -0
- data/sig/bidi2pdf/session_warmer.rbs +33 -0
- data/sig/bidi2pdf/version.rbs +1 -1
- metadata +13 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: b879e9c84ccd1584f5b2eb1b406972624cd8b64a50c92466e78f7275ef229441
|
|
4
|
+
data.tar.gz: 50d517dac875995c2a06ca39dcd8c521c0b18fd238b6950d5e0d9674187941dd
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 7fca9af905a088093ff4c8ac3a1c78b4f6138068c27ad4cf87616c7ded67f44b3d57aab73b40d83b6c9d8d63097f2851b10f5b0531a91bc22088ec15aab21769
|
|
7
|
+
data.tar.gz: 835727303208ada398c0ec7456e29d49201d2b798a34edac35007c40ae5123e2a382a6e325e978237fd328ef0d58e6f573d61b1447df54bae51f9188775ab4c6
|
data/CHANGELOG.md
CHANGED
|
@@ -8,10 +8,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
-
[unreleased]: https://github.com/dieter-medium/bidi2pdf/compare/v0.1.
|
|
11
|
+
[unreleased]: https://github.com/dieter-medium/bidi2pdf/compare/v0.1.17..HEAD
|
|
12
12
|
|
|
13
13
|
<!-- generated by git-cliff end -->
|
|
14
14
|
|
|
15
|
+
## [0.1.17] - 2026-09-29
|
|
16
|
+
|
|
17
|
+
### 🐛 Fixed
|
|
18
|
+
- Let Chromium start on a read-only root
|
|
19
|
+
- Do not raise when our own close ends a write
|
|
20
|
+
|
|
21
|
+
### 📝 Docs
|
|
22
|
+
- Describe latest as the newest Chromium build
|
|
23
|
+
|
|
24
|
+
### 🔄 Changed
|
|
25
|
+
- Merge pull request #143 from dieter-medium/feat/chromedriver-sha-tags
|
|
26
|
+
- Merge pull request #140 from dieter-medium/fix/chromium-read-only-root
|
|
27
|
+
- Merge pull request #139 from dieter-medium/fix/websocket-close-during-write
|
|
28
|
+
- Merge pull request #138 from dieter-medium/feat/session-warmer-orphan-sweep
|
|
29
|
+
- Merge pull request #132 from dieter-medium/dependabot/bundler/main/rubyzip-3.6.0
|
|
30
|
+
|
|
31
|
+
### 🔧 Build
|
|
32
|
+
- Update rubyzip requirement from ~> 2.4 to >= 2.4, < 4.0
|
|
33
|
+
|
|
34
|
+
### 🚀 Added
|
|
35
|
+
- Tag every chromedriver image build by commit
|
|
36
|
+
- Close leftover warm sessions on start
|
|
37
|
+
|
|
15
38
|
## [0.1.16] - 2026-09-22
|
|
16
39
|
|
|
17
40
|
### 🐛 Fixed
|
|
@@ -472,7 +495,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
472
495
|
|
|
473
496
|
### 🔄 Released
|
|
474
497
|
|
|
475
|
-
- [unreleased](https://github.com/dieter-medium/bidi2pdf/compare/v0.1.
|
|
498
|
+
- [unreleased](https://github.com/dieter-medium/bidi2pdf/compare/v0.1.17..HEAD)
|
|
499
|
+
- [0.1.17](https://github.com/dieter-medium/bidi2pdf/compare/v0.1.16..v0.1.17)
|
|
476
500
|
- [0.1.16](https://github.com/dieter-medium/bidi2pdf/compare/v0.1.15..v0.1.16)
|
|
477
501
|
- [0.1.15](https://github.com/dieter-medium/bidi2pdf/compare/v0.1.14..v0.1.15)
|
|
478
502
|
- [0.1.14](https://github.com/dieter-medium/bidi2pdf/compare/v0.1.13..v0.1.14)
|
data/README.md
CHANGED
|
@@ -435,11 +435,30 @@ docker run -it --rm \
|
|
|
435
435
|
|
|
436
436
|
✅ Tip: Mount your local directory (e.g. ./output) to /reports in the container to easily access the generated PDFs.
|
|
437
437
|
|
|
438
|
+
✅ All images run with a read-only root filesystem (`--read-only`) as long as `/tmp` is writable,
|
|
439
|
+
e.g. `--tmpfs /tmp`: they point Chromium's `XDG_CONFIG_HOME`/`XDG_CACHE_HOME` there, without
|
|
440
|
+
which Chromium's crash handler fails to start and every launch aborts with
|
|
441
|
+
`chrome_crashpad_handler: --database is required`.
|
|
442
|
+
|
|
438
443
|
✅ Both published images also install [`pdf-reader`](https://github.com/yob/pdf-reader) - not a
|
|
439
444
|
runtime dependency of the gem itself (see [Agent and Automation Usage](#agent-and-automation-usage)) -
|
|
440
445
|
so `pages` and the `page_count`/`pdf_text_present`/`pdf_not_blank` recipe assertions work out of
|
|
441
446
|
the box in either image, no extra install step needed.
|
|
442
447
|
|
|
448
|
+
### ChromeDriver image tags
|
|
449
|
+
|
|
450
|
+
[`dieters877565/chromedriver`](https://hub.docker.com/r/dieters877565/chromedriver) is built for
|
|
451
|
+
`linux/amd64` and `linux/arm64` with these tags:
|
|
452
|
+
|
|
453
|
+
| Tag | Moves? | Use |
|
|
454
|
+
|---|---|---|
|
|
455
|
+
| `0.1.17` (a release) | no | a fixed Chromium, pinned together with the gem version |
|
|
456
|
+
| `sha-<short commit>` | only if that commit is built again by hand | a fix on `main` not released yet |
|
|
457
|
+
| `latest`, `main` | yes, every push to `main` | the newest build and its Chromium security fixes |
|
|
458
|
+
|
|
459
|
+
Chromium comes from Debian's packages at build time, so every build can carry a different
|
|
460
|
+
Chromium - for a byte-exact pin use the digest (`docker buildx imagetools inspect <image:tag>`).
|
|
461
|
+
|
|
443
462
|
### Docker Compose
|
|
444
463
|
|
|
445
464
|
```bash
|
|
@@ -551,6 +570,24 @@ Bidi2pdf::SessionWarmer.shutdown
|
|
|
551
570
|
| `headless` | `true` | Run Chrome headless. |
|
|
552
571
|
| `chrome_args` | `DEFAULT_CHROME_ARGS` | Chrome launch arguments. |
|
|
553
572
|
| `remote_browser_url` | `nil` | Connect each slot to a remote chromedriver instead of starting a local one. |
|
|
573
|
+
| `orphan_age` | `:auto` | Remote only: on start, close sessions other warmers left behind older than this (`:auto` = 2 × `max_idle_age`, `nil` = off). |
|
|
574
|
+
| `registry_dir` | `Dir.tmpdir` | Where the session registry file lives - every process that should clean up after the others must share it. |
|
|
575
|
+
|
|
576
|
+
#### Leftover sessions on a shared chromedriver
|
|
577
|
+
|
|
578
|
+
A remote chromedriver keeps a session - a whole Chrome - until someone deletes it. A warmer closes
|
|
579
|
+
its own sessions when they idle past `max_idle_age` and when the process shuts down cleanly, but a
|
|
580
|
+
process that is killed or crashes leaves its sessions open, and enough of them stop the container
|
|
581
|
+
from starting any new Chrome. So in remote mode each warmer records the sessions it opens in a small
|
|
582
|
+
registry file (`<registry_dir>/bidi2pdf-sessions-<hash of the URL>.json`, mode 0600), and on start
|
|
583
|
+
closes recorded sessions older than `orphan_age`. The default, twice `max_idle_age`, is beyond the
|
|
584
|
+
point where any live warmer would already have recycled its own spare, so a running process never
|
|
585
|
+
loses one. Sessions nobody recorded (other tools on the same chromedriver) are never touched.
|
|
586
|
+
chromedriver drops custom capabilities, so a session cannot carry a tag of its own - hence the file.
|
|
587
|
+
|
|
588
|
+
Everything here is fail-open: if the registry directory is not writable, the warmer logs one
|
|
589
|
+
warning, instruments `session_warmer.registry_unavailable.bidi2pdf`, and keeps rendering - only the
|
|
590
|
+
cleanup is off. Closed leftovers are reported as `session_warmer.orphans_closed.bidi2pdf`.
|
|
554
591
|
|
|
555
592
|
> **Security note:** an idle warm session is an open, unauthenticated automation endpoint
|
|
556
593
|
> (chromedriver's port, Chrome's debugging port - loopback only for a local chromedriver) for as
|
data/docker/Dockerfile
CHANGED
|
@@ -42,6 +42,13 @@ RUN gem install ./bidi2pdf-*.gem && \
|
|
|
42
42
|
chown -R appuser:appuser /app
|
|
43
43
|
|
|
44
44
|
# Switch to non-root user
|
|
45
|
+
# Chromium's crash handler gets its database path from the XDG config dir. On a read-only root
|
|
46
|
+
# (docker run --read-only) that dir cannot be created, the handler refuses to start
|
|
47
|
+
# ("--database is required") and Chrome aborts on every launch. /tmp is the one place such a
|
|
48
|
+
# container is expected to mount writable (--tmpfs /tmp).
|
|
49
|
+
ENV XDG_CONFIG_HOME=/tmp/xdg-config \
|
|
50
|
+
XDG_CACHE_HOME=/tmp/xdg-cache
|
|
51
|
+
|
|
45
52
|
USER appuser
|
|
46
53
|
|
|
47
54
|
CMD ["/usr/bin/bash"]
|
|
@@ -53,6 +53,13 @@ WORKDIR /app
|
|
|
53
53
|
RUN mkdir -p /tmp/.X11-unix && chmod 1777 /tmp/.X11-unix
|
|
54
54
|
|
|
55
55
|
# Switch to non-root user
|
|
56
|
+
# Chromium's crash handler gets its database path from the XDG config dir. On a read-only root
|
|
57
|
+
# (docker run --read-only) that dir cannot be created, the handler refuses to start
|
|
58
|
+
# ("--database is required") and Chrome aborts on every launch. /tmp is the one place such a
|
|
59
|
+
# container is expected to mount writable (--tmpfs /tmp).
|
|
60
|
+
ENV XDG_CONFIG_HOME=/tmp/xdg-config \
|
|
61
|
+
XDG_CACHE_HOME=/tmp/xdg-cache
|
|
62
|
+
|
|
56
63
|
USER appuser
|
|
57
64
|
|
|
58
65
|
# RUN gem install chromedriver-binary && ruby -e 'require "chromedriver/binary"; puts Chromedriver::Binary::ChromedriverDownloader.update'
|
data/docker/Dockerfile.slim
CHANGED
|
@@ -73,6 +73,13 @@ WORKDIR /app
|
|
|
73
73
|
RUN chown -R appuser:appuser /app
|
|
74
74
|
|
|
75
75
|
# Switch to non-root user
|
|
76
|
+
# Chromium's crash handler gets its database path from the XDG config dir. On a read-only root
|
|
77
|
+
# (docker run --read-only) that dir cannot be created, the handler refuses to start
|
|
78
|
+
# ("--database is required") and Chrome aborts on every launch. /tmp is the one place such a
|
|
79
|
+
# container is expected to mount writable (--tmpfs /tmp).
|
|
80
|
+
ENV XDG_CONFIG_HOME=/tmp/xdg-config \
|
|
81
|
+
XDG_CACHE_HOME=/tmp/xdg-cache
|
|
82
|
+
|
|
76
83
|
USER appuser
|
|
77
84
|
|
|
78
85
|
CMD ["/usr/bin/bash"]
|
|
@@ -92,8 +92,17 @@ module Bidi2pdf
|
|
|
92
92
|
@listeners_mutex.synchronize { @listeners[event].dup }.each { |listener| listener.call(*) }
|
|
93
93
|
end
|
|
94
94
|
|
|
95
|
+
# The reader closes the socket from its own thread when the peer hangs up, and #close does not
|
|
96
|
+
# wait for a write in flight - so a write can fail with "stream closed in another thread" (or
|
|
97
|
+
# EPIPE) because *we* closed it. By then the connection is over and :close has been emitted;
|
|
98
|
+
# there is nothing left to report. Seen in the handshake write of #connect, which had no
|
|
99
|
+
# rescue: a server that answered and hung up at once made #connect raise although the
|
|
100
|
+
# handshake went through. Any other write failure still raises (#send and #say_goodbye
|
|
101
|
+
# handle it).
|
|
95
102
|
def write(bytes)
|
|
96
103
|
@write_mutex.synchronize { @socket&.write bytes }
|
|
104
|
+
rescue IOError, SystemCallError, OpenSSL::SSL::SSLError
|
|
105
|
+
raise unless @closed
|
|
97
106
|
end
|
|
98
107
|
|
|
99
108
|
def say_goodbye
|
|
@@ -74,6 +74,10 @@ module Bidi2pdf
|
|
|
74
74
|
# @return [Array<String>] The Chrome arguments for the session.
|
|
75
75
|
attr_reader :chrome_args
|
|
76
76
|
|
|
77
|
+
# @return [String, nil] chromedriver's id for this session, once it was created - what
|
|
78
|
+
# SessionWarmer records so a later process can close it if this one dies uncleanly.
|
|
79
|
+
attr_reader :session_id
|
|
80
|
+
|
|
77
81
|
# Initializes a new session.
|
|
78
82
|
#
|
|
79
83
|
# @param [String] session_url The URL for the session.
|
|
@@ -227,7 +231,7 @@ module Bidi2pdf
|
|
|
227
231
|
value = session_data["value"]
|
|
228
232
|
handle_error(value) if value.nil? || value["error"]
|
|
229
233
|
|
|
230
|
-
session_id = value["sessionId"]
|
|
234
|
+
@session_id = value["sessionId"]
|
|
231
235
|
ws_url = value["capabilities"]["webSocketUrl"]
|
|
232
236
|
|
|
233
237
|
Bidi2pdf.logger.info "Created session with ID: #{session_id}"
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "digest"
|
|
4
|
+
require "json"
|
|
5
|
+
require "tmpdir"
|
|
6
|
+
|
|
7
|
+
module Bidi2pdf
|
|
8
|
+
# Remembers which sessions a SessionWarmer opened on a shared (remote) chromedriver, and when, in a
|
|
9
|
+
# small JSON file per chromedriver URL - so a process started later can find and close the ones a
|
|
10
|
+
# process that died without a clean shutdown left behind (see SessionSweeper). chromedriver itself
|
|
11
|
+
# keeps a session until someone deletes it, and it drops any tag we could attach to one (a custom
|
|
12
|
+
# capability is accepted but echoed nowhere - checked against chromedriver 153), hence a file.
|
|
13
|
+
#
|
|
14
|
+
# Everything here is best effort and fail-open: a directory that is not writable, a locked-out or
|
|
15
|
+
# corrupt file only switches orphan tracking off for this process (one warning, one
|
|
16
|
+
# +session_warmer.registry_unavailable.bidi2pdf+ notification) - rendering never depends on it.
|
|
17
|
+
# The file is created 0600 and every change happens under an exclusive lock, so processes sharing
|
|
18
|
+
# the directory (a dev server, a job worker, a spec run) can record at the same time.
|
|
19
|
+
class SessionRegistry
|
|
20
|
+
FILE_PREFIX = "bidi2pdf-sessions-"
|
|
21
|
+
|
|
22
|
+
attr_reader :path
|
|
23
|
+
|
|
24
|
+
def initialize(session_url, dir: nil)
|
|
25
|
+
@path = File.join(dir || Dir.tmpdir, "#{FILE_PREFIX}#{Digest::SHA256.hexdigest(session_url.to_s)[0, 16]}.json")
|
|
26
|
+
@warned = false
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
# Records a session the caller just opened.
|
|
30
|
+
def record(session_id, created_at: Time.now.to_i)
|
|
31
|
+
return false if session_id.nil?
|
|
32
|
+
|
|
33
|
+
update { |entries| entries[session_id.to_s] = created_at.to_i }
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
# Forgets a session the caller closed itself (or found gone).
|
|
37
|
+
def forget(session_id)
|
|
38
|
+
return false if session_id.nil?
|
|
39
|
+
|
|
40
|
+
update { |entries| entries.delete(session_id.to_s) }
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
# The recorded session ids opened at or before +cutoff+ (epoch seconds).
|
|
44
|
+
def recorded_before(cutoff)
|
|
45
|
+
entries = read
|
|
46
|
+
entries.select { |_, created_at| created_at <= cutoff }.keys
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
private
|
|
50
|
+
|
|
51
|
+
def read
|
|
52
|
+
return {} unless File.exist?(path)
|
|
53
|
+
|
|
54
|
+
File.open(path, File::RDONLY) do |file|
|
|
55
|
+
file.flock(File::LOCK_SH)
|
|
56
|
+
parse(file.read)
|
|
57
|
+
end
|
|
58
|
+
rescue SystemCallError, IOError => e
|
|
59
|
+
unavailable!(e)
|
|
60
|
+
{}
|
|
61
|
+
end
|
|
62
|
+
|
|
63
|
+
def update
|
|
64
|
+
File.open(path, File::RDWR | File::CREAT, 0o600) do |file|
|
|
65
|
+
file.flock(File::LOCK_EX)
|
|
66
|
+
entries = parse(file.read)
|
|
67
|
+
yield entries
|
|
68
|
+
file.rewind
|
|
69
|
+
file.truncate(0)
|
|
70
|
+
file.write(JSON.generate(entries))
|
|
71
|
+
file.flush
|
|
72
|
+
end
|
|
73
|
+
true
|
|
74
|
+
rescue SystemCallError, IOError => e
|
|
75
|
+
unavailable!(e)
|
|
76
|
+
false
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
# A corrupt or foreign file counts as empty rather than as an error.
|
|
80
|
+
def parse(text)
|
|
81
|
+
data = text.to_s.empty? ? {} : JSON.parse(text)
|
|
82
|
+
return {} unless data.is_a?(Hash)
|
|
83
|
+
|
|
84
|
+
data.select { |id, created_at| id.is_a?(String) && created_at.is_a?(Integer) }
|
|
85
|
+
rescue JSON::ParserError
|
|
86
|
+
{}
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
def unavailable!(error)
|
|
90
|
+
return if @warned
|
|
91
|
+
|
|
92
|
+
@warned = true
|
|
93
|
+
Bidi2pdf.logger.warn "session_warmer: session registry #{path} unavailable, leftover sessions will not be tracked: #{error.message}"
|
|
94
|
+
Bidi2pdf.notification_service.instrument("session_warmer.registry_unavailable.bidi2pdf", { path: path, error: error.class.name })
|
|
95
|
+
end
|
|
96
|
+
end
|
|
97
|
+
end
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "net/http"
|
|
4
|
+
require "uri"
|
|
5
|
+
|
|
6
|
+
module Bidi2pdf
|
|
7
|
+
# Closes sessions a SessionWarmer recorded (SessionRegistry) on a shared chromedriver that are
|
|
8
|
+
# older than a cutoff - left behind by a process that died without a clean shutdown, and holding a
|
|
9
|
+
# whole Chrome each until someone deletes them. Run once when a warmer starts.
|
|
10
|
+
#
|
|
11
|
+
# Only recorded sessions are ever touched - another tool's sessions on the same chromedriver never
|
|
12
|
+
# are. A recorded session that is already gone ("invalid session id" / 404) is just forgotten.
|
|
13
|
+
# Fail-open: any error is logged and ends the sweep; it never raises.
|
|
14
|
+
class SessionSweeper
|
|
15
|
+
# @param session_url [String] the chromedriver's new-session URL (".../session").
|
|
16
|
+
# @param registry [SessionRegistry]
|
|
17
|
+
# @param http [#call, nil] (method, url) -> status code; injectable for tests.
|
|
18
|
+
def initialize(session_url, registry, http: nil)
|
|
19
|
+
@session_url = session_url.to_s.chomp("/")
|
|
20
|
+
@registry = registry
|
|
21
|
+
@http = http || method(:net_http)
|
|
22
|
+
end
|
|
23
|
+
|
|
24
|
+
# Closes recorded sessions opened more than +older_than+ seconds before +now+.
|
|
25
|
+
#
|
|
26
|
+
# @return [Integer] how many sessions were actually closed.
|
|
27
|
+
def sweep(older_than:, now: Time.now.to_i)
|
|
28
|
+
closed = @registry.recorded_before(now - older_than).count { |id| closed_now?(id) }
|
|
29
|
+
report(closed)
|
|
30
|
+
closed
|
|
31
|
+
rescue StandardError => e
|
|
32
|
+
Bidi2pdf.logger.warn "session_warmer: sweeping leftover sessions failed: #{e.message}"
|
|
33
|
+
Bidi2pdf.notification_service.instrument("session_warmer.sweep_failed.bidi2pdf", { error: e.class.name })
|
|
34
|
+
0
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
private
|
|
38
|
+
|
|
39
|
+
# True when this call closed the session; a session already gone is forgotten but not counted.
|
|
40
|
+
def closed_now?(id)
|
|
41
|
+
status = @http.call(:delete, "#{@session_url}/#{id}")
|
|
42
|
+
return false unless [200, 404].include?(status)
|
|
43
|
+
|
|
44
|
+
@registry.forget(id)
|
|
45
|
+
status == 200
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def report(closed)
|
|
49
|
+
return if closed.zero?
|
|
50
|
+
|
|
51
|
+
Bidi2pdf.logger.info "session_warmer: closed #{closed} leftover session(s) on #{@session_url}"
|
|
52
|
+
Bidi2pdf.notification_service.instrument("session_warmer.orphans_closed.bidi2pdf", { count: closed })
|
|
53
|
+
end
|
|
54
|
+
|
|
55
|
+
# chromedriver answers a DELETE of a session it no longer has with 404 "invalid session id".
|
|
56
|
+
def net_http(method, url)
|
|
57
|
+
uri = URI(url)
|
|
58
|
+
request = method == :delete ? Net::HTTP::Delete.new(uri) : Net::HTTP::Get.new(uri)
|
|
59
|
+
Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https", open_timeout: 5, read_timeout: 30) do |http|
|
|
60
|
+
http.request(request).code.to_i
|
|
61
|
+
end
|
|
62
|
+
end
|
|
63
|
+
end
|
|
64
|
+
end
|
|
@@ -50,6 +50,17 @@ module Bidi2pdf
|
|
|
50
50
|
# 1.25x this value even when no render ever comes. +nil+ disables the limit.
|
|
51
51
|
attr_accessor :max_idle_age
|
|
52
52
|
|
|
53
|
+
# @return [Numeric, Symbol, nil] Seconds after which a session recorded by any warmer on the
|
|
54
|
+
# same remote chromedriver counts as left behind - its process died without closing it - and
|
|
55
|
+
# is closed when a warmer starts (SessionSweeper). +:auto+ (the default) is twice
|
|
56
|
+
# max_idle_age: a live warmer recycles its own spares within ~1.25x of that, so no live
|
|
57
|
+
# process's slot is ever taken. +nil+ disables the sweep. Remote mode only.
|
|
58
|
+
attr_accessor :orphan_age
|
|
59
|
+
|
|
60
|
+
# @return [String, nil] Directory for the session registry file (SessionRegistry); +nil+ means
|
|
61
|
+
# Dir.tmpdir. Every process that should clean up after the others must share it.
|
|
62
|
+
attr_accessor :registry_dir
|
|
63
|
+
|
|
53
64
|
DEFAULT_MAX_IDLE_AGE = 300
|
|
54
65
|
|
|
55
66
|
def initialize
|
|
@@ -59,12 +70,21 @@ module Bidi2pdf
|
|
|
59
70
|
@remote_browser_url = nil
|
|
60
71
|
@slot_factory = nil
|
|
61
72
|
@max_idle_age = DEFAULT_MAX_IDLE_AGE
|
|
73
|
+
@orphan_age = :auto
|
|
74
|
+
@registry_dir = nil
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
# The orphan age in seconds, or nil when the sweep is off (also when +:auto+ has no
|
|
78
|
+
# max_idle_age to derive it from).
|
|
79
|
+
def effective_orphan_age
|
|
80
|
+
orphan_age == :auto ? max_idle_age && (max_idle_age * 2) : orphan_age
|
|
62
81
|
end
|
|
63
82
|
|
|
64
83
|
# @raise [ArgumentError] if a setting can't be honored - checked once, at construction.
|
|
65
84
|
def validate!
|
|
66
85
|
validate_size!
|
|
67
86
|
validate_max_idle_age!
|
|
87
|
+
validate_orphan_age!
|
|
68
88
|
end
|
|
69
89
|
|
|
70
90
|
private
|
|
@@ -80,6 +100,12 @@ module Bidi2pdf
|
|
|
80
100
|
|
|
81
101
|
raise ArgumentError, "max_idle_age must be nil or a positive number of seconds, got #{max_idle_age.inspect}"
|
|
82
102
|
end
|
|
103
|
+
|
|
104
|
+
def validate_orphan_age!
|
|
105
|
+
return if orphan_age.nil? || orphan_age == :auto || (orphan_age.is_a?(Numeric) && orphan_age.positive?)
|
|
106
|
+
|
|
107
|
+
raise ArgumentError, "orphan_age must be :auto, nil or a positive number of seconds, got #{orphan_age.inspect}"
|
|
108
|
+
end
|
|
83
109
|
end
|
|
84
110
|
|
|
85
111
|
class << self
|
|
@@ -181,6 +207,8 @@ module Bidi2pdf
|
|
|
181
207
|
@warming = 0
|
|
182
208
|
@shutdown = false
|
|
183
209
|
@reaper_wakeup = Thread::Queue.new
|
|
210
|
+
@registry = build_registry
|
|
211
|
+
sweep_leftovers
|
|
184
212
|
prewarm
|
|
185
213
|
@reaper = Thread.new { reap_loop } if @config.max_idle_age
|
|
186
214
|
end
|
|
@@ -235,7 +263,26 @@ module Bidi2pdf
|
|
|
235
263
|
end
|
|
236
264
|
|
|
237
265
|
def create_slot
|
|
238
|
-
@slot_factory.call
|
|
266
|
+
@slot_factory.call.tap { |slot| @registry&.record(session_id_of(slot)) }
|
|
267
|
+
end
|
|
268
|
+
|
|
269
|
+
# Remote mode with an orphan age: sessions are recorded, so a later start can close the ones
|
|
270
|
+
# this process leaves behind if it dies uncleanly - and this start closes others' leftovers.
|
|
271
|
+
def build_registry
|
|
272
|
+
return unless @config.remote_browser_url && @config.effective_orphan_age
|
|
273
|
+
|
|
274
|
+
Bidi2pdf::SessionRegistry.new(@config.remote_browser_url, dir: @config.registry_dir)
|
|
275
|
+
end
|
|
276
|
+
|
|
277
|
+
def sweep_leftovers
|
|
278
|
+
return unless @registry
|
|
279
|
+
|
|
280
|
+
Bidi2pdf::SessionSweeper.new(@config.remote_browser_url, @registry).sweep(older_than: @config.effective_orphan_age)
|
|
281
|
+
end
|
|
282
|
+
|
|
283
|
+
def session_id_of(slot)
|
|
284
|
+
session = slot[:session]
|
|
285
|
+
session.session_id if session.respond_to?(:session_id)
|
|
239
286
|
end
|
|
240
287
|
|
|
241
288
|
# Session#started? is just an internal flag set once at startup - it never flips back if Chrome,
|
|
@@ -366,8 +413,11 @@ module Bidi2pdf
|
|
|
366
413
|
retire(slot) if discard
|
|
367
414
|
end
|
|
368
415
|
|
|
416
|
+
# Forgotten only after closing: a process that dies in between leaves a record whose session is
|
|
417
|
+
# already gone - the next sweep just drops it - never an open session nobody knows about.
|
|
369
418
|
def retire(slot)
|
|
370
419
|
self.class.retire_slot(session: slot[:session], manager: slot[:manager])
|
|
420
|
+
@registry&.forget(session_id_of(slot))
|
|
371
421
|
end
|
|
372
422
|
|
|
373
423
|
def safe_close(label, &)
|
data/lib/bidi2pdf/version.rb
CHANGED
data/lib/bidi2pdf.rb
CHANGED
|
@@ -10,6 +10,8 @@ require_relative "bidi2pdf/dsl"
|
|
|
10
10
|
require_relative "bidi2pdf/notifications"
|
|
11
11
|
require_relative "bidi2pdf/notifications/logging_subscriber"
|
|
12
12
|
require_relative "bidi2pdf/notifications/json_subscriber"
|
|
13
|
+
require_relative "bidi2pdf/session_registry"
|
|
14
|
+
require_relative "bidi2pdf/session_sweeper"
|
|
13
15
|
require_relative "bidi2pdf/session_warmer"
|
|
14
16
|
require_relative "bidi2pdf/verbose_logger"
|
|
15
17
|
|
|
@@ -59,6 +59,13 @@ module Bidi2pdf
|
|
|
59
59
|
|
|
60
60
|
def emit: (untyped event, *untyped) -> untyped
|
|
61
61
|
|
|
62
|
+
# The reader closes the socket from its own thread when the peer hangs up, and #close does not
|
|
63
|
+
# wait for a write in flight - so a write can fail with "stream closed in another thread" (or
|
|
64
|
+
# EPIPE) because *we* closed it. By then the connection is over and :close has been emitted;
|
|
65
|
+
# there is nothing left to report. Seen in the handshake write of #connect, which had no
|
|
66
|
+
# rescue: a server that answered and hung up at once made #connect raise although the
|
|
67
|
+
# handshake went through. Any other write failure still raises (#send and #say_goodbye
|
|
68
|
+
# handle it).
|
|
62
69
|
def write: (untyped bytes) -> untyped
|
|
63
70
|
|
|
64
71
|
def say_goodbye: () -> untyped
|
|
@@ -33,6 +33,8 @@ module Bidi2pdf
|
|
|
33
33
|
|
|
34
34
|
@websocket_url: untyped
|
|
35
35
|
|
|
36
|
+
@session_id: untyped
|
|
37
|
+
|
|
36
38
|
# Events to subscribe to during the session.
|
|
37
39
|
SUBSCRIBE_EVENTS: ::Array["script"]
|
|
38
40
|
|
|
@@ -48,6 +50,10 @@ module Bidi2pdf
|
|
|
48
50
|
# @return [Array<String>] The Chrome arguments for the session.
|
|
49
51
|
attr_reader chrome_args: untyped
|
|
50
52
|
|
|
53
|
+
# @return [String, nil] chromedriver's id for this session, once it was created - what
|
|
54
|
+
# SessionWarmer records so a later process can close it if this one dies uncleanly.
|
|
55
|
+
attr_reader session_id: untyped
|
|
56
|
+
|
|
51
57
|
# Initializes a new session.
|
|
52
58
|
#
|
|
53
59
|
# @param [String] session_url The URL for the session.
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
module Bidi2pdf
|
|
2
|
+
# Remembers which sessions a SessionWarmer opened on a shared (remote) chromedriver, and when, in a
|
|
3
|
+
# small JSON file per chromedriver URL - so a process started later can find and close the ones a
|
|
4
|
+
# process that died without a clean shutdown left behind (see SessionSweeper). chromedriver itself
|
|
5
|
+
# keeps a session until someone deletes it, and it drops any tag we could attach to one (a custom
|
|
6
|
+
# capability is accepted but echoed nowhere - checked against chromedriver 153), hence a file.
|
|
7
|
+
#
|
|
8
|
+
# Everything here is best effort and fail-open: a directory that is not writable, a locked-out or
|
|
9
|
+
# corrupt file only switches orphan tracking off for this process (one warning, one
|
|
10
|
+
# +session_warmer.registry_unavailable.bidi2pdf+ notification) - rendering never depends on it.
|
|
11
|
+
# The file is created 0600 and every change happens under an exclusive lock, so processes sharing
|
|
12
|
+
# the directory (a dev server, a job worker, a spec run) can record at the same time.
|
|
13
|
+
class SessionRegistry
|
|
14
|
+
@path: untyped
|
|
15
|
+
|
|
16
|
+
@warned: untyped
|
|
17
|
+
|
|
18
|
+
FILE_PREFIX: "bidi2pdf-sessions-"
|
|
19
|
+
|
|
20
|
+
attr_reader path: untyped
|
|
21
|
+
|
|
22
|
+
def initialize: (untyped session_url, ?dir: untyped?) -> void
|
|
23
|
+
|
|
24
|
+
# Records a session the caller just opened.
|
|
25
|
+
def record: (untyped session_id, ?created_at: untyped) -> (false | untyped)
|
|
26
|
+
|
|
27
|
+
# Forgets a session the caller closed itself (or found gone).
|
|
28
|
+
def forget: (untyped session_id) -> (false | untyped)
|
|
29
|
+
|
|
30
|
+
# The recorded session ids opened at or before +cutoff+ (epoch seconds).
|
|
31
|
+
def recorded_before: (untyped cutoff) -> untyped
|
|
32
|
+
|
|
33
|
+
private
|
|
34
|
+
|
|
35
|
+
def read: () -> untyped
|
|
36
|
+
|
|
37
|
+
def update: () { (untyped) -> untyped } -> untyped
|
|
38
|
+
|
|
39
|
+
# A corrupt or foreign file counts as empty rather than as an error.
|
|
40
|
+
def parse: (untyped text) -> untyped
|
|
41
|
+
|
|
42
|
+
def unavailable!: (untyped error) -> (nil | untyped)
|
|
43
|
+
end
|
|
44
|
+
end
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
module Bidi2pdf
|
|
2
|
+
# Closes sessions a SessionWarmer recorded (SessionRegistry) on a shared chromedriver that are
|
|
3
|
+
# older than a cutoff - left behind by a process that died without a clean shutdown, and holding a
|
|
4
|
+
# whole Chrome each until someone deletes them. Run once when a warmer starts.
|
|
5
|
+
#
|
|
6
|
+
# Only recorded sessions are ever touched - another tool's sessions on the same chromedriver never
|
|
7
|
+
# are. A recorded session that is already gone ("invalid session id" / 404) is just forgotten.
|
|
8
|
+
# Fail-open: any error is logged and ends the sweep; it never raises.
|
|
9
|
+
class SessionSweeper
|
|
10
|
+
@session_url: untyped
|
|
11
|
+
|
|
12
|
+
@registry: untyped
|
|
13
|
+
|
|
14
|
+
@http: untyped
|
|
15
|
+
|
|
16
|
+
# @param session_url [String] the chromedriver's new-session URL (".../session").
|
|
17
|
+
# @param registry [SessionRegistry]
|
|
18
|
+
# @param http [#call, nil] (method, url) -> status code; injectable for tests.
|
|
19
|
+
def initialize: (untyped session_url, untyped registry, ?http: untyped?) -> void
|
|
20
|
+
|
|
21
|
+
# Closes recorded sessions opened more than +older_than+ seconds before +now+.
|
|
22
|
+
#
|
|
23
|
+
# @return [Integer] how many sessions were actually closed.
|
|
24
|
+
def sweep: (older_than: untyped, ?now: untyped) -> untyped
|
|
25
|
+
|
|
26
|
+
private
|
|
27
|
+
|
|
28
|
+
# True when this call closed the session; a session already gone is forgotten but not counted.
|
|
29
|
+
def closed_now?: (untyped id) -> (false | untyped)
|
|
30
|
+
|
|
31
|
+
def report: (untyped closed) -> (nil | untyped)
|
|
32
|
+
|
|
33
|
+
# chromedriver answers a DELETE of a session it no longer has with 404 "invalid session id".
|
|
34
|
+
def net_http: (untyped method, untyped url) -> untyped
|
|
35
|
+
end
|
|
36
|
+
end
|
|
@@ -42,6 +42,8 @@ module Bidi2pdf
|
|
|
42
42
|
|
|
43
43
|
@reaper_wakeup: untyped
|
|
44
44
|
|
|
45
|
+
@registry: untyped
|
|
46
|
+
|
|
45
47
|
@reaper: untyped
|
|
46
48
|
|
|
47
49
|
# Configuration for the session warmer.
|
|
@@ -58,6 +60,10 @@ module Bidi2pdf
|
|
|
58
60
|
|
|
59
61
|
@max_idle_age: untyped
|
|
60
62
|
|
|
63
|
+
@orphan_age: untyped
|
|
64
|
+
|
|
65
|
+
@registry_dir: untyped
|
|
66
|
+
|
|
61
67
|
# @return [Integer] Number of Chrome slots to keep pre-warmed.
|
|
62
68
|
attr_accessor size: untyped
|
|
63
69
|
|
|
@@ -84,10 +90,25 @@ module Bidi2pdf
|
|
|
84
90
|
# 1.25x this value even when no render ever comes. +nil+ disables the limit.
|
|
85
91
|
attr_accessor max_idle_age: untyped
|
|
86
92
|
|
|
93
|
+
# @return [Numeric, Symbol, nil] Seconds after which a session recorded by any warmer on the
|
|
94
|
+
# same remote chromedriver counts as left behind - its process died without closing it - and
|
|
95
|
+
# is closed when a warmer starts (SessionSweeper). +:auto+ (the default) is twice
|
|
96
|
+
# max_idle_age: a live warmer recycles its own spares within ~1.25x of that, so no live
|
|
97
|
+
# process's slot is ever taken. +nil+ disables the sweep. Remote mode only.
|
|
98
|
+
attr_accessor orphan_age: untyped
|
|
99
|
+
|
|
100
|
+
# @return [String, nil] Directory for the session registry file (SessionRegistry); +nil+ means
|
|
101
|
+
# Dir.tmpdir. Every process that should clean up after the others must share it.
|
|
102
|
+
attr_accessor registry_dir: untyped
|
|
103
|
+
|
|
87
104
|
DEFAULT_MAX_IDLE_AGE: 300
|
|
88
105
|
|
|
89
106
|
def initialize: () -> void
|
|
90
107
|
|
|
108
|
+
# The orphan age in seconds, or nil when the sweep is off (also when +:auto+ has no
|
|
109
|
+
# max_idle_age to derive it from).
|
|
110
|
+
def effective_orphan_age: () -> untyped
|
|
111
|
+
|
|
91
112
|
# @raise [ArgumentError] if a setting can't be honored - checked once, at construction.
|
|
92
113
|
def validate!: () -> untyped
|
|
93
114
|
|
|
@@ -96,6 +117,8 @@ module Bidi2pdf
|
|
|
96
117
|
def validate_size!: () -> (nil | untyped)
|
|
97
118
|
|
|
98
119
|
def validate_max_idle_age!: () -> (nil | untyped)
|
|
120
|
+
|
|
121
|
+
def validate_orphan_age!: () -> (nil | untyped)
|
|
99
122
|
end
|
|
100
123
|
|
|
101
124
|
# Configures the warmer and eagerly (re)creates the singleton, warming config.size slots right
|
|
@@ -161,6 +184,14 @@ module Bidi2pdf
|
|
|
161
184
|
|
|
162
185
|
def create_slot: () -> untyped
|
|
163
186
|
|
|
187
|
+
# Remote mode with an orphan age: sessions are recorded, so a later start can close the ones
|
|
188
|
+
# this process leaves behind if it dies uncleanly - and this start closes others' leftovers.
|
|
189
|
+
def build_registry: () -> (nil | untyped)
|
|
190
|
+
|
|
191
|
+
def sweep_leftovers: () -> (nil | untyped)
|
|
192
|
+
|
|
193
|
+
def session_id_of: (untyped slot) -> untyped
|
|
194
|
+
|
|
164
195
|
# Session#started? is just an internal flag set once at startup - it never flips back if Chrome,
|
|
165
196
|
# ChromeDriver, or the WebSocket dies externally while a slot sits idle in the cache. The
|
|
166
197
|
# client's own #open? is kept live by the reader thread noticing a real socket error, so
|
|
@@ -214,6 +245,8 @@ module Bidi2pdf
|
|
|
214
245
|
# reservation is released here, in the same critical section as the stash.
|
|
215
246
|
def stash_or_retire: (untyped slot) -> untyped
|
|
216
247
|
|
|
248
|
+
# Forgotten only after closing: a process that dies in between leaves a record whose session is
|
|
249
|
+
# already gone - the next sweep just drops it - never an open session nobody knows about.
|
|
217
250
|
def retire: (untyped slot) -> untyped
|
|
218
251
|
|
|
219
252
|
def safe_close: (untyped label) { (?) -> untyped } -> untyped
|
data/sig/bidi2pdf/version.rbs
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: bidi2pdf
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.17
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Dieter S.
|
|
@@ -87,16 +87,22 @@ dependencies:
|
|
|
87
87
|
name: rubyzip
|
|
88
88
|
requirement: !ruby/object:Gem::Requirement
|
|
89
89
|
requirements:
|
|
90
|
-
- - "
|
|
90
|
+
- - ">="
|
|
91
91
|
- !ruby/object:Gem::Version
|
|
92
92
|
version: '2.4'
|
|
93
|
+
- - "<"
|
|
94
|
+
- !ruby/object:Gem::Version
|
|
95
|
+
version: '4.0'
|
|
93
96
|
type: :runtime
|
|
94
97
|
prerelease: false
|
|
95
98
|
version_requirements: !ruby/object:Gem::Requirement
|
|
96
99
|
requirements:
|
|
97
|
-
- - "
|
|
100
|
+
- - ">="
|
|
98
101
|
- !ruby/object:Gem::Version
|
|
99
102
|
version: '2.4'
|
|
103
|
+
- - "<"
|
|
104
|
+
- !ruby/object:Gem::Version
|
|
105
|
+
version: '4.0'
|
|
100
106
|
- !ruby/object:Gem::Dependency
|
|
101
107
|
name: sys-proctable
|
|
102
108
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -481,7 +487,9 @@ files:
|
|
|
481
487
|
- lib/bidi2pdf/result.rb
|
|
482
488
|
- lib/bidi2pdf/result_collector.rb
|
|
483
489
|
- lib/bidi2pdf/schema.rb
|
|
490
|
+
- lib/bidi2pdf/session_registry.rb
|
|
484
491
|
- lib/bidi2pdf/session_runner.rb
|
|
492
|
+
- lib/bidi2pdf/session_sweeper.rb
|
|
485
493
|
- lib/bidi2pdf/session_warmer.rb
|
|
486
494
|
- lib/bidi2pdf/test_helpers.rb
|
|
487
495
|
- lib/bidi2pdf/test_helpers/configuration.rb
|
|
@@ -579,7 +587,9 @@ files:
|
|
|
579
587
|
- sig/bidi2pdf/result.rbs
|
|
580
588
|
- sig/bidi2pdf/result_collector.rbs
|
|
581
589
|
- sig/bidi2pdf/schema.rbs
|
|
590
|
+
- sig/bidi2pdf/session_registry.rbs
|
|
582
591
|
- sig/bidi2pdf/session_runner.rbs
|
|
592
|
+
- sig/bidi2pdf/session_sweeper.rbs
|
|
583
593
|
- sig/bidi2pdf/session_warmer.rbs
|
|
584
594
|
- sig/bidi2pdf/test_helpers.rbs
|
|
585
595
|
- sig/bidi2pdf/test_helpers/configuration.rbs
|