duel-postgresql 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +77 -0
- data/LICENSE +21 -0
- data/README.md +140 -0
- data/lib/duel/postgresql/version.rb +7 -0
- data/lib/duel/postgresql.rb +415 -0
- data/sig/duel/postgresql.rbs +178 -0
- metadata +80 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: b059d32bfc73bf0bad50a8df7cb8f8aa30351001d91eb1f996d5dd66b76da775
|
|
4
|
+
data.tar.gz: e7d114e18e4f85175936566dac86065dd9c679e574901c857a8210f931b2dd5a
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: 14ace51e70cd9a0d5313e371cd22a7afab64f5b18cd866d002e6472971b68d120237a11b8f6bd49b309b3fb3e1e2f1dd9e9191747327590bdd0f805c1bc24c37
|
|
7
|
+
data.tar.gz: e5e1a8bafafafdce2180812c8efa763948065fdbe2c9819a9ab24f91a278a8472e6a979fc876cc81d231a311fe7de31aaa7b99e45d5132b43609cad170ca2d21
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/2.0.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
## [Unreleased]
|
|
9
|
+
|
|
10
|
+
## [0.1.0] - 2026-10-24
|
|
11
|
+
|
|
12
|
+
The first release. duel-postgresql is a Duel plugin, open source under
|
|
13
|
+
the MIT license, and needs Duel 0.1, Ruby 3.4 or later (CRuby, or
|
|
14
|
+
TruffleRuby 40: the pg gem is a C extension), and PostgreSQL 14 or
|
|
15
|
+
later, with `pg_stat_statements`.
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
|
|
19
|
+
- `c.use :postgresql`: a PostgreSQL server for each Duel worker,
|
|
20
|
+
on a Unix socket, its cluster copied from one made once (with Duel's
|
|
21
|
+
`w.cached`, in the project's `tmp/duel/cache`), and
|
|
22
|
+
the `postgres_queries`, `postgres_rows`, `postgres_blocks` and
|
|
23
|
+
`postgres_time` metrics, from `pg_stat_statements`, as running totals
|
|
24
|
+
(`c.metric`).
|
|
25
|
+
- `Duel::PostgreSQL.url`, `Duel::PostgreSQL.execute` and
|
|
26
|
+
`Duel::PostgreSQL.reset`, for benchmarks: `execute` and `reset` open a
|
|
27
|
+
connection of their own, from the worker's `DUEL_POSTGRESQL_URL`.
|
|
28
|
+
They're for `setup` hooks, and `once` and `untimed` blocks: while a
|
|
29
|
+
benchmark is measured (`Duel.measuring?`) they raise `Duel::Error`, as what they run
|
|
30
|
+
would be counted.
|
|
31
|
+
- Each worker's server's URL is in `DUEL_POSTGRESQL_URL`, and in
|
|
32
|
+
`DATABASE_URL` too, by default (`env:`). It names its user in its
|
|
33
|
+
query, `postgres:///duel?host=...&port=5432&user=postgres`, so that
|
|
34
|
+
Rails, which parses `DATABASE_URL` with Ruby's URI, boots with it
|
|
35
|
+
(duel-rails with `database: :postgresql`).
|
|
36
|
+
- A regression is explained by the statements that did the most of it,
|
|
37
|
+
on each version (Duel's `details`): how many times each ran an
|
|
38
|
+
iteration, for `postgres_queries`, and their shares, for the rest.
|
|
39
|
+
- `postgres_time` is in µs (`unit: "µs"`), shown as time.
|
|
40
|
+
`postgres_queries`, `postgres_rows` and `postgres_blocks` are in
|
|
41
|
+
queries, rows and blocks (`unit:`, each with its plural), so reports
|
|
42
|
+
read "1 query → 2 queries".
|
|
43
|
+
- `postgres_blocks` and `postgres_time` are `kind: :ratio` measures, and
|
|
44
|
+
`postgres_time` is left out of Duel's across-the-board check
|
|
45
|
+
(`across: false`).
|
|
46
|
+
- A query sent with the extended protocol (as Active Record sends
|
|
47
|
+
every one) inside a transaction is counted as it runs, not as the
|
|
48
|
+
connection's next statement does: before each reading, every
|
|
49
|
+
connection of the worker's left idle in a transaction has its unnamed
|
|
50
|
+
portal closed. So an iteration's last query, inside an
|
|
51
|
+
`around_iteration` hook's transaction (duel-rails's
|
|
52
|
+
`Duel::Rails::ROLLBACK`), is the iteration's, and the hook's `BEGIN`
|
|
53
|
+
and `ROLLBACK` aren't.
|
|
54
|
+
- `c.use :postgresql` returns the plugin's own connection, opened
|
|
55
|
+
in each worker (`c.in_worker`) and closed as it ends (`at_stop`). Its
|
|
56
|
+
`value` fails outside a worker. The module keeps no connection.
|
|
57
|
+
- Built on Duel's plugin API: the server is spawned and set up with
|
|
58
|
+
`Duel::Resource#spawn` and `#run`, its measures can be tuned or
|
|
59
|
+
skipped like any (`c.measure("postgres_time", limit: 1.1)`), and its tests
|
|
60
|
+
use `duel/testing`.
|
|
61
|
+
- Its metrics are costly to read (Duel's `costly`): a benchmark with an
|
|
62
|
+
`around_iteration` hook reads the server's statistics only as each
|
|
63
|
+
sample starts and stops, not twice an iteration, and what the hook
|
|
64
|
+
sends the server (a transaction's `BEGIN` and `ROLLBACK`, say) is
|
|
65
|
+
learned in each worker and subtracted.
|
|
66
|
+
- A reading of `pg_stat_statements` makes numbers only for the
|
|
67
|
+
statements that changed since the last reading, and shares the rest
|
|
68
|
+
with it, and a sample's statements are told from those that changed
|
|
69
|
+
alone. pg decodes the counts and times (a type map), so no String is
|
|
70
|
+
made for a number. Reading 2,000 statements, 10 of them changed,
|
|
71
|
+
allocates 26 objects.
|
|
72
|
+
- Mutation tested: mutant kills every change to the plugin's code, in
|
|
73
|
+
CI.
|
|
74
|
+
- RBS signatures in sig/, checked with Steep.
|
|
75
|
+
|
|
76
|
+
[Unreleased]: https://github.com/sferik/duel-ruby/compare/duel-postgresql-v0.1.0...HEAD
|
|
77
|
+
[0.1.0]: https://github.com/sferik/duel-ruby/releases/tag/duel-postgresql-v0.1.0
|
data/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Erik Berlin
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# duel-postgresql
|
|
2
|
+
|
|
3
|
+
[](https://github.com/sferik/duel-ruby)
|
|
4
|
+
[](https://github.com/sferik/duel-ruby/actions/workflows/plugin-postgresql.yml)
|
|
5
|
+
[](https://rubygems.org/gems/duel-postgresql)
|
|
6
|
+
|
|
7
|
+
A [Duel](https://github.com/sferik/duel-ruby) plugin for code that talks
|
|
8
|
+
to PostgreSQL: each Duel worker gets a PostgreSQL server of its own, and
|
|
9
|
+
every benchmark is judged on what PostgreSQL did for it too.
|
|
10
|
+
|
|
11
|
+
| Metric | What it is | A regression |
|
|
12
|
+
|---|---|---|
|
|
13
|
+
| `postgres_queries` | the statements it ran (`BEGIN` and `COMMIT` too) | any increase |
|
|
14
|
+
| `postgres_rows` | the rows they returned or changed | any increase |
|
|
15
|
+
| `postgres_blocks` | the shared buffer blocks they touched, cached or not | judged like CPU time |
|
|
16
|
+
| `postgres_time` | the time it spent planning and running them, in µs | judged like CPU time |
|
|
17
|
+
|
|
18
|
+
So a change that adds a query per record (an N+1), or fetches rows it
|
|
19
|
+
doesn't use, fails even when its CPU time on the Ruby side doesn't
|
|
20
|
+
change. One that stops a query using an index fails on the blocks it
|
|
21
|
+
reads instead. Blocks are nearly the same from run to run, but not
|
|
22
|
+
quite (a table that's updated grows, say), so they're judged like time,
|
|
23
|
+
on paired ratios, not counted.
|
|
24
|
+
|
|
25
|
+
A regression says which statements did it: under each version's
|
|
26
|
+
numbers, the three that did the most of that measure (Duel's
|
|
27
|
+
`details`), with how many times each ran an iteration for
|
|
28
|
+
`postgres_queries`, and their shares of it for the rest:
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
postgres_queries: 2 queries → 6 queries an iteration
|
|
32
|
+
postgres_queries (candidate): an iteration: 5× SELECT count(*) FROM comments WHERE post_id = $1; 1× SELECT * FROM posts
|
|
33
|
+
postgres_queries (baseline): an iteration: 1× SELECT * FROM comments WHERE post_id = ANY($1); 1× SELECT * FROM posts
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
They're of the sample the numbers came from, but for its `untimed`
|
|
37
|
+
blocks, if it has any. Queries, rows and blocks are counted in their
|
|
38
|
+
units (`1 query`, `12 rows`, though the summary line leaves out the one
|
|
39
|
+
a name ends in: `postgres_rows 12→40`), and `postgres_time` is shown as time
|
|
40
|
+
(`postgres_time: 412µs → 1.31ms an iteration`).
|
|
41
|
+
|
|
42
|
+
## Installation
|
|
43
|
+
|
|
44
|
+
```ruby
|
|
45
|
+
# Gemfile
|
|
46
|
+
group :development do
|
|
47
|
+
gem "duel-postgresql"
|
|
48
|
+
end
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
> [!NOTE]
|
|
52
|
+
> It needs PostgreSQL 14 or later, with `pg_stat_statements` (it's in
|
|
53
|
+
> PostgreSQL's contrib, which most packages include), and the
|
|
54
|
+
> [pg](https://github.com/ged/ruby-pg) gem, so CRuby or TruffleRuby.
|
|
55
|
+
|
|
56
|
+
## Usage
|
|
57
|
+
|
|
58
|
+
```ruby
|
|
59
|
+
# .duel.rb
|
|
60
|
+
Duel.configure { |c| c.use :postgresql }
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
`c.use` requires the plugin, so `.duel.rb` doesn't need to.
|
|
64
|
+
|
|
65
|
+
Each worker's server listens only on a Unix socket in the worker's own
|
|
66
|
+
directory, with nothing written to disk that needn't be, and no
|
|
67
|
+
autovacuum (which would change what a query reads, and its plan, while
|
|
68
|
+
it's measured). Its URL is in `DUEL_POSTGRESQL_URL`, and in
|
|
69
|
+
`DATABASE_URL` too, and libpq's `PGHOST`, `PGPORT`, `PGUSER` and
|
|
70
|
+
`PGDATABASE` point at it, so a client that reads any of them finds it
|
|
71
|
+
and nothing reaches another database. Benchmarks can call
|
|
72
|
+
`Duel::PostgreSQL.url`, `Duel::PostgreSQL.execute` and
|
|
73
|
+
`Duel::PostgreSQL.reset` (each of these two opens a connection of its
|
|
74
|
+
own, from the worker's `DUEL_POSTGRESQL_URL`, and closes it).
|
|
75
|
+
|
|
76
|
+
> [!IMPORTANT]
|
|
77
|
+
> `execute` and `reset` are for setting up: call them in a `setup` hook,
|
|
78
|
+
> or a `once` or `untimed` block. While a benchmark is measured (`Duel.measuring?`) they
|
|
79
|
+
> raise `Duel::Error`, rather than have what they run counted.
|
|
80
|
+
|
|
81
|
+
```ruby
|
|
82
|
+
# duel/orders_duel.rb
|
|
83
|
+
setup do
|
|
84
|
+
Duel::PostgreSQL.reset # empties every table
|
|
85
|
+
Duel::PostgreSQL.execute("INSERT INTO orders (total) SELECT g FROM generate_series(1, 100) g")
|
|
86
|
+
@store = OrderStore.new(Duel::PostgreSQL.url) # its URL
|
|
87
|
+
end
|
|
88
|
+
|
|
89
|
+
duel("orders: the largest ten") { @store.largest(10) }
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Options:
|
|
93
|
+
|
|
94
|
+
```ruby
|
|
95
|
+
c.use :postgresql,
|
|
96
|
+
bin: "/usr/lib/postgresql/17/bin", # its programs (default: DUEL_POSTGRESQL_BIN, or found)
|
|
97
|
+
settings: {work_mem: "64MB"}, # more of its settings
|
|
98
|
+
setup: File.read("db/structure.sql"), # SQL each worker's database runs first
|
|
99
|
+
env: %w[DATABASE_URL PRIMARY_URL] # variables given its URL besides DUEL_POSTGRESQL_URL
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
> [!WARNING]
|
|
103
|
+
> `env` matters with another plugin that sets `DATABASE_URL`
|
|
104
|
+
> (`duel-mysql`, say): two prepare blocks that set a variable differently
|
|
105
|
+
> fail, so give one of them other names (`env: %w[PRIMARY_URL]`, or
|
|
106
|
+
> `env: []`).
|
|
107
|
+
|
|
108
|
+
`bin` is found as the directory of the `postgres` on the `PATH`, links
|
|
109
|
+
followed (Homebrew's `initdb` on the `PATH` may be libpq's, of another
|
|
110
|
+
version), or Debian's newest under `/usr/lib/postgresql`.
|
|
111
|
+
|
|
112
|
+
A database cluster takes `initdb`, and that's slow, so it's done once
|
|
113
|
+
(Duel's `w.cached`): the first worker to start makes one, with the
|
|
114
|
+
`duel` database and `pg_stat_statements` created in it, in your
|
|
115
|
+
project's `tmp/duel/cache`, and every worker (in that run and the next
|
|
116
|
+
ones) starts with a copy. One is made for each server's programs (and
|
|
117
|
+
version of the plugin): another PostgreSQL gets its own. `duel --clean`
|
|
118
|
+
deletes them.
|
|
119
|
+
|
|
120
|
+
The metrics come from `pg_stat_statements`, which PostgreSQL updates as
|
|
121
|
+
each statement ends, before its client hears it has, read by statement
|
|
122
|
+
over a connection of the plugin's own, opened as each worker starts
|
|
123
|
+
(Duel's `c.in_worker`, and closed as it ends): running totals (Duel's
|
|
124
|
+
`c.metric`). `c.use` returns that connection, for a `.duel.rb` that
|
|
125
|
+
reads the server too (`connection.value`, in a worker only). The
|
|
126
|
+
reading's own statement is left out (by its query id), so there's
|
|
127
|
+
nothing to take away (`calibrate: false`). `postgres_time` counts
|
|
128
|
+
planning too (`pg_stat_statements.track_planning`), as a change that
|
|
129
|
+
stops reusing prepared statements plans every time. Up to 10,000
|
|
130
|
+
distinct statements are kept: a benchmark that makes more (with `IN`
|
|
131
|
+
lists of every length, say) loses the oldest, and its counts with them.
|
|
132
|
+
|
|
133
|
+
`postgres_time` is the server's time on the clock, not CPU time, and
|
|
134
|
+
another process's, so what else the machine does shows in it: it's
|
|
135
|
+
judged for each benchmark, but left out of Duel's across-the-board
|
|
136
|
+
check (`across: false`), whose margin is a quarter of its limit.
|
|
137
|
+
|
|
138
|
+
Like any measure, each can be tuned or left out in `.duel.rb`:
|
|
139
|
+
`c.measure("postgres_time", limit: 1.1)`,
|
|
140
|
+
`c.measure("postgres_rows", skip: true)`.
|
|
@@ -0,0 +1,415 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "duel"
|
|
4
|
+
require "digest"
|
|
5
|
+
require "fileutils"
|
|
6
|
+
require "pg"
|
|
7
|
+
require_relative "postgresql/version"
|
|
8
|
+
|
|
9
|
+
module Duel
|
|
10
|
+
# A PostgreSQL server of each worker's own, and what it did for each
|
|
11
|
+
# benchmark, from pg_stat_statements:
|
|
12
|
+
#
|
|
13
|
+
# - postgres_queries: the statements it ran (counted)
|
|
14
|
+
# - postgres_rows: the rows they returned or changed (counted)
|
|
15
|
+
# - postgres_blocks: the shared buffer blocks they touched, from the
|
|
16
|
+
# cache or not (judged like CPU time: nearly steady, but a table
|
|
17
|
+
# that grows, say, takes more)
|
|
18
|
+
# - postgres_time: the time it spent planning and running them, in
|
|
19
|
+
# microseconds (judged like CPU time)
|
|
20
|
+
#
|
|
21
|
+
# Where one regressed, it says which statements did the most of it, on
|
|
22
|
+
# each version (Statements#details).
|
|
23
|
+
#
|
|
24
|
+
# # .duel.rb
|
|
25
|
+
# Duel.configure { |c| c.use :postgresql }
|
|
26
|
+
#
|
|
27
|
+
# Benchmarks can call Duel::PostgreSQL.url, and .execute and .reset in
|
|
28
|
+
# setup (or untimed blocks): while a benchmark is measured, they fail.
|
|
29
|
+
#
|
|
30
|
+
# The server listens only on a Unix socket in the worker's directory;
|
|
31
|
+
# its URL is in DUEL_POSTGRESQL_URL, and in DATABASE_URL too (unless
|
|
32
|
+
# +env+ names others), and libpq's PGHOST, PGPORT, PGUSER and
|
|
33
|
+
# PGDATABASE point at it, so a client finds it and nothing reaches
|
|
34
|
+
# another database.
|
|
35
|
+
module PostgreSQL
|
|
36
|
+
Duel.plugin(:postgresql, self)
|
|
37
|
+
|
|
38
|
+
URL = "DUEL_POSTGRESQL_URL"
|
|
39
|
+
DATABASE = "duel"
|
|
40
|
+
USER = "postgres"
|
|
41
|
+
# It names the socket (.s.PGSQL.5432): the server listens on no port.
|
|
42
|
+
PORT = 5432
|
|
43
|
+
|
|
44
|
+
# The server's settings: nothing to disk that needn't be, nothing
|
|
45
|
+
# running on its own (autovacuum would change what a query touches,
|
|
46
|
+
# and its plan, while it's measured), and pg_stat_statements.
|
|
47
|
+
SETTINGS = {
|
|
48
|
+
"listen_addresses" => "",
|
|
49
|
+
"fsync" => "off",
|
|
50
|
+
"synchronous_commit" => "off",
|
|
51
|
+
"full_page_writes" => "off",
|
|
52
|
+
"autovacuum" => "off",
|
|
53
|
+
"jit" => "off",
|
|
54
|
+
"shared_buffers" => "32MB",
|
|
55
|
+
"shared_preload_libraries" => "pg_stat_statements",
|
|
56
|
+
"compute_query_id" => "on",
|
|
57
|
+
"pg_stat_statements.max" => "10000",
|
|
58
|
+
"pg_stat_statements.track_planning" => "on"
|
|
59
|
+
}.freeze
|
|
60
|
+
|
|
61
|
+
# Configures Duel for the plugin (c.use :postgresql): the server and
|
|
62
|
+
# its metrics. +bin+ is the directory of its programs (postgres and
|
|
63
|
+
# initdb: by default, the real directory of the postgres on the
|
|
64
|
+
# PATH), +settings+ more of its settings, +setup+ SQL each worker's
|
|
65
|
+
# database runs when it's started (a schema, say), and +env+ the
|
|
66
|
+
# variables given its URL besides DUEL_POSTGRESQL_URL. A cluster is
|
|
67
|
+
# initdb'd once (Resource#cached), and each worker's is a copy.
|
|
68
|
+
# Returns the plugin's own connection to it, a worker's (see
|
|
69
|
+
# Config#in_worker, and in tests, Duel::Testing.in_worker).
|
|
70
|
+
def self.configure(config, bin: ENV.fetch("DUEL_POSTGRESQL_BIN", nil), settings: {}, setup: nil, env: %w[DATABASE_URL])
|
|
71
|
+
config.prepare(:worker) do |w|
|
|
72
|
+
programs = File.expand_path(bin || Cluster.find_bin ||
|
|
73
|
+
w.failed("no postgres on the PATH (give its directory as bin:, or DUEL_POSTGRESQL_BIN)"))
|
|
74
|
+
data = w.path("data")
|
|
75
|
+
Cluster.new(w, programs).create(data)
|
|
76
|
+
# (Stopped by INT: its clients are disconnected, not waited for, as
|
|
77
|
+
# TERM would.)
|
|
78
|
+
# (Of a setting given twice, postgres takes the last: +settings+'.)
|
|
79
|
+
w.spawn(File.join(programs, "postgres"), "-D", data, "-k", w.dir, "-p", PORT.to_s,
|
|
80
|
+
*SETTINGS.merge(settings).flat_map { |name, value| ["-c", "#{name}=#{value}"] }, stop_signal: "INT")
|
|
81
|
+
# (The user in the query, with no authority at all: Ruby's URI, which
|
|
82
|
+
# Rails parses DATABASE_URL with, takes a user without a host for
|
|
83
|
+
# a malformed host.)
|
|
84
|
+
url = "postgres:///#{DATABASE}?host=#{w.dir}&port=#{PORT}&user=#{USER}"
|
|
85
|
+
w.wait_until { PG::Connection.ping(url) == PG::PQPING_OK }
|
|
86
|
+
run(url, setup) if setup
|
|
87
|
+
{URL => url, "PGHOST" => w.dir, "PGPORT" => PORT.to_s, "PGUSER" => USER, "PGDATABASE" => DATABASE}
|
|
88
|
+
.merge(env.to_h { |name| [name, url] })
|
|
89
|
+
end
|
|
90
|
+
# (In the worker, libpq's environment is its server's.)
|
|
91
|
+
connection = config.in_worker { |w| PG.connect.tap { |pg| w.at_stop { pg.close } } }
|
|
92
|
+
statements = Statements.new(connection)
|
|
93
|
+
config.metric(METRICS) { statements.read }.costly.details { |stretches, sample| statements.details(stretches, sample.n) }
|
|
94
|
+
connection
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
# Runs SQL on a connection of its own to +url+; returns the last
|
|
98
|
+
# statement's result. (async_exec is exec, by its other name: one a
|
|
99
|
+
# mutation can't make Kernel#exec.)
|
|
100
|
+
def self.run(url, sql)
|
|
101
|
+
connection = PG.connect(url)
|
|
102
|
+
begin
|
|
103
|
+
connection.async_exec(sql)
|
|
104
|
+
ensure
|
|
105
|
+
connection.close
|
|
106
|
+
end
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
# Server time isn't the worker's CPU time: another process's, on the
|
|
110
|
+
# clock, so what else the machine does shows in it. It's judged on
|
|
111
|
+
# each benchmark, but left out across the board, where the margin is
|
|
112
|
+
# too small for it. Its readings leave themselves out (STATEMENTS), so
|
|
113
|
+
# there's nothing to take away.
|
|
114
|
+
METRICS = {
|
|
115
|
+
"postgres_queries" => {kind: :counted, unit: %w[query queries], calibrate: false},
|
|
116
|
+
"postgres_rows" => {kind: :counted, unit: %w[row rows], calibrate: false},
|
|
117
|
+
"postgres_blocks" => {kind: :ratio, unit: %w[block blocks], floor: 100, sizes_samples: false},
|
|
118
|
+
"postgres_time" => {kind: :ratio, unit: "µs", floor: 300, across: false}
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
# What the worker's server has done so far, by statement (query id),
|
|
122
|
+
# but for these readings themselves (their statement is left out by
|
|
123
|
+
# its query id), so nothing's left to take away. pg_stat_statements
|
|
124
|
+
# is updated as each statement ends, before the client hears it has
|
|
125
|
+
# (but for one left open in a transaction: see .settle).
|
|
126
|
+
STATEMENTS = <<~SQL
|
|
127
|
+
SELECT queryid, sum(calls), sum(rows), sum(shared_blks_hit + shared_blks_read), sum(total_plan_time + total_exec_time)
|
|
128
|
+
FROM pg_stat_statements
|
|
129
|
+
WHERE queryid IS DISTINCT FROM (SELECT query_id FROM pg_stat_activity WHERE pid = pg_backend_pid())
|
|
130
|
+
GROUP BY queryid
|
|
131
|
+
SQL
|
|
132
|
+
|
|
133
|
+
# STATEMENTS' columns, as pg decodes them: the query id as text, the
|
|
134
|
+
# counts as Integers and the time (ms) as a Float, with no String
|
|
135
|
+
# made for a number only to be parsed.
|
|
136
|
+
COLUMNS = PG::TypeMapByColumn.new([nil, PG::TextDecoder::Integer.new, PG::TextDecoder::Integer.new,
|
|
137
|
+
PG::TextDecoder::Integer.new, PG::TextDecoder::Float.new])
|
|
138
|
+
|
|
139
|
+
# Statements' texts, by query id (a PostgreSQL array of them).
|
|
140
|
+
TEXTS = "SELECT DISTINCT ON (queryid) queryid, coalesce(query, '') FROM pg_stat_statements WHERE queryid = ANY($1::bigint[])"
|
|
141
|
+
|
|
142
|
+
# What the worker's server has done so far, by statement, read over
|
|
143
|
+
# +connection+ (the plugin's own): a column each of their query ids,
|
|
144
|
+
# queries, rows, blocks and time (ms), a statement to a row.
|
|
145
|
+
def self.columns(connection)
|
|
146
|
+
settle
|
|
147
|
+
# (None is NULL: every statement has a query id.)
|
|
148
|
+
result = connection.async_exec(STATEMENTS).map_types!(COLUMNS)
|
|
149
|
+
[result.column_values(0), result.column_values(1), result.column_values(2), result.column_values(3), result.column_values(4)]
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
# The empty query's parameters (see .settle).
|
|
153
|
+
NO_PARAMS = [] #: Array[String]
|
|
154
|
+
NO_PARAMS.freeze
|
|
155
|
+
|
|
156
|
+
# Ends the statement each of the process's +connections+ (the
|
|
157
|
+
# benchmark's, the app's) has left open in a transaction, for a
|
|
158
|
+
# reading to count it. pg_stat_statements counts a statement as its
|
|
159
|
+
# executor ends, and one sent with the extended protocol (pg's
|
|
160
|
+
# exec_params and exec_prepared, so every query of Active Record's)
|
|
161
|
+
# keeps its executor open, in the unnamed portal, until the
|
|
162
|
+
# connection's next statement, or its transaction's end. Inside an
|
|
163
|
+
# around_iteration hook's transaction, an iteration's last query
|
|
164
|
+
# would be counted only as the hook's ROLLBACK ran, as the hook's.
|
|
165
|
+
# Closing the portal (libpq 17's Close, or before it, an empty query,
|
|
166
|
+
# which takes its place) ends it, and counts nothing of its own. One
|
|
167
|
+
# outside a transaction, busy, or in a failed one has none to end.
|
|
168
|
+
def self.settle(connections = ObjectSpace.each_object(PG::Connection))
|
|
169
|
+
connections.each do |pg|
|
|
170
|
+
next if pg.finished? || pg.transaction_status != PG::PQTRANS_INTRANS
|
|
171
|
+
|
|
172
|
+
pg.respond_to?(:close_portal) ? pg.close_portal("") : pg.exec_params("", NO_PARAMS)
|
|
173
|
+
end
|
|
174
|
+
end
|
|
175
|
+
|
|
176
|
+
# What the worker's server has done so far, by statement, read over
|
|
177
|
+
# +connection+: each one's [queries, rows, blocks, time (ns)], by
|
|
178
|
+
# query id.
|
|
179
|
+
def self.statements(connection)
|
|
180
|
+
ids, calls, rows, blocks, ms = columns(connection)
|
|
181
|
+
ids.each_with_index.to_h { |id, i| [id, [calls.fetch(i), rows.fetch(i), blocks.fetch(i), ns(ms.fetch(i))]] }
|
|
182
|
+
end
|
|
183
|
+
|
|
184
|
+
# A time in ms (pg_stat_statements'), in ns.
|
|
185
|
+
def self.ns(ms) = (ms * 1_000_000).round
|
|
186
|
+
|
|
187
|
+
# What the worker's server has done so far, all told, by measure, read
|
|
188
|
+
# over +connection+.
|
|
189
|
+
def self.stats(connection) = Statements.total(statements(connection).values)
|
|
190
|
+
|
|
191
|
+
# The texts of the statements with these query ids, by query id
|
|
192
|
+
# (normalized: $1 for each constant), read over +connection+.
|
|
193
|
+
def self.texts(connection, ids)
|
|
194
|
+
connection.exec_params(TEXTS, ["{#{ids.join(",")}}"]).values.to_h
|
|
195
|
+
end
|
|
196
|
+
|
|
197
|
+
# For benchmarks: the worker's server.
|
|
198
|
+
def self.url = ENV.fetch(URL)
|
|
199
|
+
|
|
200
|
+
# Runs SQL (statements, ;-separated) on it, over a connection of its
|
|
201
|
+
# own; returns the last one's result. For setting up: it fails while
|
|
202
|
+
# a benchmark is measured (Duel.measuring?), as what it runs would be
|
|
203
|
+
# counted.
|
|
204
|
+
def self.execute(sql)
|
|
205
|
+
Duel.unmeasured!("Duel::PostgreSQL.execute")
|
|
206
|
+
run(url, sql)
|
|
207
|
+
end
|
|
208
|
+
|
|
209
|
+
# Empties every table (but PostgreSQL's own), restarting their
|
|
210
|
+
# sequences. For setting up, as execute is.
|
|
211
|
+
def self.reset
|
|
212
|
+
Duel.unmeasured!("Duel::PostgreSQL.reset")
|
|
213
|
+
run(url, RESET)
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
# What reset runs.
|
|
217
|
+
RESET = <<~SQL
|
|
218
|
+
DO $$
|
|
219
|
+
DECLARE tables text;
|
|
220
|
+
BEGIN
|
|
221
|
+
SELECT string_agg(format('%I.%I', schemaname, tablename), ', ') INTO tables
|
|
222
|
+
FROM pg_tables WHERE schemaname NOT IN ('pg_catalog', 'information_schema');
|
|
223
|
+
IF tables IS NOT NULL THEN
|
|
224
|
+
EXECUTE 'TRUNCATE ' || tables || ' RESTART IDENTITY CASCADE';
|
|
225
|
+
END IF;
|
|
226
|
+
END $$
|
|
227
|
+
SQL
|
|
228
|
+
|
|
229
|
+
# Running totals of what the worker's server did, all told (#read),
|
|
230
|
+
# and by statement, to explain a regression with the statements that
|
|
231
|
+
# did the most of it (#details). A reading makes numbers only for the
|
|
232
|
+
# statements that changed since the last one, and a sample's are told
|
|
233
|
+
# from those: the rest are the last reading's.
|
|
234
|
+
class Statements
|
|
235
|
+
# The measures, in the order of a statement's numbers.
|
|
236
|
+
NAMES = METRICS.keys.freeze
|
|
237
|
+
|
|
238
|
+
# The statements a regression is explained with, and how much of
|
|
239
|
+
# each one's text.
|
|
240
|
+
SHOWN = 3
|
|
241
|
+
TEXT = 80
|
|
242
|
+
|
|
243
|
+
# A statement that did nothing yet.
|
|
244
|
+
NOTHING = [0, 0, 0, 0].freeze
|
|
245
|
+
|
|
246
|
+
# A reading's context: every statement's numbers, by query id (a
|
|
247
|
+
# Hash it shares with the last reading, if none changed), the last
|
|
248
|
+
# reading's, and the query ids of those that changed since.
|
|
249
|
+
Reading = Data.define(:numbers, :previous, :changed)
|
|
250
|
+
|
|
251
|
+
# Statements' numbers, all told, by measure (time in µs).
|
|
252
|
+
def self.total(statements) = by_measure(sum(statements))
|
|
253
|
+
|
|
254
|
+
# Statements' numbers, added up.
|
|
255
|
+
def self.sum(statements) = NAMES.each_index.map { |i| statements.sum { |numbers| numbers.fetch(i) } }
|
|
256
|
+
|
|
257
|
+
# A statement's numbers, or several's added up, by measure (time in
|
|
258
|
+
# µs).
|
|
259
|
+
def self.by_measure(numbers)
|
|
260
|
+
NAMES.each_with_index.to_h { |name, i| [name, (name == "postgres_time") ? numbers.fetch(i).fdiv(1000) : numbers.fetch(i)] }
|
|
261
|
+
end
|
|
262
|
+
|
|
263
|
+
# Read over +connection+, the plugin's own (a worker's).
|
|
264
|
+
def initialize(connection)
|
|
265
|
+
@connection = connection
|
|
266
|
+
@texts = {} # (each statement's, once)
|
|
267
|
+
# (The last reading's numbers, and theirs added up.)
|
|
268
|
+
@numbers = {} #: Hash[String, numbers]
|
|
269
|
+
@sums = NOTHING.dup
|
|
270
|
+
end
|
|
271
|
+
|
|
272
|
+
# What the server has done so far, all told, by measure, and by
|
|
273
|
+
# statement (the reading's context, a Reading, for #details).
|
|
274
|
+
def read
|
|
275
|
+
ids, calls, rows, blocks, ms = PostgreSQL.columns(@connection.value)
|
|
276
|
+
previous = @numbers
|
|
277
|
+
numbers = previous
|
|
278
|
+
changed = [] #: Array[String]
|
|
279
|
+
ids.each_with_index do |id, i|
|
|
280
|
+
was = previous[id]
|
|
281
|
+
time = PostgreSQL.ns(ms.fetch(i))
|
|
282
|
+
next if was && was.fetch(0) == calls.fetch(i) && was.fetch(1) == rows.fetch(i) && was.fetch(2) == blocks.fetch(i) &&
|
|
283
|
+
was.fetch(3) == time
|
|
284
|
+
|
|
285
|
+
# (The last reading's Hash is its own: copied, at the first change.)
|
|
286
|
+
numbers = previous.dup if numbers.equal?(previous)
|
|
287
|
+
now = numbers[id] = [calls.fetch(i), rows.fetch(i), blocks.fetch(i), time]
|
|
288
|
+
changed << id
|
|
289
|
+
was ||= NOTHING
|
|
290
|
+
NAMES.each_index { |n| @sums[n] += now.fetch(n) - was.fetch(n) }
|
|
291
|
+
end
|
|
292
|
+
numbers = forget(numbers, ids) if numbers.size > ids.size
|
|
293
|
+
@numbers = numbers.freeze
|
|
294
|
+
[Statements.by_measure(@sums), Reading.new(numbers, previous, changed)]
|
|
295
|
+
end
|
|
296
|
+
|
|
297
|
+
# The statements that did the most of each measure in a sample's
|
|
298
|
+
# stretches (pairs of #read's readings: see Snapshot#details), by
|
|
299
|
+
# measure: how many times each ran an iteration (+n+ of them), "an
|
|
300
|
+
# iteration: 2× SELECT name FROM items WHERE id = $1", and each
|
|
301
|
+
# one's share of the rest, "SELECT name FROM items WHERE id = $1
|
|
302
|
+
# (75%)".
|
|
303
|
+
def details(stretches, n)
|
|
304
|
+
grew = grew(stretches)
|
|
305
|
+
texts = texts(grew.keys)
|
|
306
|
+
NAMES.each_with_index.filter_map do |name, i|
|
|
307
|
+
top = grew.filter_map { |id, more| [more.fetch(i), texts.fetch(id, id)] if more.fetch(i).positive? } #: Array[[Integer, String]]
|
|
308
|
+
next if top.empty?
|
|
309
|
+
|
|
310
|
+
shown = top.sort_by { |value, text| [-value, text] }.first(SHOWN)
|
|
311
|
+
told = if i.zero? # (queries)
|
|
312
|
+
"an iteration: #{shown.map { |value, text| "#{format("%.4g", value.fdiv(n))}× #{text}" }.join("; ")}"
|
|
313
|
+
else
|
|
314
|
+
all = top.sum { |value, _| value }
|
|
315
|
+
shown.map { |value, text| "#{text} (#{format("%.3g%%", value * 100.0 / all)})" }.join("; ")
|
|
316
|
+
end
|
|
317
|
+
[name, told]
|
|
318
|
+
end.to_h
|
|
319
|
+
end
|
|
320
|
+
|
|
321
|
+
private
|
|
322
|
+
|
|
323
|
+
# What each statement did in the +stretches+, all told, by query
|
|
324
|
+
# id: those that did anything.
|
|
325
|
+
def grew(stretches)
|
|
326
|
+
grew = {} #: Hash[String, numbers]
|
|
327
|
+
stretches.each do |started, stopped|
|
|
328
|
+
before = started.context.numbers
|
|
329
|
+
after = stopped.context
|
|
330
|
+
# (Read right after +started+, it says which changed since: the
|
|
331
|
+
# others can't have done anything.)
|
|
332
|
+
ids = after.previous.equal?(before) ? after.changed : after.numbers.keys
|
|
333
|
+
ids.each do |id|
|
|
334
|
+
now = after.numbers.fetch(id)
|
|
335
|
+
was = before.fetch(id, NOTHING)
|
|
336
|
+
so_far = grew.fetch(id, NOTHING)
|
|
337
|
+
grew[id] = Array.new(now.size) { |i| so_far.fetch(i) + now.fetch(i) - was.fetch(i) }
|
|
338
|
+
end
|
|
339
|
+
end
|
|
340
|
+
grew.select { |_, more| more.any?(&:positive?) }
|
|
341
|
+
end
|
|
342
|
+
|
|
343
|
+
# The +numbers+ of the statements with these query ids (+ids+, of
|
|
344
|
+
# every statement in pg_stat_statements: those gone, the oldest past
|
|
345
|
+
# pg_stat_statements.max, or every one after a reset, forgotten),
|
|
346
|
+
# and their sums.
|
|
347
|
+
def forget(numbers, ids)
|
|
348
|
+
kept = numbers.slice(*ids)
|
|
349
|
+
@sums = Statements.sum(kept.values)
|
|
350
|
+
kept
|
|
351
|
+
end
|
|
352
|
+
|
|
353
|
+
# Each statement's text, squeezed onto a line, by query id: those
|
|
354
|
+
# not yet known read from the server.
|
|
355
|
+
def texts(ids)
|
|
356
|
+
unknown = ids.reject { |id| @texts.key?(id) }
|
|
357
|
+
PostgreSQL.texts(@connection.value, unknown).each { |id, query| @texts[id] = shorten(query.split.join(" ")) } unless unknown.empty?
|
|
358
|
+
@texts
|
|
359
|
+
end
|
|
360
|
+
|
|
361
|
+
def shorten(text) = (text.length > TEXT) ? "#{text[0, TEXT - 1]}…" : text
|
|
362
|
+
end
|
|
363
|
+
|
|
364
|
+
# A worker's database cluster: a copy of one initdb'd once, with the
|
|
365
|
+
# database and pg_stat_statements created in it.
|
|
366
|
+
class Cluster
|
|
367
|
+
# The real directory of the postgres on +path+ (where its initdb is:
|
|
368
|
+
# the one on the PATH may be another version's, libpq's, say), or of
|
|
369
|
+
# the newest Debian's; nil if there's none.
|
|
370
|
+
def self.find_bin(path = ENV.fetch("PATH", ""), debian: "/usr/lib/postgresql/*/bin")
|
|
371
|
+
postgres = path.split(File::PATH_SEPARATOR).map { |dir| File.join(dir, "postgres") }.find { |file| File.executable?(file) }
|
|
372
|
+
return File.dirname(File.realpath(postgres)) if postgres
|
|
373
|
+
|
|
374
|
+
Dir[debian].max_by { |dir| File.basename(File.dirname(dir)).to_i }
|
|
375
|
+
end
|
|
376
|
+
|
|
377
|
+
def initialize(resource, bin)
|
|
378
|
+
@resource = resource
|
|
379
|
+
@bin = bin
|
|
380
|
+
end
|
|
381
|
+
|
|
382
|
+
# Makes one in +data+: a copy of the one made for these programs.
|
|
383
|
+
def create(data)
|
|
384
|
+
FileUtils.cp_r(@resource.cached(key) { |dir| make(dir) }, data)
|
|
385
|
+
File.chmod(0o700, data) # (as postgres insists)
|
|
386
|
+
end
|
|
387
|
+
|
|
388
|
+
private
|
|
389
|
+
|
|
390
|
+
# What a cluster is good for: these programs (until they change),
|
|
391
|
+
# and this version of the plugin.
|
|
392
|
+
def key
|
|
393
|
+
postgres = File.realpath(File.join(@bin, "postgres"))
|
|
394
|
+
"postgresql-#{VERSION}-#{Digest::SHA256.hexdigest([postgres, File.mtime(postgres).to_f].join("\0"))[0, 16]}"
|
|
395
|
+
end
|
|
396
|
+
|
|
397
|
+
def make(data)
|
|
398
|
+
run("initdb", "--pgdata=#{data}", "-U", USER, "-A", "trust", "-E", "UTF8", "--locale=C", "--no-sync")
|
|
399
|
+
single(data, "postgres", "CREATE DATABASE #{DATABASE}")
|
|
400
|
+
single(data, DATABASE, "CREATE EXTENSION pg_stat_statements")
|
|
401
|
+
end
|
|
402
|
+
|
|
403
|
+
# Runs a statement in the cluster's +database+, in single-user mode:
|
|
404
|
+
# no server to start, and stop.
|
|
405
|
+
def single(data, database, sql)
|
|
406
|
+
input = @resource.path("single.sql")
|
|
407
|
+
File.write(input, "#{sql}\n")
|
|
408
|
+
run("postgres", "--single", "-D", data, "-c", "exit_on_error=on", database, in: input)
|
|
409
|
+
end
|
|
410
|
+
|
|
411
|
+
# Runs one of the programs (to its end), logged as its name.
|
|
412
|
+
def run(program, *args, **options) = @resource.run(File.join(@bin, program), *args, **options)
|
|
413
|
+
end
|
|
414
|
+
end
|
|
415
|
+
end
|
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
module Duel
|
|
2
|
+
# A PostgreSQL server of each worker's own, and what it did for each
|
|
3
|
+
# benchmark (postgres_queries, postgres_rows, postgres_blocks,
|
|
4
|
+
# postgres_time).
|
|
5
|
+
module PostgreSQL
|
|
6
|
+
VERSION: String
|
|
7
|
+
|
|
8
|
+
URL: String
|
|
9
|
+
DATABASE: String
|
|
10
|
+
USER: String
|
|
11
|
+
|
|
12
|
+
# It names the socket: the server listens on no port.
|
|
13
|
+
PORT: Integer
|
|
14
|
+
|
|
15
|
+
# The server's settings.
|
|
16
|
+
SETTINGS: Hash[String, String]
|
|
17
|
+
|
|
18
|
+
METRICS: Hash[String, Metric::settings]
|
|
19
|
+
|
|
20
|
+
# What the worker's server has done so far, by statement, but for
|
|
21
|
+
# these readings.
|
|
22
|
+
STATEMENTS: String
|
|
23
|
+
|
|
24
|
+
# STATEMENTS' columns, as pg decodes them.
|
|
25
|
+
COLUMNS: ::PG::TypeMapByColumn
|
|
26
|
+
|
|
27
|
+
# Statements' texts, by query id.
|
|
28
|
+
TEXTS: String
|
|
29
|
+
|
|
30
|
+
# A statement's numbers: queries, rows, blocks, time (ns).
|
|
31
|
+
type numbers = Array[Integer]
|
|
32
|
+
|
|
33
|
+
# Configures Duel for the plugin (c.use :postgresql): the server
|
|
34
|
+
# (+bin+ the directory of its programs, +settings+ more of its
|
|
35
|
+
# settings, +setup+ SQL each worker's database runs when it's started,
|
|
36
|
+
# +env+ the variables besides DUEL_POSTGRESQL_URL given its URL) and
|
|
37
|
+
# its metrics. Returns the plugin's own connection to it, a worker's.
|
|
38
|
+
def self.configure: (Config config, ?bin: String?, ?settings: Hash[String | Symbol, String | Integer], ?setup: String?,
|
|
39
|
+
?env: Array[String]) -> InWorker[::PG::Connection]
|
|
40
|
+
|
|
41
|
+
# Runs SQL on a connection of its own to +url+; returns the last
|
|
42
|
+
# statement's result.
|
|
43
|
+
def self.run: (String url, String sql) -> ::PG::Result
|
|
44
|
+
|
|
45
|
+
# The empty query's parameters (see .settle).
|
|
46
|
+
NO_PARAMS: Array[String]
|
|
47
|
+
|
|
48
|
+
# Ends the statement each of the process's +connections+ has left
|
|
49
|
+
# open in a transaction, for a reading to count it.
|
|
50
|
+
def self.settle: (?Enumerable[::PG::Connection] connections) -> void
|
|
51
|
+
|
|
52
|
+
# What the worker's server has done so far, by statement, read over
|
|
53
|
+
# +connection+: a column each of their query ids, queries, rows,
|
|
54
|
+
# blocks and time (ms).
|
|
55
|
+
def self.columns: (::PG::Connection connection) -> [Array[String], Array[Integer], Array[Integer], Array[Integer], Array[Float]]
|
|
56
|
+
|
|
57
|
+
# What the worker's server has done so far, by statement (query id),
|
|
58
|
+
# read over +connection+.
|
|
59
|
+
def self.statements: (::PG::Connection connection) -> Hash[String, numbers]
|
|
60
|
+
|
|
61
|
+
# A time in ms, in ns.
|
|
62
|
+
def self.ns: (Float ms) -> Integer
|
|
63
|
+
|
|
64
|
+
# What the worker's server has done so far, all told, read over
|
|
65
|
+
# +connection+.
|
|
66
|
+
def self.stats: (::PG::Connection connection) -> Snapshot::totals[Metric::value]
|
|
67
|
+
|
|
68
|
+
# The texts of the statements with these query ids, by query id, read
|
|
69
|
+
# over +connection+.
|
|
70
|
+
def self.texts: (::PG::Connection connection, Array[String] ids) -> Hash[String, String]
|
|
71
|
+
|
|
72
|
+
# For benchmarks: the worker's server.
|
|
73
|
+
def self.url: () -> String
|
|
74
|
+
|
|
75
|
+
# Runs SQL on it, over a connection of its own; returns the last
|
|
76
|
+
# statement's result. For setting up: it fails while a benchmark is
|
|
77
|
+
# measured.
|
|
78
|
+
def self.execute: (String sql) -> ::PG::Result
|
|
79
|
+
|
|
80
|
+
# Empties every table, restarting their sequences. For setting up,
|
|
81
|
+
# as execute is.
|
|
82
|
+
def self.reset: () -> ::PG::Result
|
|
83
|
+
|
|
84
|
+
# What reset runs.
|
|
85
|
+
RESET: String
|
|
86
|
+
|
|
87
|
+
# Running totals of what the worker's server did, all told, and by
|
|
88
|
+
# statement, to explain a regression.
|
|
89
|
+
class Statements
|
|
90
|
+
# The measures, in the order of a statement's numbers.
|
|
91
|
+
NAMES: Array[String]
|
|
92
|
+
|
|
93
|
+
# The statements a regression is explained with, and how much of
|
|
94
|
+
# each one's text.
|
|
95
|
+
SHOWN: Integer
|
|
96
|
+
TEXT: Integer
|
|
97
|
+
|
|
98
|
+
# A statement that did nothing yet.
|
|
99
|
+
NOTHING: numbers
|
|
100
|
+
|
|
101
|
+
# A reading's context: every statement's numbers, by query id, the
|
|
102
|
+
# last reading's, and the query ids of those that changed since.
|
|
103
|
+
class Reading < Data
|
|
104
|
+
attr_reader numbers: Hash[String, numbers]
|
|
105
|
+
attr_reader previous: Hash[String, numbers]
|
|
106
|
+
attr_reader changed: Array[String]
|
|
107
|
+
|
|
108
|
+
def self.new: (Hash[String, numbers] numbers, Hash[String, numbers] previous, Array[String] changed) -> Reading
|
|
109
|
+
end
|
|
110
|
+
|
|
111
|
+
@connection: InWorker[::PG::Connection]
|
|
112
|
+
@texts: Hash[String, String]
|
|
113
|
+
@numbers: Hash[String, numbers]
|
|
114
|
+
@sums: numbers
|
|
115
|
+
|
|
116
|
+
# Statements' numbers, all told, by measure.
|
|
117
|
+
def self.total: (Array[numbers] statements) -> Snapshot::totals[Metric::value]
|
|
118
|
+
|
|
119
|
+
# Statements' numbers, added up.
|
|
120
|
+
def self.sum: (Array[numbers] statements) -> numbers
|
|
121
|
+
|
|
122
|
+
# A statement's numbers, or several's added up, by measure.
|
|
123
|
+
def self.by_measure: (numbers numbers) -> Snapshot::totals[Metric::value]
|
|
124
|
+
|
|
125
|
+
# Read over +connection+, the plugin's own (a worker's).
|
|
126
|
+
def initialize: (InWorker[::PG::Connection] connection) -> void
|
|
127
|
+
|
|
128
|
+
# What the server has done so far, all told, by measure, and by
|
|
129
|
+
# statement (the reading's context, for #details).
|
|
130
|
+
def read: () -> [Snapshot::totals[Metric::value], Reading]
|
|
131
|
+
|
|
132
|
+
# The statements that did the most of each measure in a sample's
|
|
133
|
+
# stretches, by measure: how many times each ran an iteration (+n+
|
|
134
|
+
# of them), and each one's share of the rest.
|
|
135
|
+
def details: (Snapshot::stretches[Metric::value, Reading] stretches, Integer n) -> Hash[String, String]
|
|
136
|
+
|
|
137
|
+
private
|
|
138
|
+
|
|
139
|
+
# What each statement did in the stretches, all told, by query id.
|
|
140
|
+
def grew: (Snapshot::stretches[Metric::value, Reading] stretches) -> Hash[String, numbers]
|
|
141
|
+
|
|
142
|
+
# The +numbers+ of the statements with these query ids, those gone
|
|
143
|
+
# forgotten, and their sums.
|
|
144
|
+
def forget: (Hash[String, numbers] numbers, Array[String] ids) -> Hash[String, numbers]
|
|
145
|
+
|
|
146
|
+
# Each statement's text, by query id.
|
|
147
|
+
def texts: (Array[String] ids) -> Hash[String, String]
|
|
148
|
+
|
|
149
|
+
def shorten: (String text) -> String
|
|
150
|
+
end
|
|
151
|
+
|
|
152
|
+
# A worker's database cluster: a copy of one initdb'd once.
|
|
153
|
+
class Cluster
|
|
154
|
+
@resource: Resource
|
|
155
|
+
@bin: String
|
|
156
|
+
|
|
157
|
+
# The real directory of the postgres on +path+, or of the newest
|
|
158
|
+
# Debian's; nil if there's none.
|
|
159
|
+
def self.find_bin: (?String path, ?debian: String) -> String?
|
|
160
|
+
|
|
161
|
+
def initialize: (Resource resource, String bin) -> void
|
|
162
|
+
|
|
163
|
+
# Makes one in +data+: a copy of the one made for these programs.
|
|
164
|
+
def create: (String data) -> void
|
|
165
|
+
|
|
166
|
+
private
|
|
167
|
+
|
|
168
|
+
def key: () -> String
|
|
169
|
+
|
|
170
|
+
def make: (String data) -> void
|
|
171
|
+
|
|
172
|
+
def single: (String data, String database, String sql) -> void
|
|
173
|
+
|
|
174
|
+
# Runs one of the programs to its end.
|
|
175
|
+
def run: (String program, *String args, **untyped options) -> void
|
|
176
|
+
end
|
|
177
|
+
end
|
|
178
|
+
end
|
metadata
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
|
2
|
+
name: duel-postgresql
|
|
3
|
+
version: !ruby/object:Gem::Version
|
|
4
|
+
version: 0.1.0
|
|
5
|
+
platform: ruby
|
|
6
|
+
authors:
|
|
7
|
+
- Erik Berlin
|
|
8
|
+
bindir: bin
|
|
9
|
+
cert_chain: []
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
11
|
+
dependencies:
|
|
12
|
+
- !ruby/object:Gem::Dependency
|
|
13
|
+
name: duel
|
|
14
|
+
requirement: !ruby/object:Gem::Requirement
|
|
15
|
+
requirements:
|
|
16
|
+
- - "~>"
|
|
17
|
+
- !ruby/object:Gem::Version
|
|
18
|
+
version: '0.1'
|
|
19
|
+
type: :runtime
|
|
20
|
+
prerelease: false
|
|
21
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
22
|
+
requirements:
|
|
23
|
+
- - "~>"
|
|
24
|
+
- !ruby/object:Gem::Version
|
|
25
|
+
version: '0.1'
|
|
26
|
+
- !ruby/object:Gem::Dependency
|
|
27
|
+
name: pg
|
|
28
|
+
requirement: !ruby/object:Gem::Requirement
|
|
29
|
+
requirements:
|
|
30
|
+
- - "~>"
|
|
31
|
+
- !ruby/object:Gem::Version
|
|
32
|
+
version: '1.5'
|
|
33
|
+
type: :runtime
|
|
34
|
+
prerelease: false
|
|
35
|
+
version_requirements: !ruby/object:Gem::Requirement
|
|
36
|
+
requirements:
|
|
37
|
+
- - "~>"
|
|
38
|
+
- !ruby/object:Gem::Version
|
|
39
|
+
version: '1.5'
|
|
40
|
+
description: 'A Duel plugin: each worker gets a PostgreSQL server of its own, and
|
|
41
|
+
every benchmark is judged on the queries it sent, the rows they returned, the buffers
|
|
42
|
+
they touched and their time too.'
|
|
43
|
+
email:
|
|
44
|
+
- sferik@gmail.com
|
|
45
|
+
executables: []
|
|
46
|
+
extensions: []
|
|
47
|
+
extra_rdoc_files: []
|
|
48
|
+
files:
|
|
49
|
+
- CHANGELOG.md
|
|
50
|
+
- LICENSE
|
|
51
|
+
- README.md
|
|
52
|
+
- lib/duel/postgresql.rb
|
|
53
|
+
- lib/duel/postgresql/version.rb
|
|
54
|
+
- sig/duel/postgresql.rbs
|
|
55
|
+
homepage: https://github.com/sferik/duel-ruby/tree/main/plugins/postgresql
|
|
56
|
+
licenses:
|
|
57
|
+
- MIT
|
|
58
|
+
metadata:
|
|
59
|
+
source_code_uri: https://github.com/sferik/duel-ruby/tree/main/plugins/postgresql
|
|
60
|
+
changelog_uri: https://github.com/sferik/duel-ruby/blob/main/plugins/postgresql/CHANGELOG.md
|
|
61
|
+
bug_tracker_uri: https://github.com/sferik/duel-ruby/issues
|
|
62
|
+
rubygems_mfa_required: 'true'
|
|
63
|
+
rdoc_options: []
|
|
64
|
+
require_paths:
|
|
65
|
+
- lib
|
|
66
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
|
67
|
+
requirements:
|
|
68
|
+
- - ">="
|
|
69
|
+
- !ruby/object:Gem::Version
|
|
70
|
+
version: '3.4'
|
|
71
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
|
72
|
+
requirements:
|
|
73
|
+
- - ">="
|
|
74
|
+
- !ruby/object:Gem::Version
|
|
75
|
+
version: '0'
|
|
76
|
+
requirements: []
|
|
77
|
+
rubygems_version: 4.0.23
|
|
78
|
+
specification_version: 4
|
|
79
|
+
summary: A PostgreSQL server for each Duel worker, and what it did for each benchmark
|
|
80
|
+
test_files: []
|