pg-workload 0.4.2__tar.gz → 0.6.1__tar.gz

{pg_workload-0.4.2/src/pg_workload.egg-info → pg_workload-0.6.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pg-workload
-Version: 0.4.2
+Version: 0.6.1
 Summary: PostgreSQL workload emulator for diagnostic and observability testing
 Author-email: O2eg <oleg.ispu@gmail.com>
 License-Expression: MIT
@@ -191,7 +191,11 @@ instead of `--passfile`; `PGPASSWORD` is accepted only as an admin-password comp
 `prepare-db --recreate` drops only the selected workload database after terminating its sessions;
 it does not drop the workload role.
 Unless `--extensions` is supplied, it creates both `pg_stat_statements` and
-`pg_buffercache`, matching the default pg_diag metric set used by pg_play.
+`pg_buffercache`, and also installs `pg_stat_kcache` and `pg_wait_sampling` when
+the server provides them. Managed `pg_stand` images include and preload the required
+libraries, and new databases inherit the extensions from `template1`. An explicit
+`--extensions` list replaces both default lists. Optional extension choices are
+included in the machine plan.
 
 ## Targets
 
@@ -225,7 +229,7 @@ Use `--bin-dir /path/to/postgresql/bin` when client binaries are outside
 | `simple_stock` | Write-heavy CRUD over related stock tables | synthetic Python generator |
 | `simple_stock_spec_symbols` | Identifier, Unicode, and parser edge cases | synthetic Python generator |
 | `imdb` | Movie-domain analytical joins and skew | synthetic Python generator |
-| `pagila` | Mixed Pagila OLTP | synthetic Python generator |
+| `pagila` | Pagila OLTP with a 50/25/20/5 script mix | synthetic Python generator |
 | `many_objects` | Metadata-heavy schemas and partitions | scale-aware SQL object creation |
 | `emulate_errors` | Intentional SQL errors | SQL seed rows only |
 | `pss_overflow` | `pg_stat_statements` churn | scaled Python generator; extension/preload required |
@@ -236,8 +240,8 @@ Use `--bin-dir /path/to/postgresql/bin` when client binaries are outside
 
 No profile requires a dump, CSV file, or network download. The Pagila schema is redistributed
 under its upstream license; see `THIRD_PARTY_NOTICES.md`. Its rows are generated locally. The
-bundled `imdb` profile is an original, compact movie-domain model and does not contain Join Order
-Benchmark SQL or IMDB source data.
+bundled `imdb` profile contains a full synthetic 21-table movie-domain model, 38 analytical
+scripts, and 113 SELECT variants; it contains no IMDB source data.
 
 Every profile directory contains a `README.md` with the scenario description, its jobs, the
 pg_diag sections it is meant to exercise, and scale/observation guidance.
@@ -352,7 +356,7 @@ pg-workload run --profile simple_stock --pgbench-transactions 100
 |---|---|
 | `simple_stock` | 150,000 product groups; 1,000,000 SKUs; 1,000,000 stock rows |
 | `simple_stock_spec_symbols` | the same cardinalities with hostile and Unicode identifiers/values |
-| `imdb` | 10,000 companies; 100,000 people; 100,000 titles; 1,300,000 fact rows |
+| `imdb` | 21 tables; 10,000 companies; 100,000 people; 100,000 titles; ~2,000,000 related rows |
 | `pagila` | 600 customers; 1,000 films; 4,500 inventory; 16,000 rentals; 16,500 payments |
 | `queue_skip_locked` | 20,000 pending + 5,000 done queue tasks |
 | `jsonb_docs` | 50,000 JSONB documents |
@@ -380,6 +384,14 @@ to `psql`, so generated rows never pass through repository files. Because a loca
 generator can execute arbitrary code with the runner account, only run profiles from a trusted
 project directory.
 
+`pagila` and `imdb` use the generators and SQL from `pg_perf_bench` v0.6.1, adapted for
+scheduled workloads. Their local `pg_workload.initialization` loader commits bounded data
+batches (100,000 rows by default), then creates indexes and constraints and analyzes every
+table. It uses the selected `psql` and keeps normal durability settings. These two generators
+also accept `--batch-rows N` when called directly; changing batch size preserves generated
+values (apart from wall-clock `last_update` timestamps). Failed initialization stops the
+command; reinstall the profile to recreate its schema and retry.
+
 ## Desired-state scheduler
 
 The scheduler runs in the foreground and owns only its child workload processes. It does not

{pg_workload-0.4.2 → pg_workload-0.6.1}/README.md

@@ -152,7 +152,11 @@ instead of `--passfile`; `PGPASSWORD` is accepted only as an admin-password comp
 `prepare-db --recreate` drops only the selected workload database after terminating its sessions;
 it does not drop the workload role.
 Unless `--extensions` is supplied, it creates both `pg_stat_statements` and
-`pg_buffercache`, matching the default pg_diag metric set used by pg_play.
+`pg_buffercache`, and also installs `pg_stat_kcache` and `pg_wait_sampling` when
+the server provides them. Managed `pg_stand` images include and preload the required
+libraries, and new databases inherit the extensions from `template1`. An explicit
+`--extensions` list replaces both default lists. Optional extension choices are
+included in the machine plan.
 
 ## Targets
 
@@ -186,7 +190,7 @@ Use `--bin-dir /path/to/postgresql/bin` when client binaries are outside
 | `simple_stock` | Write-heavy CRUD over related stock tables | synthetic Python generator |
 | `simple_stock_spec_symbols` | Identifier, Unicode, and parser edge cases | synthetic Python generator |
 | `imdb` | Movie-domain analytical joins and skew | synthetic Python generator |
-| `pagila` | Mixed Pagila OLTP | synthetic Python generator |
+| `pagila` | Pagila OLTP with a 50/25/20/5 script mix | synthetic Python generator |
 | `many_objects` | Metadata-heavy schemas and partitions | scale-aware SQL object creation |
 | `emulate_errors` | Intentional SQL errors | SQL seed rows only |
 | `pss_overflow` | `pg_stat_statements` churn | scaled Python generator; extension/preload required |
@@ -197,8 +201,8 @@ Use `--bin-dir /path/to/postgresql/bin` when client binaries are outside
 
 No profile requires a dump, CSV file, or network download. The Pagila schema is redistributed
 under its upstream license; see `THIRD_PARTY_NOTICES.md`. Its rows are generated locally. The
-bundled `imdb` profile is an original, compact movie-domain model and does not contain Join Order
-Benchmark SQL or IMDB source data.
+bundled `imdb` profile contains a full synthetic 21-table movie-domain model, 38 analytical
+scripts, and 113 SELECT variants; it contains no IMDB source data.
 
 Every profile directory contains a `README.md` with the scenario description, its jobs, the
 pg_diag sections it is meant to exercise, and scale/observation guidance.
@@ -313,7 +317,7 @@ pg-workload run --profile simple_stock --pgbench-transactions 100
 |---|---|
 | `simple_stock` | 150,000 product groups; 1,000,000 SKUs; 1,000,000 stock rows |
 | `simple_stock_spec_symbols` | the same cardinalities with hostile and Unicode identifiers/values |
-| `imdb` | 10,000 companies; 100,000 people; 100,000 titles; 1,300,000 fact rows |
+| `imdb` | 21 tables; 10,000 companies; 100,000 people; 100,000 titles; ~2,000,000 related rows |
 | `pagila` | 600 customers; 1,000 films; 4,500 inventory; 16,000 rentals; 16,500 payments |
 | `queue_skip_locked` | 20,000 pending + 5,000 done queue tasks |
 | `jsonb_docs` | 50,000 JSONB documents |
@@ -341,6 +345,14 @@ to `psql`, so generated rows never pass through repository files. Because a loca
 generator can execute arbitrary code with the runner account, only run profiles from a trusted
 project directory.
 
+`pagila` and `imdb` use the generators and SQL from `pg_perf_bench` v0.6.1, adapted for
+scheduled workloads. Their local `pg_workload.initialization` loader commits bounded data
+batches (100,000 rows by default), then creates indexes and constraints and analyzes every
+table. It uses the selected `psql` and keeps normal durability settings. These two generators
+also accept `--batch-rows N` when called directly; changing batch size preserves generated
+values (apart from wall-clock `last_update` timestamps). Failed initialization stops the
+command; reinstall the profile to recreate its schema and retry.
+
 ## Desired-state scheduler
 
 The scheduler runs in the foreground and owns only its child workload processes. It does not

pg_workload-0.6.1/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,64 @@
+# Third-party notices
+
+## Pagila
+
+The packaged `pagila` profile includes a modified copy of the Pagila schema
+from <https://github.com/devrimgunduz/pagila>. Pagila's data dump is not
+distributed in the source repository or Python package. The profile uses an
+original local generator to create synthetic rows and requires no upstream data.
+
+Copyright (c) Devrim Gündüz <devrim@gunduz.org>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+## Synthetic movie-domain workload
+
+The bundled `imdb` profile includes SQL query shapes migrated and adapted from
+the predecessor `pg_workload_generator` implementation. Its data generator is
+local and deterministic. No source IMDB dataset, CSV archive, or downloaded row
+data is included in the source distribution or wheel.
+
+## pg_perf_bench profile sources
+
+The Pagila and IMDb schemas, load plans, index/constraint definitions and workload SQL
+are adapted from `pg_perf_bench` v0.6.1, commit
+`5eec23ce4c0edf3001bdbdb70339406221a3c903`, <https://github.com/O2eg/pg_perf_bench>.
+The psql loader and scheduled-run adaptations are maintained in pg_workload.
+
+MIT License
+
+Copyright (c) 2023 Tantor Labs
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

{pg_workload-0.4.2 → pg_workload-0.6.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pg-workload"
-version = "0.4.2"
+version = "0.6.1"
 description = "PostgreSQL workload emulator for diagnostic and observability testing"
 readme = "README.md"
 requires-python = ">=3.10"

{pg_workload-0.4.2 → pg_workload-0.6.1}/src/pg_workload/__init__.py

@@ -1,6 +1,6 @@
 """PostgreSQL workload generation and scheduling tools."""
 
-__version__ = "0.4.2"
+__version__ = "0.6.1"
 
 from pg_workload.assets import bundled_profiles_root, initialize_project
 from pg_workload.cli import build_parser

pg_workload-0.6.1/src/pg_workload/bundled/profiles/imdb/README.md

@@ -0,0 +1,52 @@
+# imdb
+
+Analytical read-only workload over a synthetic movie-domain model: multi-way joins,
+aggregations, selective lookups, correlated subgraphs, and skewed popularity distributions.
+It is a workload generator rather than a benchmark: the goal is to keep PostgreSQL busy and
+produce varied plans and statistics for `auto_explain`, `pg_stat_statements`, and DBA tooling.
+No IMDB source rows, CSV files, archives, or network downloads are required.
+
+The schema, generator and all 38 SQL scripts come from `pg_perf_bench` v0.6.1
+(commit `5eec23ce4c0edf3001bdbdb70339406221a3c903`), with a local psql loader.
+The workload queries are unchanged.
+
+## Objects (at scale 1)
+
+- The full 21-table relational shape used by the earlier profile, including alternate names and
+  titles, characters, company and role types, complete-cast metadata, indexed movie attributes,
+  person information, and movie links.
+- 10,000 companies; 100,000 people; 50,000 characters; 20,000 keywords; 100,000 titles; and about
+  2 million fact, relationship, and attribute rows.
+- Deterministic synthetic anchor rows cover the names, years, countries, notes, roles, keywords,
+  genres, ratings, and link types selected by every bundled query family.
+- Independent deterministic hash streams choose participants, companies, keywords and
+  attribute types/values, avoiding cyclic correlations between a film and its relationships.
+- Identifiers are bigint. The local loader recreates the schema and commits bounded data
+  batches, then builds primary keys, join/filter indexes and constraints and runs `ANALYZE`
+  on all 21 tables. It preserves normal logging and server durability settings.
+- The generator accepts `--batch-rows` (default 100,000); batch size does not change the data.
+  After a loading failure, reinstall the profile to recreate its schema and retry.
+
+## Jobs
+
+| Job | Type | Interval | What it does |
+|---|---|---|---|
+| `analytical_selects` | pgbench | 60 s | 38 scripts and 113 SELECT variants: five compact analytics plus 33 multi-join families |
+
+The `select_1.sql` through `select_33.sql` files retain the diverse join shapes of the previous
+implementation. Their predicates target values deliberately generated by `generator.py`, so the
+aggregate results describe existing synthetic rows instead of repeatedly exercising empty joins.
+
+## What to watch in pg_diag
+
+- `sql_workload` — top statements by total/mean time with real join plans
+- `object_workload` — sequential vs index scan split on fact tables
+- `buffer_cache` — relation caching behaviour under repeated analytical reads
+- `snapshot_charts_db` — read I/O rates during the observation window
+
+## Recommended scale and observation window
+
+- Scale 1 for meaningful planning; scale 0.1 still keeps join selectivity sensible. Very small
+  scales retain the complete anchor dataset and at least 1,000 titles for CI and training.
+- 5–10 minutes of scheduled runs; analytical queries dominate `pg_stat_statements`
+  quickly because the job interval is 60 s.