pg-workload 0.3.0__tar.gz → 0.4.0__tar.gz

{pg_workload-0.3.0/src/pg_workload.egg-info → pg_workload-0.4.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: pg-workload
-Version: 0.3.0
-Summary: PostgreSQL backend activity emulator for diverse plans, logs, and runtime statistics.
+Version: 0.4.0
+Summary: PostgreSQL workload emulator for diagnostic and observability testing
 Author-email: O2eg <oleg.ispu@yandex.ru>
 License-Expression: MIT
 Project-URL: Homepage, https://github.com/O2eg/pg_workload
@@ -57,6 +57,24 @@ The distribution name and command are `pg-workload`; the import package is `pg_w
 wheel contains immutable profile templates and a public `pg_workload/v1` JSON Schema. Runtime
 state, logs, credentials, and generated table contents are never packaged.
 
+## Related projects
+
+pg_workload is conceptually different from a benchmark tool such as
+[pg_perf_bench](https://github.com/O2eg/pg_perf_bench): a benchmark measures a system
+(maximum TPS across a controlled load sweep, with a recreated database and captured
+environment evidence for every point), while pg_workload does not measure anything — it
+emulates diverse background activity so that an otherwise empty database starts to "live":
+queries, plans, locks, WAL, autovacuum, errors, and logs. That living database is then
+observed from the other side: diagnostic reports, log parsing, dashboards, and monitoring
+pipelines are tested against it.
+
+| Project | How it is used together with pg_workload |
+|---|---|
+| [pg_stand](https://github.com/O2eg/pg_stand) | Deploys the disposable Docker stands (PostgreSQL 10–18, replication topologies, diagnostic extensions preloaded) that pg_workload loads — the canonical `pg-stand -> pg-workload -> pg-diag` target. |
+| [pg_diag](https://github.com/O2eg/pg_diag) | Captures JSON/HTML diagnostic reports of the activity pg_workload generates. Bundled profile READMEs document which pg_diag sections each profile exercises, and `prepare-db` installs the matching default extensions (`pg_stat_statements`, `pg_buffercache`). |
+| [pg_play](https://github.com/O2eg/pg_play) | Orchestrates pg_workload through the versioned machine contract `pg_play/component/v1` (plan hashes, desired-state scheduler control) — see [docs/pg_play-integration.md](docs/pg_play-integration.md). |
+| [pg_configurator](https://github.com/O2eg/pg_configurator) | Generates versioned PostgreSQL configuration candidates; apply a candidate to the stand and rerun the same pg_workload profiles to observe how behavior changes with the settings. |
+
 ## Installation
 
 ### Install from PyPI
@@ -211,14 +229,25 @@ Use `--bin-dir /path/to/postgresql/bin` when client binaries are outside
 | `many_objects` | Metadata-heavy schemas and partitions | scale-aware SQL object creation |
 | `emulate_errors` | Intentional SQL errors | SQL seed rows only |
 | `pss_overflow` | `pg_stat_statements` churn | scaled Python generator; extension/preload required |
+| `queue_skip_locked` | Producer/consumer task queue with `FOR UPDATE SKIP LOCKED` | synthetic Python generator |
+| `jsonb_docs` | JSONB document store with GIN containment and `jsonb_set` updates | synthetic Python generator |
+| `partition_aging` | Daily-partitioned time-series with partition pruning and aging DDL | synthetic Python generator |
+| `bloat_vacuum` | Dead-tuple churn from non-HOT updates with scheduled `VACUUM` | synthetic Python generator |
 
 No profile requires a dump, CSV file, or network download. The Pagila schema is redistributed
 under its upstream license; see `THIRD_PARTY_NOTICES.md`. Its rows are generated locally. The
 bundled `imdb` profile is an original, compact movie-domain model and does not contain Join Order
 Benchmark SQL or IMDB source data.
 
+Every profile directory contains a `README.md` with the scenario description, its jobs, the
+pg_diag sections it is meant to exercise, and scale/observation guidance.
+
 ## Profile contract
 
+See [docs/profile-authoring-cookbook.md](docs/profile-authoring-cookbook.md) for a complete
+guide to writing custom profiles, including generator conventions and worked recipes for
+queue, JSONB, partitioning, and bloat scenarios.
+
 Profiles live at `data/<profile>/profile.yml`. Version 1 is identified by the stable string
 `pg_workload/v1`:
 
@@ -228,6 +257,7 @@ name: my_workload
 schema: my_workload
 description: Small example workload.
 requires_write: false
+min_pg_version: 12            # optional major-version floor for the target server
 
 prepare:
   steps:
@@ -265,6 +295,7 @@ conditions that JSON Schema alone cannot safely enforce:
 - job names are unique and types are `pgbench` or `psql`;
 - a pgbench job defines exactly one of `duration` or `transactions`;
 - threads do not exceed clients;
+- `min_pg_version`, when present, is an integer major version of at least 10;
 - SQL files, generators, script globs, and logs stay inside their profile boundary even when
   symlinks are present;
 - generator steps reference an existing Python file;
@@ -272,7 +303,10 @@ conditions that JSON Schema alone cannot safely enforce:
 
 `validate` checks the complete contract and all local paths without connecting to PostgreSQL.
 `install` executes `prepare.steps` in declared order. A normal data profile therefore creates its
-schema, runs its generator, and only then creates indexes and statistics.
+schema, runs its generator, and only then creates indexes and statistics. When a profile declares
+`min_pg_version`, `install` also queries the target `server_version_num` and refuses to install
+on an older server — this fails fast instead of breaking later on a missing feature
+(for example, `jsonb_docs` requires PostgreSQL 12 for SQL/JSON path queries).
 
 ```bash
 pg-workload validate
@@ -320,6 +354,10 @@ pg-workload run --profile simple_stock --pgbench-transactions 100
 | `simple_stock_spec_symbols` | the same cardinalities with hostile and Unicode identifiers/values |
 | `imdb` | 10,000 companies; 100,000 people; 100,000 titles; 1,300,000 fact rows |
 | `pagila` | 600 customers; 1,000 films; 4,500 inventory; 16,000 rentals; 16,500 payments |
+| `queue_skip_locked` | 20,000 pending + 5,000 done queue tasks |
+| `jsonb_docs` | 50,000 JSONB documents |
+| `partition_aging` | 33 daily partitions; 200,000 events skewed toward recent days |
+| `bloat_vacuum` | 100,000 accounts, pre-bloated by one non-HOT update round |
 
 For example, `--scale 2` approximately doubles scalable tables. Very small values retain a
 profile-specific minimum so foreign-key structure and query selectivity remain meaningful in CI.
@@ -397,6 +435,7 @@ After `init`, the working directory is:
 project/
   data/<profile>/
     profile.yml
+    README.md             # scenario, jobs, pg_diag watch list, scale guidance
     generator.py          # only when the profile needs table data
     sql/
     log/                 # runtime, ignored

{pg_workload-0.3.0 → pg_workload-0.4.0}/README.md

@@ -18,6 +18,24 @@ The distribution name and command are `pg-workload`; the import package is `pg_w
 wheel contains immutable profile templates and a public `pg_workload/v1` JSON Schema. Runtime
 state, logs, credentials, and generated table contents are never packaged.
 
+## Related projects
+
+pg_workload is conceptually different from a benchmark tool such as
+[pg_perf_bench](https://github.com/O2eg/pg_perf_bench): a benchmark measures a system
+(maximum TPS across a controlled load sweep, with a recreated database and captured
+environment evidence for every point), while pg_workload does not measure anything — it
+emulates diverse background activity so that an otherwise empty database starts to "live":
+queries, plans, locks, WAL, autovacuum, errors, and logs. That living database is then
+observed from the other side: diagnostic reports, log parsing, dashboards, and monitoring
+pipelines are tested against it.
+
+| Project | How it is used together with pg_workload |
+|---|---|
+| [pg_stand](https://github.com/O2eg/pg_stand) | Deploys the disposable Docker stands (PostgreSQL 10–18, replication topologies, diagnostic extensions preloaded) that pg_workload loads — the canonical `pg-stand -> pg-workload -> pg-diag` target. |
+| [pg_diag](https://github.com/O2eg/pg_diag) | Captures JSON/HTML diagnostic reports of the activity pg_workload generates. Bundled profile READMEs document which pg_diag sections each profile exercises, and `prepare-db` installs the matching default extensions (`pg_stat_statements`, `pg_buffercache`). |
+| [pg_play](https://github.com/O2eg/pg_play) | Orchestrates pg_workload through the versioned machine contract `pg_play/component/v1` (plan hashes, desired-state scheduler control) — see [docs/pg_play-integration.md](docs/pg_play-integration.md). |
+| [pg_configurator](https://github.com/O2eg/pg_configurator) | Generates versioned PostgreSQL configuration candidates; apply a candidate to the stand and rerun the same pg_workload profiles to observe how behavior changes with the settings. |
+
 ## Installation
 
 ### Install from PyPI
@@ -172,14 +190,25 @@ Use `--bin-dir /path/to/postgresql/bin` when client binaries are outside
 | `many_objects` | Metadata-heavy schemas and partitions | scale-aware SQL object creation |
 | `emulate_errors` | Intentional SQL errors | SQL seed rows only |
 | `pss_overflow` | `pg_stat_statements` churn | scaled Python generator; extension/preload required |
+| `queue_skip_locked` | Producer/consumer task queue with `FOR UPDATE SKIP LOCKED` | synthetic Python generator |
+| `jsonb_docs` | JSONB document store with GIN containment and `jsonb_set` updates | synthetic Python generator |
+| `partition_aging` | Daily-partitioned time-series with partition pruning and aging DDL | synthetic Python generator |
+| `bloat_vacuum` | Dead-tuple churn from non-HOT updates with scheduled `VACUUM` | synthetic Python generator |
 
 No profile requires a dump, CSV file, or network download. The Pagila schema is redistributed
 under its upstream license; see `THIRD_PARTY_NOTICES.md`. Its rows are generated locally. The
 bundled `imdb` profile is an original, compact movie-domain model and does not contain Join Order
 Benchmark SQL or IMDB source data.
 
+Every profile directory contains a `README.md` with the scenario description, its jobs, the
+pg_diag sections it is meant to exercise, and scale/observation guidance.
+
 ## Profile contract
 
+See [docs/profile-authoring-cookbook.md](docs/profile-authoring-cookbook.md) for a complete
+guide to writing custom profiles, including generator conventions and worked recipes for
+queue, JSONB, partitioning, and bloat scenarios.
+
 Profiles live at `data/<profile>/profile.yml`. Version 1 is identified by the stable string
 `pg_workload/v1`:
 
@@ -189,6 +218,7 @@ name: my_workload
 schema: my_workload
 description: Small example workload.
 requires_write: false
+min_pg_version: 12            # optional major-version floor for the target server
 
 prepare:
   steps:
@@ -226,6 +256,7 @@ conditions that JSON Schema alone cannot safely enforce:
 - job names are unique and types are `pgbench` or `psql`;
 - a pgbench job defines exactly one of `duration` or `transactions`;
 - threads do not exceed clients;
+- `min_pg_version`, when present, is an integer major version of at least 10;
 - SQL files, generators, script globs, and logs stay inside their profile boundary even when
   symlinks are present;
 - generator steps reference an existing Python file;
@@ -233,7 +264,10 @@ conditions that JSON Schema alone cannot safely enforce:
 
 `validate` checks the complete contract and all local paths without connecting to PostgreSQL.
 `install` executes `prepare.steps` in declared order. A normal data profile therefore creates its
-schema, runs its generator, and only then creates indexes and statistics.
+schema, runs its generator, and only then creates indexes and statistics. When a profile declares
+`min_pg_version`, `install` also queries the target `server_version_num` and refuses to install
+on an older server — this fails fast instead of breaking later on a missing feature
+(for example, `jsonb_docs` requires PostgreSQL 12 for SQL/JSON path queries).
 
 ```bash
 pg-workload validate
@@ -281,6 +315,10 @@ pg-workload run --profile simple_stock --pgbench-transactions 100
 | `simple_stock_spec_symbols` | the same cardinalities with hostile and Unicode identifiers/values |
 | `imdb` | 10,000 companies; 100,000 people; 100,000 titles; 1,300,000 fact rows |
 | `pagila` | 600 customers; 1,000 films; 4,500 inventory; 16,000 rentals; 16,500 payments |
+| `queue_skip_locked` | 20,000 pending + 5,000 done queue tasks |
+| `jsonb_docs` | 50,000 JSONB documents |
+| `partition_aging` | 33 daily partitions; 200,000 events skewed toward recent days |
+| `bloat_vacuum` | 100,000 accounts, pre-bloated by one non-HOT update round |
 
 For example, `--scale 2` approximately doubles scalable tables. Very small values retain a
 profile-specific minimum so foreign-key structure and query selectivity remain meaningful in CI.
@@ -358,6 +396,7 @@ After `init`, the working directory is:
 project/
   data/<profile>/
     profile.yml
+    README.md             # scenario, jobs, pg_diag watch list, scale guidance
     generator.py          # only when the profile needs table data
     sql/
     log/                 # runtime, ignored

{pg_workload-0.3.0 → pg_workload-0.4.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pg-workload"
-version = "0.3.0"
-description = "PostgreSQL backend activity emulator for diverse plans, logs, and runtime statistics."
+version = "0.4.0"
+description = "PostgreSQL workload emulator for diagnostic and observability testing"
 readme = "README.md"
 requires-python = ">=3.10"
 license = "MIT"

{pg_workload-0.3.0 → pg_workload-0.4.0}/src/pg_workload/__init__.py

@@ -1,6 +1,6 @@
 """PostgreSQL workload generation and scheduling tools."""
 
-__version__ = "0.3.0"
+__version__ = "0.4.0"
 
 from pg_workload.assets import bundled_profiles_root, initialize_project
 from pg_workload.cli import build_parser
@@ -15,7 +15,7 @@ from pg_workload.profiles import (
     validate_profile,
 )
 from pg_workload.resources import CpuSample, ResourceGuard, assert_resources_available
-from pg_workload.runner import check_profile_requirements, install_profiles, prepare_database
+from pg_workload.runner import check_min_pg_version, check_profile_requirements, install_profiles, prepare_database
 from pg_workload.scheduler import job_recover_on_failure
 from pg_workload.state import effective_schedule, load_state, update_job_state
 
@@ -32,6 +32,7 @@ __all__ = [
     "build_parser",
     "build_runtime_config",
     "bundled_profiles_root",
+    "check_min_pg_version",
     "check_profile_requirements",
     "effective_schedule",
     "file_lock",

pg_workload-0.4.0/src/pg_workload/bundled/profiles/bloat_vacuum/README.md

@@ -0,0 +1,32 @@
+# bloat_vacuum
+
+Dead-tuple generation on purpose: updates touch indexed columns (defeating HOT updates),
+so dead tuples accumulate in the heap and indexes. A scheduled manual `VACUUM` and a bloat
+reporting job make the whole cycle observable. Use it to watch autovacuum queueing, vacuum
+progress, and table/index growth versus live row count.
+
+## Objects
+
+- `bloat_vacuum.accounts` — 100,000 rows at scale 1, pre-bloated by one non-HOT update
+  round during install
+- Secondary indexes on `balance`, `status`, `updated_at` — the columns the churn job updates
+
+## Jobs
+
+| Job | Type | Interval | What it does |
+|---|---|---|---|
+| `churn` | pgbench | 60 s | non-HOT updates of random rows plus a small insert/delete tail |
+| `vacuum_worker` | psql | 300 s | `VACUUM (ANALYZE) bloat_vacuum.accounts` — explicit vacuum activity |
+| `bloat_report` | psql | 600 s | logs `n_dead_tup`, vacuum counts, and timestamps from `pg_stat_user_tables` |
+
+## What to watch in pg_diag
+
+- `storage_vacuum` — dead tuples, autovacuum queue, last vacuum/analyze timestamps
+- `maintenance_progress` — vacuum phases while `vacuum_worker` runs
+- `object_workload` — table and index size growth versus `n_live_tup`
+
+## Recommended scale and observation window
+
+- Scale 0.01–0.1; bloat accumulates per cycle, so longer runs show more.
+- 10–15 minutes to see several churn cycles, at least one manual vacuum, and a bloat
+  report entry. Compare `data/bloat_vacuum/log/bloat_report.log` over time.

pg_workload-0.4.0/src/pg_workload/bundled/profiles/bloat_vacuum/generator.py

@@ -0,0 +1,55 @@
+from __future__ import annotations
+
+import argparse
+import math
+import os
+import subprocess
+
+
+def scaled(base: int, scale: float, minimum: int) -> int:
+    return max(minimum, round(base * scale))
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Generate synthetic bloat_vacuum data")
+    parser.add_argument("--scale", type=float, required=True)
+    args = parser.parse_args()
+    if not math.isfinite(args.scale) or args.scale <= 0:
+        parser.error("--scale must be a finite number greater than zero")
+
+    account_count = scaled(100_000, args.scale, 1_000)
+
+    sql = f"""
+        SELECT setseed(0.90731255);
+
+        INSERT INTO bloat_vacuum.accounts (owner_name, balance, status, updated_at, filler)
+        SELECT
+            'account owner ' || g,
+            round((power(random(), 1.7) * 50000)::numeric, 2),
+            (ARRAY['active', 'active', 'active', 'suspended', 'closed'])[1 + floor(random() * 5)::integer],
+            now() - (random() * interval '30 days'),
+            md5(g::text) || repeat('x', 40)
+        FROM generate_series(1, {account_count}) AS g;
+
+        -- Pre-bloat: one non-HOT update round over a tenth of the table, so the
+        -- first report already shows dead tuples.
+        UPDATE bloat_vacuum.accounts
+        SET balance = balance + 1,
+            updated_at = clock_timestamp(),
+            filler = md5(random()::text) || repeat('x', 40)
+        WHERE id % 10 = 0;
+
+        ANALYZE bloat_vacuum.accounts;
+    """
+    subprocess.run(
+        [os.environ.get("PG_WORKLOAD_PSQL", "psql"), "-X", "-q", "-v", "ON_ERROR_STOP=1"],
+        input=sql,
+        text=True,
+        check=True,
+    )
+
+    print(f"Generated bloat_vacuum: accounts={account_count}")
+
+
+if __name__ == "__main__":
+    main()

pg_workload-0.4.0/src/pg_workload/bundled/profiles/bloat_vacuum/profile.yml

@@ -0,0 +1,37 @@
+api_version: pg_workload/v1
+name: bloat_vacuum
+schema: bloat_vacuum
+description: Dead-tuple churn from non-HOT updates with scheduled VACUUM and bloat reporting.
+requires_write: true
+prepare:
+  steps:
+    - type: sql
+      path: sql/accounts-schema.sql
+    - type: generator
+      path: generator.py
+jobs:
+  - name: churn
+    type: pgbench
+    interval: 60
+    log: churn.log
+    no_vacuum: true
+    clients: 2
+    threads: 2
+    transactions: 5
+    scripts:
+      - path: sql/01_churn.sql
+  - name: vacuum_worker
+    type: psql
+    interval: 300
+    log: vacuum_worker.log
+    command: "VACUUM (ANALYZE) bloat_vacuum.accounts;"
+  - name: bloat_report
+    type: psql
+    interval: 600
+    log: bloat_report.log
+    command: >-
+      SELECT relname, n_live_tup, n_dead_tup,
+             round(100.0 * n_dead_tup / NULLIF(n_live_tup + n_dead_tup, 0), 2) AS dead_pct,
+             vacuum_count, autovacuum_count, last_vacuum, last_autovacuum
+      FROM pg_stat_user_tables
+      WHERE schemaname = 'bloat_vacuum';

pg_workload-0.4.0/src/pg_workload/bundled/profiles/bloat_vacuum/sql/01_churn.sql

@@ -0,0 +1,27 @@
+-- Non-HOT updates on indexed columns: dead tuples grow in heap and indexes.
+UPDATE bloat_vacuum.accounts a
+SET balance = a.balance + (random() * 100 - 50)::numeric(14, 2),
+    updated_at = clock_timestamp(),
+    filler = md5(random()::text) || repeat('x', 40)
+WHERE a.id IN (
+    SELECT 1 + floor(random() * (SELECT max(id) FROM bloat_vacuum.accounts))::bigint
+    FROM generate_series(1, 10)
+);
+
+-- A small insert/delete tail keeps autovacuum interesting.
+INSERT INTO bloat_vacuum.accounts (owner_name, balance, status, filler)
+SELECT
+    'churn user ' || g,
+    round((random() * 10000)::numeric, 2),
+    'active',
+    md5(random()::text) || repeat('x', 40)
+FROM generate_series(1, 3) AS g;
+
+DELETE FROM bloat_vacuum.accounts
+WHERE id IN (
+    SELECT id
+    FROM bloat_vacuum.accounts
+    WHERE owner_name LIKE 'churn user %'
+    ORDER BY random()
+    LIMIT 3
+);

pg_workload-0.4.0/src/pg_workload/bundled/profiles/bloat_vacuum/sql/accounts-schema.sql

@@ -0,0 +1,17 @@
+DROP SCHEMA IF EXISTS bloat_vacuum CASCADE;
+CREATE SCHEMA bloat_vacuum;
+
+-- Several secondary indexes on columns the churn job updates: every update
+-- is non-HOT and leaves dead tuples in both the heap and the indexes.
+CREATE TABLE bloat_vacuum.accounts (
+    id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+    owner_name text NOT NULL,
+    balance numeric(14, 2) NOT NULL,
+    status text NOT NULL,
+    updated_at timestamptz NOT NULL DEFAULT now(),
+    filler text NOT NULL
+);
+
+CREATE INDEX accounts_balance_idx ON bloat_vacuum.accounts (balance);
+CREATE INDEX accounts_status_idx ON bloat_vacuum.accounts (status);
+CREATE INDEX accounts_updated_at_idx ON bloat_vacuum.accounts (updated_at);

pg_workload-0.4.0/src/pg_workload/bundled/profiles/emulate_errors/README.md

@@ -0,0 +1,28 @@
+# emulate_errors
+
+Intentional SQL errors: constraint violations, bad syntax, division by zero, and similar
+failures. The job runs with `allow_failure: true`, so errors are expected and logged. Use
+it to fill PostgreSQL error logs and transaction rollback statistics
+(`pg_stat_database.xact_rollback`) without breaking the workload runner.
+
+## Objects
+
+- `emulate_errors.accounts`, `emulate_errors.transactions` — small fixed seed
+  (`--scale` is ignored by design)
+
+## Jobs
+
+| Job | Type | Interval | What it does |
+|---|---|---|---|
+| `errors` | pgbench | 60 s | statements that fail in various ways, with `allow_failure` |
+
+## What to watch in pg_diag
+
+- PostgreSQL CSV logs — ERROR/FATAL entries with statements (pg_stand stands collect them)
+- `snapshot_delta_workload` — rollback-heavy transaction statistics
+- `sql_workload` — failing statements still visible in `pg_stat_statements`
+
+## Recommended scale and observation window
+
+- Any scale (ignored); the seed is intentionally tiny.
+- 5 minutes is enough to produce a representative error stream.

pg_workload-0.4.0/src/pg_workload/bundled/profiles/imdb/README.md

@@ -0,0 +1,32 @@
+# imdb
+
+Analytical read-only workload over a synthetic movie-domain model: multi-way joins,
+aggregations, selective lookups, and skewed popularity distributions. An original compact
+schema — not the Join Order Benchmark and no IMDB source data (both excluded for licensing
+reasons). Use it to exercise the planner and to produce interesting plans for
+`auto_explain` and `pg_stat_statements`.
+
+## Objects (at scale 1)
+
+- 10,000 companies; 100,000 people; 100,000 titles; ~1,300,000 fact rows
+  (`cast_info`, `movie_keyword`, `movie_company`, `movie_info`)
+- Secondary indexes created after the data load
+
+## Jobs
+
+| Job | Type | Interval | What it does |
+|---|---|---|---|
+| `analytical_selects` | pgbench | 60 s | five join/aggregation scripts of varying selectivity |
+
+## What to watch in pg_diag
+
+- `sql_workload` — top statements by total/mean time with real join plans
+- `object_workload` — sequential vs index scan split on fact tables
+- `buffer_cache` — relation caching behaviour under repeated analytical reads
+- `snapshot_charts_db` — read I/O rates during the observation window
+
+## Recommended scale and observation window
+
+- Scale 1 for meaningful planning; scale 0.1 still keeps join selectivity sensible.
+- 5–10 minutes of scheduled runs; analytical queries dominate `pg_stat_statements`
+  quickly because the job interval is 60 s.

pg_workload-0.4.0/src/pg_workload/bundled/profiles/jsonb_docs/README.md

@@ -0,0 +1,34 @@
+# jsonb_docs
+
+Document-store workload over a `jsonb` column: GIN-indexed containment queries, jsonpath
+extraction, and in-place `jsonb_set` updates of wide documents. Use it to exercise GIN
+indexes, TOAST, and the WAL volume produced by JSONB updates.
+
+## Objects
+
+- `jsonb_docs.documents` — 50,000 documents at scale 1, skewed `customer_id` and tags
+- `documents_doc_gin` — GIN index with `jsonb_path_ops` (containment)
+- `documents_status_idx` — expression index on `(doc ->> 'status')`
+
+## Jobs
+
+| Job | Type | Interval | What it does |
+|---|---|---|---|
+| `writers` | pgbench | 60 s | insert documents; `jsonb_set` updates on random rows |
+| `readers` | pgbench | 60 s | `@>` containment, `jsonb_path_query`, status aggregation |
+| `analyze_docs` | psql | 1800 s | `ANALYZE jsonb_docs.documents` |
+
+## Requirements
+
+- PostgreSQL 12+ (`min_pg_version: 12`) — the reader script uses SQL/JSON path queries
+
+## What to watch in pg_diag
+
+- `indexes` — GIN vs expression index usage and size
+- `wal_io_checkpoints` — WAL volume from full-row JSONB updates
+- `storage_vacuum` — TOAST storage and dead tuples from updates
+
+## Recommended scale and observation window
+
+- Scale 0.01–0.1 for CI; scale 1 makes GIN vs seq-scan choices visible.
+- 5–10 minutes of scheduled runs before a snapshots window.

pg_workload-0.4.0/src/pg_workload/bundled/profiles/jsonb_docs/generator.py

@@ -0,0 +1,56 @@
+from __future__ import annotations
+
+import argparse
+import math
+import os
+import subprocess
+
+
+def scaled(base: int, scale: float, minimum: int) -> int:
+    return max(minimum, round(base * scale))
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Generate synthetic jsonb_docs data")
+    parser.add_argument("--scale", type=float, required=True)
+    args = parser.parse_args()
+    if not math.isfinite(args.scale) or args.scale <= 0:
+        parser.error("--scale must be a finite number greater than zero")
+
+    doc_count = scaled(50_000, args.scale, 1_000)
+
+    sql = f"""
+        SELECT setseed(0.73051984);
+
+        INSERT INTO jsonb_docs.documents (doc_type, doc, created_at)
+        SELECT
+            (ARRAY['order', 'invoice', 'ticket', 'profile'])[1 + floor(random() * 4)::integer],
+            jsonb_build_object(
+                'status', (ARRAY['new', 'open', 'closed'])[1 + floor(random() * 3)::integer],
+                'customer_id', 1 + floor(power(random(), 2) * 10000)::integer,
+                'amount', round((power(random(), 1.5) * 1000)::numeric, 2),
+                'priority', floor(random() * 5)::integer,
+                'tags', (
+                    SELECT jsonb_agg('tag-' || t)
+                    FROM (SELECT DISTINCT floor(random() * 50)::integer AS t
+                          FROM generate_series(1, 1 + floor(random() * 4)::integer)) AS s
+                ),
+                'comment', 'Synthetic document ' || g
+            ),
+            now() - (random() * interval '7 days')
+        FROM generate_series(1, {doc_count}) AS g;
+
+        ANALYZE jsonb_docs.documents;
+    """
+    subprocess.run(
+        [os.environ.get("PG_WORKLOAD_PSQL", "psql"), "-X", "-q", "-v", "ON_ERROR_STOP=1"],
+        input=sql,
+        text=True,
+        check=True,
+    )
+
+    print(f"Generated jsonb_docs: documents={doc_count}")
+
+
+if __name__ == "__main__":
+    main()

pg_workload-0.4.0/src/pg_workload/bundled/profiles/jsonb_docs/profile.yml

@@ -0,0 +1,38 @@
+api_version: pg_workload/v1
+name: jsonb_docs
+schema: jsonb_docs
+description: Document-store workload over jsonb with GIN containment and jsonb_set updates.
+requires_write: true
+min_pg_version: 12
+prepare:
+  steps:
+    - type: sql
+      path: sql/docs-schema.sql
+    - type: generator
+      path: generator.py
+jobs:
+  - name: writers
+    type: pgbench
+    interval: 60
+    log: writers.log
+    no_vacuum: true
+    clients: 2
+    threads: 2
+    transactions: 5
+    scripts:
+      - path: sql/01_write.sql
+  - name: readers
+    type: pgbench
+    interval: 60
+    log: readers.log
+    no_vacuum: true
+    clients: 2
+    threads: 2
+    transactions: 5
+    scripts:
+      - path: sql/02_read.sql
+  - name: analyze_docs
+    type: psql
+    interval: 1800
+    log: analyze_docs.log
+    command: "ANALYZE jsonb_docs.documents;"

pg_workload-0.4.0/src/pg_workload/bundled/profiles/jsonb_docs/sql/01_write.sql

@@ -0,0 +1,20 @@
+INSERT INTO jsonb_docs.documents (doc_type, doc)
+SELECT
+    (ARRAY['order', 'invoice', 'ticket', 'profile'])[1 + floor(random() * 4)::integer],
+    jsonb_build_object(
+        'status', 'new',
+        'customer_id', 1 + floor(power(random(), 2) * 10000)::integer,
+        'amount', round((random() * 1000)::numeric, 2),
+        'priority', floor(random() * 5)::integer,
+        'tags', jsonb_build_array('tag-' || floor(random() * 50)::integer)
+    )
+FROM generate_series(1, 10) AS g;
+
+-- Non-HOT-friendly update of a wide jsonb column on random rows.
+UPDATE jsonb_docs.documents d
+SET doc = jsonb_set(d.doc, '{status}', to_jsonb('closed'::text), false),
+    updated_at = clock_timestamp()
+WHERE d.id IN (
+    SELECT 1 + floor(random() * (SELECT max(id) FROM jsonb_docs.documents))::bigint
+    FROM generate_series(1, 5)
+);

pg_workload-0.4.0/src/pg_workload/bundled/profiles/jsonb_docs/sql/02_read.sql

@@ -0,0 +1,18 @@
+\set customer_id random(1, 10000)
+
+-- GIN-indexed containment lookup.
+SELECT count(*)
+FROM jsonb_docs.documents
+WHERE doc @> jsonb_build_object('customer_id', :customer_id);
+
+-- Expression-index filter plus a jsonpath extraction.
+SELECT d.id, jsonb_path_query(d.doc, '$.tags[*]') AS tag
+FROM jsonb_docs.documents d
+WHERE d.doc ->> 'status' = 'open'
+LIMIT 50;
+
+-- Aggregation over extracted keys.
+SELECT doc ->> 'status' AS status, count(*) AS docs
+FROM jsonb_docs.documents
+GROUP BY 1
+ORDER BY 2 DESC;

pg_workload-0.4.0/src/pg_workload/bundled/profiles/jsonb_docs/sql/docs-schema.sql

@@ -0,0 +1,15 @@
+DROP SCHEMA IF EXISTS jsonb_docs CASCADE;
+CREATE SCHEMA jsonb_docs;
+
+CREATE TABLE jsonb_docs.documents (
+    id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
+    doc_type text NOT NULL,
+    doc jsonb NOT NULL,
+    created_at timestamptz NOT NULL DEFAULT now(),
+    updated_at timestamptz NOT NULL DEFAULT now()
+);
+
+-- Containment (@>) lookups.
+CREATE INDEX documents_doc_gin ON jsonb_docs.documents USING gin (doc jsonb_path_ops);
+-- Equality on the most-filtered key.
+CREATE INDEX documents_status_idx ON jsonb_docs.documents ((doc ->> 'status'));