partition_gardener 0.3.0

data/docs/partition_landscape.md

@@ -0,0 +1,323 @@
+# Partition landscape
+
+Partition Gardener targets PostgreSQL native declarative partitioning with gardener-owned maintenance plans. This page maps industry patterns to what the gem implements, documents experimental layouts, and lists what belongs outside the gem.
+
+## Implemented templates
+
+sliding_window_monthly — layout sliding_window, bucket month. Default time-series on a date or timestamp column.
+
+sliding_window_daily — layout sliding_window, bucket day. High-volume telemetry with short retention.
+
+sliding_window_weekly — layout sliding_window, bucket week. Weekly rollups or compliance windows.
+
+sliding_window_quarterly — layout sliding_window, bucket quarter. Finance or reporting quarters.
+
+calendar_year — layout calendar_year, bucket year. Long-lived yearly archives.
+
+rolling_current_monthly — layout rolling_current, bucket month. One wide current child, no heat splits inside the active window.
+
+premake_monthly — layout premake_monthly, bucket month. Bridge from cron premake; migrate to sliding window.
+
+integer_window — layout integer_window. Monotonic bigint keys with id-band pruning.
+
+list_split — layout list_split. Small stable enum or tenant discriminator.
+
+hash_branches — layout hash_branches. Even fan-out on a hash key.
+
+composite_list_hash — layout composite. LIST parent with HASH sub-trees per branch.
+
+composite_list_range — layout composite. LIST parent with RANGE sliding-window sub-trees.
+
+list_range — layout composite. Alias for composite_list_range.
+
+composite_range_hash — layout composite. RANGE parent with HASH child tables.
+
+composite_range_list — layout composite. RANGE parent with LIST child tables.
+
+Register with `Registry.register_template(:sliding_window_monthly, ...)` or `Registry.register(Templates.sliding_window_monthly(...))`.
+
+## Rolling current
+
+`rolling_current_monthly` keeps the three-area sliding window but disables heat splits inside the active window. Archive months still detach one bucket at a time. Prefer `sliding_window_monthly` unless you explicitly want a single wide current partition.
+
+## Composite trees
+
+Gardener maintains each registered table in a composite tree separately:
+
+- LIST parent (`list_split`) when `parent_mode: :list`
+- RANGE parent (`sliding_window` or `calendar_year`) when `parent_mode: :range`
+- Child tables named `parent_branch` with layout from the branch entry (`hash_branches`, `sliding_window`, or `list_split`)
+
+Native PostgreSQL sub-partition DDL (one attached tree) is your migration responsibility. Gardener plans and rebalances each registered node.
+
+## Rails horizontal sharding
+
+Rails ships horizontal sharding in Active Record (since 6.0, expanded in 8.0). You declare shard databases in `database.yml`, connect models with `connects_to shards:`, route requests with `connected_to(shard:)`, and run cross-shard work with `connected_to_all_shards` (Rails 8+).
+
+That is a different layer from PostgreSQL declarative partitioning:
+
+- Rails sharding splits rows across database servers with the same schema (tenant or capacity isolation).
+- Gardener splits rows across child tables on one PostgreSQL server (time or key bounds, retention, default drain).
+
+They compose. A sharded Rails app can still partition `events` on each shard for retention and tail layout. Gardener maintenance runs against one connection at a time; point `connection_resolver` at the active shard connection, or loop shards in a job:
+
+```ruby
+ShardRecord.connected_to_all_shards do
+  PartitionGardener.run!(table_name: "events")
+end
+```
+
+The default railtie uses `ActiveRecord::Base.connection`. Sharded models should set `connection_resolver` to their shard connection, or run maintenance inside `connected_to(shard:)` for each shard.
+
+Choose Rails sharding when the bottleneck is total row volume or tenant isolation across servers. Choose Gardener templates when the bottleneck is partition catalog size, retention, or hot-month splits inside one database.
+
+## Composite keys and partition pruning
+
+Partitioning only speeds up queries when PostgreSQL can skip child tables. That is partition pruning: the planner uses partition bounds and your `WHERE` clause to exclude partitions that cannot hold matching rows. Without pruning, one large scan becomes many smaller scans and latency often gets worse.
+
+Longer narrative and Rails context: [Tiny story PostgreSQL partitioning and Rails](https://amkisko.github.io/posts/20260306112539_tiny_story_postgresql_partitioning_and_rails.html).
+
+### PostgreSQL rules
+
+PostgreSQL enforces `UNIQUE` and `PRIMARY KEY` per partition, not globally on the parent. Every unique constraint on a partitioned table must include the partition key columns. A typical OLTP shape is composite primary key `(id, occurred_on)` when partitioning by `occurred_on`, or `(id, tenant_id)` when partitioning by `tenant_id`.
+
+Pick the partition key from real query patterns. Community guidance (production write-ups and [PostgreSQL table partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html)) converges on: the column that appears in most hot `WHERE` clauses should drive `PARTITION BY`. If no single dimension dominates, pruning will not help the queries that omit it.
+
+Filters must expose the partition key plainly:
+
+- Prefer range predicates: `occurred_on >= '2026-03-01' AND occurred_on < '2026-04-01'`
+- Avoid wrappers on the key: `DATE(occurred_on)`, `date_trunc('month', occurred_on)`, or casts that hide the bound from the planner
+- Watch `OR` across disjoint ranges; the planner may scan every child
+- Joins that only filter on `id` without the partition key often cannot prune until runtime, and sometimes not at all
+
+Verify with `EXPLAIN (ANALYZE, BUFFERS)` on production-shaped queries. Pruning-capable plans show `Append` with only the relevant children, or `Subplans Removed: N` when runtime pruning applies. Sequential scans on every child mean the application contract is missing the partition key.
+
+`enable_partition_pruning` is on by default. Pruning is driven by partition bounds, not by indexes on the key; indexes still matter inside each child for point lookups.
+
+### Rails application contract
+
+Active Record queries the parent table; models do not need to know child names. Performance still depends on scopes and APIs that carry the partition key.
+
+When the database primary key is composite only because of partitioning, but `id` remains the application identifier:
+
+```ruby
+class Event < ApplicationRecord
+  self.primary_key = :id
+  query_constraints :id, :occurred_on
+end
+```
+
+`query_constraints` (Rails 7.1+) adds listed columns to `UPDATE` and `DELETE` `WHERE` clauses so writes can prune. It does not add the partition key to `SELECT` lookups: `Event.find(id)` may still scan all partitions unless `id` is globally unique and the planner can prove a single child.
+
+For reads, encode the contract in scopes used by hot paths:
+
+```ruby
+scope :in_window, ->(range) { where(occurred_on: range.begin...range.end) }
+```
+
+Inserts must supply routable partition key values (`Event.create!(occurred_on: ...)`) so rows land in named children, not only in `default`.
+
+If you register `partition_key_column: "created_at::date"`, keep application filters on that expression or use a generated column the planner can reason about.
+
+### Gardener `conflict_key`
+
+Registry `conflict_key` must match a parent unique index and should include the partition key, for example `%w[id occurred_on]`. Gardener uses it for idempotent keyset moves during default drain and tail rebalance. The same composite shape that satisfies PostgreSQL uniqueness is what lets maintenance target one partition without cross-child ambiguity.
+
+### Sharded apps
+
+With Rails `connects_to` sharding, carry both signals on hot paths: shard routing first (`connected_to(shard:)`), then partition key filters inside that shard. Losing either one pushes the system toward broad scans. Partition Gardener on each shard does not change that query contract.
+
+### UI and product surfaces
+
+Partition boundaries are an application contract, not only a database detail. Screens and APIs should make the same separation key visible to people so routine work stays inside one prune-friendly slice. User-facing copy should describe periods, accounts, or branches in product language, not partitions, shards, or child tables.
+
+Time-partitioned data (monthly sliding window):
+
+- Default list and calendar views to the current month or a short rolling window, not all history
+- Month or week navigation that maps one-to-one to bucket boundaries (calendar pager, period tabs)
+- Require or strongly default a date range before loading large tables; block unbounded browse behind an explicit export or report flow
+- Date range pickers that emit half-open ranges (`from` inclusive, `to` exclusive) matching server scopes
+- Cursor and infinite scroll keyed by `(occurred_on, id)` or equivalent so pagination stays ordered within a window
+- Create and edit forms that set the event date explicitly; avoid backdating without a visible date field
+
+List- or tenant-partitioned data:
+
+- Workspace, region, or branch selector as the first step when categories are stable LIST keys
+- Scoped navigation so cross-category search is a deliberate action, not the default index
+- Per-tenant dashboards that never load sibling tenants in one query
+
+Sharded Rails apps:
+
+- Account or org context in the URL or session before data loads (`connected_to(shard:)` matches what the user already chose)
+- Avoid global admin search across shards; offer shard-scoped search with optional slow cross-shard report
+
+APIs and exports:
+
+- List endpoints accept `from` / `to` (or `period`) and reject or paginate narrowly when missing
+- Bulk export jobs chunked by month or tenant with progress per chunk
+- GraphQL connections and REST `page` tokens should carry the partition dimension in the cursor when the key is temporal
+
+Anti-patterns:
+
+- Homepage table of millions of rows with only sort-by-id and page number
+- Single global search box that queries by id across all time
+- "All workspaces" aggregate view on hot OLTP paths without a reporting pipeline
+
+When adding a screen, ask whether a typical click produces a query that includes the same key you partition on. If not, change the interaction before adding indexes.
+
+### Aggregates, totals, and snapshots
+
+Dashboard totals, KPI cards, and cross-period rollups should not scan every partition on each page load. Treat them as snapshots: precomputed rows stored outside the hot OLTP path, refreshed in background, with explicit drift control.
+
+Services that run maintenance only when indicators show need (instead of nightly `run!`) should treat snapshots as mandatory on hot paths, not a later optimization. Deferred layout work is viable only when request threads never depend on live aggregates across the full retention window. Design snapshots first, then choose indicator-driven maintenance cadence ([decision_flow.md](decision_flow.md#maintenance-cadence), [background_job.md](background_job.md#schedule)).
+
+Why this pairs with partitioning:
+
+- A `COUNT(*)` or `SUM(amount)` over all history touches every child; latency grows with retention even when list views are scoped
+- Wide reporting queries belong off the request thread; snapshots make the UI fast and predictable
+- Detached or dropped archive partitions still need totals for closed periods; snapshots survive layout changes
+
+Snapshot shape:
+
+- Key snapshots by the same dimensions you partition on: `(period, tenant_id, metric)` or `(calendar_month, workspace_id)`
+- One row per bucket per grain (day, month, quarter), not one global total unless the product truly needs it
+- Store `computed_at`, optional `source_watermark` (max `updated_at` or max id seen in source), and `version` for cache busting
+- Keep snapshots in a normal table or a dedicated rollup schema; they are not partition children of the fact table
+
+Prefer a regular rollup table over a materialized view when bucket keys, drift checks, and incremental updates matter. Materialized views are a poor stand-in for partition-shaped aggregates; see [Materialized views](#materialized-views) below.
+
+Background recalculation:
+
+- Incremental: on each write to the fact table, adjust the open bucket snapshot (current month, active tenant) in the same transaction or via an outbox job
+- Periodic full recompute: nightly or weekly job walks one bucket at a time (`WHERE occurred_on >= ... AND occurred_on < ...`) and replaces snapshot rows for closed periods
+- Chunk jobs by partition bucket so each job prunes to one or few children; never one job that aggregates all shards and all time without bounds
+- On sharded apps, compute per shard first, then merge shard snapshots in a coordinator job if the product needs a global number
+
+Drift control:
+
+- Reconciliation job compares snapshot totals to a fresh `COUNT` / `SUM` over the same bucket window on a schedule; record `drift` and `last_checked_at`
+- Alert when drift exceeds a threshold or when `computed_at` is older than SLA for the period (stale snapshot)
+- After Gardener moves rows (default drain, tail rebalance), rerun bucket recompute for affected windows; moves change row location but not logical totals if keys are stable
+- UI shows "as of" time on totals; optional subtle stale state when reconciliation is behind
+- Idempotent recompute: upsert snapshot rows by bucket key so retries do not double-count
+
+Anti-patterns:
+
+- Live `SELECT SUM(...)` across all partitions on the homepage
+- Caching unbounded aggregate queries in Redis without bucket keys or TTL tied to period close
+- Assuming ORM counter caches on the parent model stay correct when rows move between children during maintenance
+
+Gardener does not maintain application snapshots; it only changes partition layout. Application jobs own rollup freshness and drift checks.
+
+### Materialized views
+
+PostgreSQL does not support declarative partitioning on materialized views. You cannot attach `PARTITION BY RANGE`, `LIST`, or `HASH` to a materialized view, run `ATTACH PARTITION` / `DETACH PARTITION` on one, or add `CHECK` constraints that enable constraint exclusion on child slices. Partition Gardener operates on declarative partitioned tables only; it does not plan, audit, or refresh materialized views.
+
+What still works:
+
+- Create a materialized view whose defining query reads from a partitioned fact table. Partition pruning may reduce work during refresh when the query carries plain predicates on the partition key, but `REFRESH MATERIALIZED VIEW` (PostgreSQL 18 and earlier) still replaces the entire stored result each time.
+- `REFRESH MATERIALIZED VIEW CONCURRENTLY` avoids blocking readers but refreshes the whole view and requires a unique index on plain columns covering all rows.
+
+Manual sharding pattern (common workaround):
+
+- One materialized view per time slice, each defined with a bounded `WHERE` on the source key.
+- A regular `VIEW` with `UNION ALL` over those materialized views as the query surface.
+
+Limits of that pattern:
+
+- No partition pruning across the union; a filter like `WHERE sale_date = '2026-04-15'` typically scans every child materialized view unless the application queries a specific child by name.
+- No `CHECK` constraints on materialized views, so the planner cannot prove row bounds per child.
+- Lifecycle is manual: new period means new materialized view, alter the union view, manage dependencies, and schedule per-slice refresh jobs.
+- pg_partman and Gardener do not maintain materialized views.
+
+Partial refresh on the horizon:
+
+- PostgreSQL 19 may add `REFRESH MATERIALIZED VIEW ... WHERE predicate` for predicate-scoped refresh. That narrows refresh cost; it does not make materialized views first-class partition citizens (no attach/detach, no gardener-style layout).
+
+Recommended pairing with Gardener:
+
+- Fast dashboard totals with drift control — bucket-keyed rollup table (optionally range-partitioned by period)
+- Drop old periods with retention — partitioned fact table plus Gardener; snapshots keyed by closed bucket survive detach
+- Stale-OK full recompute of one report — single materialized view over a bounded source query
+- Independent refresh per month at very large scale — multiple materialized views plus union view, with application routing to children; accept no cross-union pruning
+
+For hot paths next to indicator-driven maintenance, use snapshot tables with `computed_at` and reconciliation ([Aggregates, totals, and snapshots](#aggregates-totals-and-snapshots)), not a monolithic materialized view over all history.
+
+After Gardener moves rows between children, refresh or recompute any materialized view or snapshot that aggregates affected buckets. Gardener does not trigger those jobs.
+
+### Common pitfalls (what teams report)
+
+- Admin search by `id` only on a time-partitioned table scans every month until the key encodes time or lookups include `occurred_on`
+- Reporting queries over wide date ranges intentionally touch many children; that is correct behavior, not a pruning bug
+- Partition count in the thousands increases planner overhead even when pruning works; Gardener sliding window keeps the active catalog bounded
+- Relying on `default` without monitoring; Gardener drains default last, but application inserts that never match bounds still accumulate there
+- Assuming `includes` / `preload` on associations will prune parent partitions when the join does not constrain the partition key
+- Product UI that encourages unbounded browse (all-time lists, global id search) while the table is partitioned by time or tenant
+- Dashboard totals computed live across all partitions instead of served from bucket-keyed snapshots with background refresh and drift checks
+- Treating one large materialized view as a partitioned rollup; refresh cost grows with retention and Gardener row moves do not update it automatically
+- Expecting `UNION ALL` over per-month materialized views to prune like declarative partitions; query the child materialized view directly or use a rollup table
+
+## Out of scope
+
+Delegate these to other tools or extensions; Gardener does not embed them.
+
+### Materialized views
+
+Declarative partitioning and Gardener maintenance apply to tables, not materialized views. See [Materialized views](partition_landscape.md#materialized-views) for manual multi-view patterns, refresh limits, and when to use rollup tables instead.
+
+### ankane Postgres tools
+
+PgHero (observe), Dexter (indexes), pgsync (database copy), and pgslice (one-time partition prep and swap) address different layers than nightly partition maintenance. See [related_postgres_tooling.md](related_postgres_tooling.md) for pairing and overlap.
+
+### pg_partman
+
+Use `maintenance_backend: :pg_partman` or `:hybrid_layout_only` when partman owns premake and detach. Gardener can still plan layout-only diffs in hybrid mode.
+
+### TimescaleDB hypertables
+
+Hypertables use chunk policies and compression, not declarative RANGE children. Use Timescale maintenance for time-series at scale.
+
+### Citus distributed tables
+
+Citus distributes tables across worker nodes inside PostgreSQL with its own shard metadata and rebalance tooling. That is neither Rails `connects_to` sharding nor native declarative partitioning on a single instance. Gardener does not drive Citus shard placement.
+
+### UUIDv7 and encoded-time keys
+
+Time-ordered UUID primary keys are increasingly common. Partitioning on `date_trunc` of a timestamp column remains the recommended Gardener path. For UUID-native range strategies, use pg_partman 5.2+ or custom DDL; Gardener does not parse UUID time components today.
+
+## Choosing a bucket
+
+General application events — start with sliding_window_monthly, active_months 12.
+
+Metrics or logs under 90 days — sliding_window_daily.
+
+Weekly compliance exports — sliding_window_weekly.
+
+Quarterly financial partitions — sliding_window_quarterly.
+
+Multi-year cold storage — calendar_year.
+
+Multi-tenant rows across database servers — Rails `connects_to` shards; optionally partition per shard with Gardener.
+
+## Source
+
+Industry intervals use the same units as pg_partman `part_config.partition_interval` and PostgreSQL `date_trunc`. Composite naming follows native `PARTITION BY LIST` and `PARTITION BY RANGE` sub-partition trees described in PostgreSQL documentation.
+
+Further reading:
+
+- [Tiny story PostgreSQL partitioning and Rails](https://amkisko.github.io/posts/20260306112539_tiny_story_postgresql_partitioning_and_rails.html)
+- [PostgreSQL: Table partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html)
+- [PostgreSQL: Materialized views](https://www.postgresql.org/docs/current/rules-materializedviews.html)
+- [PostgreSQL: REFRESH MATERIALIZED VIEW](https://www.postgresql.org/docs/current/sql-refreshmaterializedview.html)
+- [Rails Guides: Multiple databases (horizontal sharding)](https://guides.rubyonrails.org/active_record_multiple_databases.html)
+- [GitLab: Partitioned tables and composite primary keys](https://docs.gitlab.com/development/database/partitioning/)
+- [Aha!: Partitioning a large table in PostgreSQL with Rails](https://www.aha.io/engineering/articles/partitioning-a-large-table-in-postgresql-with-rails)
+
+Operations and migration docs in this repository:
+
+- [operations.md](operations.md) — runbook
+- [cutover.md](cutover.md) — hot-switch playbook
+- [application_contract.md](application_contract.md) — host app behavior
+- [monitoring.md](monitoring.md) — metrics and alerts
+- [retention.md](retention.md) — archive detach and drop
+- [audit_reference.md](audit_reference.md) — audit and plan catalog

data/docs/pg_party_recipe.md

@@ -0,0 +1,44 @@
+# pg_party recipe — partitioned table creation
+
+Use [pg_party](https://github.com/rkrage/pg_party) in migrations only. Partition Gardener owns nightly runtime maintenance after cutover.
+
+## Migration outline
+
+```ruby
+class PartitionEvents < ActiveRecord::Migration[8.0]
+  def up
+    create_range_partition :events,
+      partition_key: :occurred_on,
+      template: false
+
+    create_default_partition :events
+
+    # Minimal premake at creation; gardener extends the window after cutover
+    [Date.today.beginning_of_month, Date.today.next_month.beginning_of_month].each do |month|
+      create_range_partition_of :events,
+        name: "events_#{month.strftime('%Y_%m')}",
+        start_range: month,
+        end_range: month.next_month.beginning_of_month
+    end
+
+    add_index :events, %i[id occurred_on], unique: true
+  end
+end
+```
+
+## Cutover
+
+1. Backfill shadow partitioned table (`p_events`) from current table.
+2. Include `PartitionGardener::Migration::HotSwitchConcern` in a follow-up migration.
+3. `hot_switch_tables` with `months_ahead: 1`.
+4. Register `events` in `PartitionGardener::Registry` with `sliding_window_monthly`.
+5. Enable nightly `PartitionGardener.run!`.
+
+Full playbook: [cutover.md](cutover.md).
+
+## Do not
+
+- Call pg_party `create_range_partition_of` every night for the same bounds gardener maintains.
+- Run pg_partman premake and gardener tail rebalance on the same parent without `hybrid_layout_only`.
+
+See [tooling_split.md](tooling_split.md) for the three-way responsibility split.

data/docs/related_postgres_tooling.md

@@ -0,0 +1,195 @@
+# Related Postgres tooling (ankane)
+
+Partition Gardener owns runtime layout maintenance on declarative partitions. [Andrew Kane's Postgres tools](https://github.com/ankane) solve adjacent problems: observability, indexing, data copy, and one-time partition cutover. None of them replace Gardener's sliding window, heat splits, default drain, or incremental tail rebalance.
+
+Use this page with [tooling_split.md](tooling_split.md) (creation vs runtime maintenance vs extension) and [partition_landscape.md](partition_landscape.md) (templates and out-of-scope).
+
+## At a glance
+
+[PgHero](https://github.com/ankane/pghero) — database performance dashboard. Low overlap with Gardener (whole-database observability, not partition layout). Pair with nightly `audit` / `run!`.
+
+[Dexter](https://github.com/ankane/dexter) — automatic indexes from query workload. No overlap on partition bounds; use after large rebalance or new hot children.
+
+[pgsync](https://github.com/ankane/pgsync) — copy Postgres data between databases. Overlap on cutover backfill and staging refresh, not nightly maintenance. Use for staging refresh and offline backfill phases.
+
+[pgslice](https://github.com/ankane/pgslice) — one-time partition prep, fill, swap. High overlap on creation; low on runtime maintenance. Create with pgslice before Gardener; do not run both as maintainers.
+
+## PgHero
+
+PgHero is a performance dashboard for Postgres: slow queries, index usage, space, connections, vacuum statistics, and replication lag. It ships as a Rails engine, Docker image, or Linux package.
+
+### Usefulness with Gardener
+
+High as complementary observability, not as a partition maintainer.
+
+Gardener already exposes layout-specific signals: `audit`, `plan`, default row count, horizon days, attached child count, and `RunSummary` metrics ([monitoring.md](monitoring.md)). PgHero answers different questions: which queries are slow globally, which indexes are unused, whether autovacuum is keeping up, and whether connection pools are saturated.
+
+Pair them when:
+
+- A table shows high `rows_moved` or long `run.duration_ms` and you need query-level context (`pg_stat_statements`).
+- Partition count or catalog warnings fire and you want bloat or sequential-scan visibility across the database.
+- Operators want one dashboard for database observability and a second view (Gardener audit JSON or metrics) for partition layout.
+
+### What to learn
+
+Focused scope — one problem (Postgres observability), multiple install paths (Rails engine, Docker, package). Gardener also ships as gem plus CLI plus optional railtie.
+
+Related-project cross-links — each README points to Dexter, pgsync, pgslice. Gardener should keep creation tools (pg_party) and runtime maintenance tools (pg_partman, Gardener) linked the same way; this document is part of that.
+
+Actionable defaults — PgHero highlights issues operators recognize without reading planner internals. Gardener's `audit` warning catalog ([audit_reference.md](audit_reference.md)) should stay similarly concrete.
+
+### What not to take
+
+Do not fold general slow-query analysis into Gardener. Keep partition layout maintenance as the gem boundary ([partition_landscape.md](partition_landscape.md) out-of-scope).
+
+## Dexter
+
+Dexter recommends and optionally creates indexes from `pg_stat_statements`, live activity, or log files. It uses HypoPG to test hypothetical indexes before `CREATE INDEX CONCURRENTLY`.
+
+### Usefulness with Gardener
+
+Medium as a follow-up tool after partition work, not during layout planning.
+
+Partition maintenance changes where data lives and how many child tables exist. New hot-month splits and tail rebalances can shift which indexes matter. Dexter helps when:
+
+- A rebalance moved a large fraction of rows and query plans regressed.
+- New dedicated hot partitions need indexes copied from the parent.
+- Heat-driven splits increased partition count and planners started choosing different paths.
+
+Gardener does not analyze `pg_stat_statements` or create indexes. Dexter does not create partition bounds, drain default, or detach archive children.
+
+### What to learn
+
+Safe-by-default execution — Dexter finds indexes first; `--create` is explicit. Gardener's `plan` / `dry_run` and retention dry-run (`retention_apply: false`) follow the same operator contract.
+
+Noise filters — `--min-calls` and `--min-time` avoid indexing one-off queries. Gardener's heatmap threshold (`split_row_threshold`) follows the same idea: act only when signal is strong enough.
+
+Table allow/deny lists — `--include` / `--exclude` for write-heavy tables. Gardener's per-table registry and `maintenance_backend` skip list serve a similar ownership boundary.
+
+### What not to take
+
+Index recommendation inside Gardener would duplicate Dexter and blur the hot path. Document "run Dexter after major layout change" in [operations.md](operations.md) if operators need a checklist, not a new dependency.
+
+## pgsync
+
+pgsync copies data from one Postgres database to another: parallel table transfer, row filters, schema-first sync, sensitive-data rules, and batch mode for large append-only tables.
+
+### Usefulness with Gardener
+
+High for staging and migration data movement; none for nightly partition maintenance.
+
+Overlap with Gardener appears only in cutover and cross-environment copy:
+
+Phase 2 backfill ([cutover.md](cutover.md)) — copy historical months into a shadow partitioned parent before hot switch. pgsync can move batches with `where` clauses and `--in-batches` on id-keyed tables when you are not using Gardener's built-in delta sync.
+
+Staging refresh ([host_testing.md](host_testing.md)) — sync a subset of production partitions into staging so `plan` and `run!` see realistic shapes.
+
+Cross-database fill — pgslice's README explicitly sends readers to pgsync for syncing between databases during migration.
+
+pgsync does not add future months, split hot buckets, or detach archives. Running pgsync on a production parent while Gardener `run!` is active risks fighting over the same rows unless scopes are disjoint and schedules are coordinated.
+
+### What to learn
+
+Safety rails — default destination is localhost unless `to_safe: true`. Worth echoing in Gardener docs: dry-run plan before apply, retention dry-run, advisory lock skip instead of blocking production.
+
+data_rules — redact emails and secrets on the way out of production. Relevant when exporting partition dumps for compliance ([retention.md](retention.md)) even though Gardener does not implement sync.
+
+Foreign-key ordering — `--defer-constraints` and `--jobs 1` for ordered loads. Cutover backfill order (archive months before current) is the partition-shaped version of the same problem.
+
+### What not to take
+
+Gardener should not become a database-to-database sync tool. Keep `HotSwitchConcern` and cursor rebalance for in-place migration; point operators at pgsync for staging clones and offline copies.
+
+## pgslice
+
+pgslice is the closest cousin: CLI workflow to partition an existing table — `prep` (intermediate parent), `add_partitions`, `fill`, `analyze`, `swap`, then ongoing `add_partitions` via cron. Archive is manual: `pg_dump` partition child, `DROP TABLE`.
+
+### Usefulness with Gardener
+
+High for creation; intentionally redundant for runtime maintenance if both run uncoordinated.
+
+Create partitioned parent — pgslice `prep` plus intermediate table; Gardener pg_party, SQL, or migration ([pg_party_recipe.md](pg_party_recipe.md)).
+
+Premake future bounds — pgslice `add_partitions --future N` on cron; Gardener sliding window future zone or `premake_monthly` bridge.
+
+Move rows into children — pgslice `fill` in id batches; Gardener tail rebalance, default drain, hot-switch delta.
+
+Cutover — pgslice `swap` renames intermediate to live; Gardener `HotSwitchConcern`: `hot_switch_tables`, `hot_unswitch_tables` ([cutover.md](cutover.md)).
+
+Archive old data — pgslice operator script dump plus drop child; Gardener `retention_months` plus detach/drop ([retention.md](retention.md)).
+
+Hot month inside window — not supported in pgslice; Gardener heatmap plus dedicated hot partitions.
+
+Default partition drain — not a first-class pgslice workflow; mandatory last phase in Gardener.
+
+Recommended split (same pattern as [tooling_split.md](tooling_split.md)):
+
+1. Use pgslice (or pg_party plus SQL) once to create or convert the table.
+2. Register with Gardener and set `maintenance_backend: :gardener`.
+3. Stop pgslice `add_partitions` cron for that table — Gardener owns premake and archive layout.
+
+If you keep pgslice cron for premake only, treat it like pg_partman: register `maintenance_backend: :hybrid_layout_only` or `:pg_partman` semantics so Gardener does not create duplicate bounds.
+
+### Hot-switch API (Gardener)
+
+Rails migrations include `PartitionGardener::Migration::HotSwitchConcern`:
+
+`hot_switch_tables` — pgslice equivalent `swap`. Rename live to `_old`, shadow to live, children, sequences.
+
+`hot_unswitch_tables` — pgslice equivalent `unswap`. Reverse swap for drills or rollback.
+
+`analyze_shadow_partitions!` — pgslice equivalent `analyze`. `ANALYZE` shadow children and parent before swap.
+
+`sync_delta_data` — pgslice equivalent `fill`. UPSERT delta rows into shadow (pre-swap).
+
+`sync_delta_data(swapped: true)` — pgslice equivalent `fill --swapped`. Catch-up from `_old` into live after swap.
+
+`swap_lock_timeout` config — pgslice `lock_timeout` in swap. Fail fast on blocked renames (default `5s`).
+
+See [cutover.md](cutover.md) for the full migration example.
+
+### What to learn
+
+Intermediate plus swap — low-downtime rename pattern matches `hot_switch_tables` / `hot_unswitch_tables`. Teams using pgslice for creation can hand off to Gardener at swap ([cutover.md](cutover.md)).
+
+`--dry-run` prints SQL — operators see exact DDL before execution. Gardener `plan` JSON and CLI `plan --pretty` serve a similar trust model for nightly layout; hot-switch migrations execute SQL directly — use staging drills and `compare_table_counts` for cutover.
+
+Partition metadata in comments — pgslice stores `column`, `period`, `version` on the intermediate table comment. Gardener could optionally record layout version in run records for upgrade audits (future idea, not required today).
+
+Missing-partition monitor query — pgslice documents a `pg_class` check for expected future child names. Gardener `horizon_days` and audit warnings cover missing bounds — see [audit_reference.md](audit_reference.md).
+
+App guidance — pgslice README stresses filters on the partition key for pruning. Gardener already covers this in [partition_landscape.md](partition_landscape.md); keep one canonical pruning section.
+
+### What not to take
+
+pgslice's fill loop is id-batch oriented for monolithic to partitioned copy. Gardener's incremental tail rebalance and default drain are ongoing lifecycle concerns — different algorithms, different schedules. Do not merge pgslice fill into Gardener.
+
+## Suggested operator stack
+
+Typical production stack for a Rails app with time-series partitions:
+
+1. Creation — pg_party or pgslice `prep` / `swap` (or Gardener hot-switch migration).
+2. Runtime maintenance — Partition Gardener nightly `run!` plus `audit` cron.
+3. Observe — PgHero (or existing APM) plus Gardener metrics ([monitoring.md](monitoring.md)).
+4. Indexes — Dexter or manual review after large rebalance or template upgrade.
+5. Staging — pgsync from production subset; run `plan` before enabling `run!` in staging.
+
+## Optional follow-ups
+
+Concrete patterns for future work. None are required for current releases.
+
+Handoff checklist in [cutover.md](cutover.md): after pgslice swap, register template and disable pgslice cron.
+
+Optional audit warning when expected future child name is missing (pgslice-style monitor), if not already covered by `horizon_days`.
+
+Operations runbook: run `ANALYZE` on touched children after large `rows_moved` (pgslice `analyze` step; Gardener `analyze_after_rebalance` is the hook).
+
+Cross-link [Dexter](https://github.com/ankane/dexter) in [operations.md](operations.md) when `split_row_threshold` fires often — new hot children may need index review.
+
+## Source
+
+- [PgHero](https://github.com/ankane/pghero)
+- [Dexter](https://github.com/ankane/dexter)
+- [pgsync](https://github.com/ankane/pgsync)
+- [pgslice](https://github.com/ankane/pgslice)
+- In-repo: [tooling_split.md](tooling_split.md), [cutover.md](cutover.md), [monitoring.md](monitoring.md), [partition_landscape.md](partition_landscape.md)

data/docs/retention.md

@@ -0,0 +1,65 @@
+# Retention and compliance
+
+How archive partitions leave the database when `retention_months` is set, and how to coordinate drops with policy, backups, and legal hold.
+
+Configuration reference: [configuration.md](configuration.md) (`retention_months`, `retention_apply`, `retention_keep_table`, `retention_detach_concurrently`).
+
+## How gardener applies retention
+
+After layout work on a date-range sliding window, `ArchiveRetention`:
+
+1. Computes cutoff: `today - retention_months` (calendar months via `DateCalendar`).
+2. Walks attached non-default children.
+3. Skips default, current/open/future tail slots, and managed tail names.
+4. For archive buckets strictly before cutoff month: detach, then drop unless `retention_keep_table`.
+
+Detach uses `ALTER TABLE … DETACH PARTITION`; drop uses `DROP TABLE` on the child. With `retention_detach_concurrently: true`, detach runs `CONCURRENTLY` (PostgreSQL 14+).
+
+Preview by default: when `retention_months` is set and `retention_apply` is omitted or `false`, retention only logs would-drop partitions via `notifier`. Set `retention_apply: true` on the registry entry before archive children detach or drop.
+
+## Detach vs drop
+
+Drop (default) — `retention_keep_table: false`. Child removed from parent and dropped.
+
+Archive tables — `retention_keep_table: true`. Child detached but table remains in schema for cold storage or export.
+
+Use `retention_keep_table: true` when compliance requires a quarantine period before physical drop, or when moving data to object storage from detached tables.
+
+## Before first production drop
+
+1. Confirm backup scope includes detached children or that exports completed.
+2. Run `plan` and review retention segment in operations log.
+3. Verify no legal hold on periods older than cutoff.
+4. Invalidate or freeze snapshot rows for dropped buckets ([partition_landscape.md](partition_landscape.md#aggregates-totals-and-snapshots)).
+5. Document dropped partition names ([naming.md](naming.md)) in change records.
+
+## Legal hold and GDPR
+
+Partition Gardener has no legal-hold flag. Host applications implement hold by:
+
+- Omitting `retention_months` until hold clears, or
+- Setting `retention_keep_table: true` and managing detached tables manually, or
+- Excluding specific buckets via registry change and manual attach (last resort).
+
+Right-to-erasure across partitions still requires row-level delete or targeted child operations; dropping an entire month may suffice when all rows in that period are in scope for deletion and policy allows bulk removal.
+
+Coordinate with counsel before using partition drop as an erasure mechanism.
+
+## pg_partman retention
+
+When `maintenance_backend: :pg_partman`, gardener skips the table; partman owns retention. Hybrid mode: partman premake plus gardener layout only ([tooling_split.md](tooling_split.md)). Do not configure conflicting retention on both sides.
+
+## Operational checklist
+
+- [ ] `retention_months` matches product policy document
+- [ ] Backup verified for oldest archive month before first automated drop
+- [ ] `retention_apply: true` only after backup and policy sign-off
+- [ ] Monitoring alert on retention notifier messages
+- [ ] Rollup snapshots updated or marked inactive for dropped buckets
+- [ ] Runbook entry for restoring from backup if drop was premature ([operations.md](operations.md))
+
+## Related
+
+- [decision_flow.md](decision_flow.md) — retention in layout choice
+- [operations.md](operations.md) — incident response
+- [monitoring.md](monitoring.md) — alerts