partition_gardener 0.3.5 → 0.3.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +7 -0
- data/README.md +1 -1
- data/docs/THIRD_PARTY_LICENSE_MANIFEST.tsv +4 -5
- data/docs/application_contract.md +29 -3
- data/docs/audit_reference.md +1 -1
- data/docs/changelogs/20260813142300_rebalance-before-archive-attach.md +19 -0
- data/docs/changelogs/20260813144516_relock-appraisal-advisory-json.md +21 -0
- data/docs/changelogs/20260902114500_align-child-columns-before-attach.md +19 -0
- data/docs/changelogs/20260902124500_cut-patch-release-0-3-4.md +24 -0
- data/docs/changelogs/20260909143500_occupied-window-current-origin.md +19 -0
- data/docs/changelogs/20260909154900_tail-slot-rfc.md +21 -0
- data/docs/changelogs/20260909162554_high-end-future-occupancy.md +21 -0
- data/docs/changelogs/20260909164000_shipped-contract-rfcs.md +21 -0
- data/docs/changelogs/20260909165500_cut-patch-release-0-3-5.md +24 -0
- data/docs/changelogs/20260916144600_public-identifiers-contract.md +23 -0
- data/docs/changelogs/20261005135700_checkpoint-table-name.md +23 -0
- data/docs/changelogs/20261005141621_cut-patch-release-0-3-6.md +23 -0
- data/docs/configuration.md +3 -1
- data/docs/dependencies/20260813144516_json-cve-2026-71847.md +27 -0
- data/docs/issues/20260909143500_occupied-window-current-origin.md +25 -0
- data/docs/issues/20260909164000_shipped-contract-rfcs.md +25 -0
- data/docs/issues/20260916113300_opaque-composite-public-ids.md +55 -0
- data/docs/issues/20260917123253_ci-drop-automatic-dependency-audit.md +28 -0
- data/docs/operations.md +1 -1
- data/docs/partition_landscape.md +6 -4
- data/lib/partition_gardener/configuration.rb +9 -1
- data/lib/partition_gardener/sql_run_record_store.rb +40 -5
- data/lib/partition_gardener/version.rb +1 -1
- metadata +18 -15
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 168cd3dce806746b67801bac88f82c1bff76d9617aa7d11bb543a21476f92810
|
|
4
|
+
data.tar.gz: 3b89596e3ee530b5ea262b3d9e7e6396077b6198ad811951867e113e959c335a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f55dcaca8b968cb74fbd77af80a7f8c14dbc7419274fdbbbab2fd0cfb11a2098e0a36b70908722180cbe344b87b18b532b68951e20be054cb1738c011566c009
|
|
7
|
+
data.tar.gz: f416553eda6be0049cc5dfdcd0fcbd4ef929c85b3e6411dc1fce3c210b87927f8ea846a6220705b7ca3ffc892b5539548eb5f342e8dd5b0f83324b68c6f5de7b
|
data/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
## Unreleased
|
|
4
4
|
|
|
5
|
+
## 0.3.6 (2026-10-05)
|
|
6
|
+
|
|
7
|
+
- Store rebalance checkpoints in `partition_gardener_checkpoints` by default (previously `partition_gardener_run_records`)
|
|
8
|
+
- Add `run_record_table_name` so a host can keep another checkpoint table name
|
|
9
|
+
- Rename leftover `partition_gardener_runs` or `partition_gardener_run_records` on first use of the default table name
|
|
10
|
+
- Document public identifiers for partitioned rows: four contracts, Rails signed_id versus scalar primary_key, fail-closed decode ([application_contract.md](docs/application_contract.md#public-identifiers))
|
|
11
|
+
|
|
5
12
|
## 0.3.5 (2026-09-09)
|
|
6
13
|
|
|
7
14
|
- Keep attached monthly children that already cover the start of the sliding-window active range, and fill the remainder as `_open` rather than overlapping `_current`
|
data/README.md
CHANGED
|
@@ -37,7 +37,7 @@ Complements migration gems (e.g. [pg_party](https://github.com/rkrage/pg_party))
|
|
|
37
37
|
### Application and migration
|
|
38
38
|
|
|
39
39
|
- [docs/cutover.md](docs/cutover.md) — hot-switch playbook and template upgrades
|
|
40
|
-
- [docs/application_contract.md](docs/application_contract.md) — writes, moves, routing when the key is missing, bulk load, replicas
|
|
40
|
+
- [docs/application_contract.md](docs/application_contract.md) — writes, moves, routing when the key is missing, public identifiers, bulk load, replicas
|
|
41
41
|
- [docs/naming.md](docs/naming.md) — child partition naming catalog
|
|
42
42
|
- [docs/host_testing.md](docs/host_testing.md) — CI and staging for host apps
|
|
43
43
|
|
|
@@ -17,15 +17,15 @@ root language_server-protocol 3.17.0.6 MIT
|
|
|
17
17
|
root lint_roller 1.1.0 MIT
|
|
18
18
|
root logger 1.7.0 Ruby|BSD-2-Clause
|
|
19
19
|
root minitest 6.0.6 MIT
|
|
20
|
-
root parallel 2.
|
|
20
|
+
root parallel 2.1.0 MIT
|
|
21
21
|
root parser 3.3.12.0 MIT
|
|
22
|
-
root partition_gardener 0.3.
|
|
22
|
+
root partition_gardener 0.3.6 MIT
|
|
23
23
|
root pg 1.6.3 BSD-2-Clause
|
|
24
|
-
root polyrun 2.2.
|
|
24
|
+
root polyrun 2.2.3 MIT
|
|
25
25
|
root prism 1.9.0 MIT
|
|
26
26
|
root prosopite 2.2.0 Apache-2.0
|
|
27
27
|
root racc 1.8.1 Ruby|BSD-2-Clause
|
|
28
|
-
root rack 3.2.
|
|
28
|
+
root rack 3.2.6 MIT
|
|
29
29
|
root rainbow 3.1.1 MIT
|
|
30
30
|
root rake 13.4.2 MIT
|
|
31
31
|
root rbs 4.2.0 BSD-2-Clause|Ruby
|
|
@@ -35,7 +35,6 @@ root rspec-core 3.13.6 MIT
|
|
|
35
35
|
root rspec-expectations 3.13.5 MIT
|
|
36
36
|
root rspec-mocks 3.13.8 MIT
|
|
37
37
|
root rspec-support 3.13.7 MIT
|
|
38
|
-
root rspec_junit_formatter 0.6.0 MIT
|
|
39
38
|
root rubocop 1.88.2 MIT
|
|
40
39
|
root rubocop-ast 1.50.0 MIT
|
|
41
40
|
root rubocop-capybara 2.23.0 MIT
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Application contract
|
|
2
2
|
|
|
3
|
-
How host applications should behave around partitioned tables: queries, writes, maintenance side effects, bulk load, and replicas. For UI scoping and aggregate snapshots see [partition_landscape.md](partition_landscape.md). For operator runbooks see [operations.md](operations.md).
|
|
3
|
+
How host applications should behave around partitioned tables: queries, writes, public identifiers, maintenance side effects, bulk load, and replicas. For UI scoping and aggregate snapshots see [partition_landscape.md](partition_landscape.md). For operator runbooks see [operations.md](operations.md).
|
|
4
4
|
|
|
5
5
|
## Query and write basics
|
|
6
6
|
|
|
@@ -8,6 +8,7 @@ How host applications should behave around partitioned tables: queries, writes,
|
|
|
8
8
|
- `conflict_key` columns match the parent unique index; updates and deletes use `query_constraints` when the logical id is not globally unique ([partition_landscape.md](partition_landscape.md#rails-application-contract)).
|
|
9
9
|
- Inserts supply a routable partition key value so rows land in named children, not only in `default`.
|
|
10
10
|
- When only a logical id or parent reference is available, follow the recovery ladder in [partition_landscape.md](partition_landscape.md#routing-hints-when-the-key-is-not-in-hand).
|
|
11
|
+
- Public URLs and API ids follow [Public identifiers](#public-identifiers).
|
|
11
12
|
- After maintenance with high `rows_moved`, refresh `id → partition_key` mappings when the partition key can change.
|
|
12
13
|
|
|
13
14
|
## During maintenance (row moves)
|
|
@@ -31,6 +32,31 @@ Duplicate key across children — overlapping manual DDL. Application response:
|
|
|
31
32
|
|
|
32
33
|
Applications should not catch routing errors and retry without the partition key.
|
|
33
34
|
|
|
35
|
+
## Public identifiers
|
|
36
|
+
|
|
37
|
+
Treat database uniqueness, public identifier, routing hint, and authorization as four contracts. PostgreSQL uniqueness on a partitioned parent is composite because the partition key must sit in every unique index. That does not by itself decide what a person sees in a URL or API.
|
|
38
|
+
|
|
39
|
+
When a single sequence or UUIDv7 keeps `id` unique across children, keep that logical id as the application identifier: `self.primary_key = :id` so `to_param` and JSON stay a scalar, not a Rails underscore composite such as `42_100`. Still attach a routing hint on point lookups so `SELECT` can prune. Do not invent that hint from current month.
|
|
40
|
+
|
|
41
|
+
When `id` is unique only inside a child, the public identifier for point lookups must carry every uniqueness column. Prefer one opaque path segment over Rails `extract_value` underscore form (`4_2`) or a separate query parameter the caller can edit independently.
|
|
42
|
+
|
|
43
|
+
Prefer Rails APIs over a homemade encoding:
|
|
44
|
+
|
|
45
|
+
- True composite `primary_key` (array): `signed_id` / `find_signed` or `to_sgid` / `SignedGlobalID`. Both uniqueness columns travel; `find` can prune. `signed_id` uses HMAC SHA256, JSON, and `url_safe: true`. The payload is encoded, not encrypted.
|
|
46
|
+
- Scalar `primary_key = :id` plus `query_constraints`: `signed_id` and `generates_token_for` still sign and find by `id` alone. `query_constraints` never feeds `SELECT`. Sign `[id, partition_key]` with a host `ActiveSupport::MessageVerifier` (`url_safe: true`), or resolve through an unpartitioned mapping table.
|
|
47
|
+
- `has_secure_token` stores a random Base58 mapping value. A unique index on that column alone is illegal on a partitioned parent unless the partition key is in the index. Put the mapping off the partitioned fact table, or keep a globally unique logical id on the fact row.
|
|
48
|
+
- Default `MessageVerifier` is not URL-safe. `signed_id` and `GlobalID::Verifier` opt into `url_safe`. Do not encrypt identifiers with `MessageEncryptor`.
|
|
49
|
+
- Unsigned GlobalID (`gid://app/Event/123/2026-03-15`) and `to_param` underscore composites are tunable. Keep them off public point-lookup URLs.
|
|
50
|
+
- Extra JSON in `generates_token_for` is compared after fetch. It is not a `WHERE` predicate and is plaintext in the token.
|
|
51
|
+
|
|
52
|
+
Active Record 7.1 is this gem's development floor. Composite `find_by_token_for` needs 7.2 or 8.0. Composite `find_signed` wrapping `primary_key => [id]` needs 8.1. Hosts on 7.1 should verify finders before relying on signed composite lookup.
|
|
53
|
+
|
|
54
|
+
After decode, query with both columns and fail closed. Do not fall back to id-only scan. A bad or unsigned token is 404, not a bounded retry across children. Encoding is packaging, not authorization. Lookups still go through an ownership set.
|
|
55
|
+
|
|
56
|
+
List screens still show period, account, or branch in product language ([partition_landscape.md](partition_landscape.md#ui-and-product-surfaces)). Tenant and list keys for hot paths come from session or parent context, not from a tunable URL field. Pagination cursors already belong in the same family as opaque tokens.
|
|
57
|
+
|
|
58
|
+
Gardener does not ship an encoder. Host applications own secrets, token version, and decode.
|
|
59
|
+
|
|
34
60
|
## Bulk import and backfill
|
|
35
61
|
|
|
36
62
|
- Chunk `COPY` or `insert_all` by bucket window within partition bounds (month, week, day).
|
|
@@ -69,7 +95,7 @@ Registry JSON may differ per shard only if layouts differ (unusual).
|
|
|
69
95
|
## Admin and operator surfaces
|
|
70
96
|
|
|
71
97
|
- Default filters: current month or selected tenant, not all history.
|
|
72
|
-
- Global id search is an advanced, slow path; require date or tenant hint, or resolve through a mapping table before scanning children.
|
|
98
|
+
- Global id search is an advanced, slow path; require date or tenant hint, or resolve through a mapping table before scanning children ([Public identifiers](#public-identifiers)).
|
|
73
99
|
- Export flows chunk by bucket; show progress per period.
|
|
74
100
|
- Totals read from snapshot tables with `computed_at`, not live `SUM` across all children or a stale materialized view over the full fact table ([partition_landscape.md](partition_landscape.md#materialized-views)).
|
|
75
101
|
|
|
@@ -79,6 +105,6 @@ See [host_testing.md](host_testing.md) for CI registry fixtures and integration
|
|
|
79
105
|
|
|
80
106
|
## Related
|
|
81
107
|
|
|
82
|
-
- [partition_landscape.md](partition_landscape.md) — pruning, routing layers and hints, UI, snapshots
|
|
108
|
+
- [partition_landscape.md](partition_landscape.md) — pruning, routing layers and hints, UI, snapshots; `query_constraints` versus `SELECT`
|
|
83
109
|
- [cutover.md](cutover.md) — backfill and switch
|
|
84
110
|
- [naming.md](naming.md) — child table names
|
data/docs/audit_reference.md
CHANGED
|
@@ -98,7 +98,7 @@ Per-table metrics (`RunMetrics#to_h`):
|
|
|
98
98
|
|
|
99
99
|
## Run record table
|
|
100
100
|
|
|
101
|
-
When `run_record_enabled`, PostgreSQL table `
|
|
101
|
+
When `run_record_enabled`, PostgreSQL table `partition_gardener_checkpoints` (or `run_record_table_name`) stores:
|
|
102
102
|
|
|
103
103
|
`table_name` — parent.
|
|
104
104
|
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Rebalance before archive attach
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Date-range maintenance rebalances the sliding-window tail before it finalizes monthly archives or applies retention. That lets one run detach a current partition that still starts on 1 July, attach a new current partition from 1 August, send July rows to default, and attach a non-overlapping monthly child from default.
|
|
10
|
+
|
|
11
|
+
Archive attach is skipped when the bucket start is at or after the attached current partition lower bound. Those buckets wait until rebalance moves the current span forward. The lower bound is read with Connection.current_partition_lower_bound.
|
|
12
|
+
|
|
13
|
+
## Effects
|
|
14
|
+
|
|
15
|
+
A July-to-August maintenance sequence attaches the July monthly child and a current partition from 1 August.
|
|
16
|
+
|
|
17
|
+
## Source
|
|
18
|
+
|
|
19
|
+
Requested in the partition_gardener sliding-window rollover fix. Specs: spec/integration/sliding_window_maintenance_spec.rb and spec/partition_gardener/attach_retry_spec.rb.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Relock appraisal graphs for json advisory
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Appraisal lockfiles are part of the CI install graph. After 0.3.3 they must pin partition_gardener 0.3.3 and json 2.21.2 so frozen bundle install and bundle-audit both pass. The dependency audit workflow scans Gemfile.lock and *.gemfile.lock.
|
|
10
|
+
|
|
11
|
+
## Effects
|
|
12
|
+
|
|
13
|
+
Relocked gemfiles/rails72.gemfile.lock, gemfiles/rails8ruby34.gemfile.lock, and gemfiles/rails8ruby4.gemfile.lock. Widened the find in .github/workflows/dependency-audit.yml.
|
|
14
|
+
|
|
15
|
+
## Next
|
|
16
|
+
|
|
17
|
+
Merge the patch so main test and dependency audit jobs install a consistent graph.
|
|
18
|
+
|
|
19
|
+
## Source
|
|
20
|
+
|
|
21
|
+
Triggered by failing Dependency audit and test workflows after the 0.3.3 merge. Advisory note: usr/docs/dependencies/20260813144516_json-cve-2026-71847.md.
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Align child columns before attach
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
ATTACH PARTITION now runs additive column align first (default on). CREATE TABLE LIKE and DETACH snapshot columns; a later parent ADD COLUMN does not reach that child. Align adds all missing ordinary columns in one ALTER TABLE statement (skip identity and generated), preserving type, non-default collation, default, and NOT NULL. PostgreSQL rejects the complete statement when existing rows cannot satisfy NOT NULL, so standalone execution does not leave a partial alignment. Extra child columns stay. Existing types and other constraints are not rewritten.
|
|
10
|
+
|
|
11
|
+
Audit warns when an unattached managed child still lags the parent. Built-in strategies filter unrelated prefixed tables and load all candidate column metadata in one catalog query. Custom partition name formatters retain prefix discovery because detached tables have no parent provenance. align_child_columns: false keeps fail-fast attach. JSON registry import rejects non-boolean align_child_columns values.
|
|
12
|
+
|
|
13
|
+
## Effects
|
|
14
|
+
|
|
15
|
+
LIKE and keep-table reattach paths attach after a parent ADD COLUMN without a host workaround. A missing conflict-key column is aligned before its child index is created.
|
|
16
|
+
|
|
17
|
+
## Source
|
|
18
|
+
|
|
19
|
+
https://github.com/amkisko/partition_gardener.rb/issues/18
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# Cut patch release 0.3.4
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Cut patch release 0.3.4 for child column align before ATTACH PARTITION, shipped after 0.3.3.
|
|
10
|
+
|
|
11
|
+
## Effects
|
|
12
|
+
|
|
13
|
+
Moved Unreleased notes into CHANGELOG.md under 0.3.4. Bumped lib/partition_gardener/version.rb and path-gem lockfiles to 0.3.4.
|
|
14
|
+
|
|
15
|
+
## Next
|
|
16
|
+
|
|
17
|
+
Merge trunk/0.3.4 to main, then run make release.
|
|
18
|
+
|
|
19
|
+
## Source
|
|
20
|
+
|
|
21
|
+
https://github.com/amkisko/partition_gardener.rb/issues/18
|
|
22
|
+
https://github.com/amkisko/partition_gardener.rb/pull/19
|
|
23
|
+
|
|
24
|
+
Feature note: usr/docs/changelogs/20260902114500_align-child-columns-before-attach.md
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Occupied window current origin
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Date-range planning keeps attached monthly children whose ranges overlap the active window. The remainder after a contiguous occupying prefix from the window start is named open. Current is reserved for a filler that starts at the window origin. OccupiedWindow reads Connection.attached_partitions bounds. ZoneSegments walks those catalog segments before heat splits and fillers. Monthly name parse also accepts an optional first-of-month suffix so those child names resolve as that month.
|
|
10
|
+
|
|
11
|
+
A later pass changed the remainder name from current to open so the slot matches the documented origin rule.
|
|
12
|
+
|
|
13
|
+
## Effects
|
|
14
|
+
|
|
15
|
+
A monthly catalog through the current month no longer plans a current partition that overlaps the occupying child. Postgres attach of the remainder filler starts at the next month as open.
|
|
16
|
+
|
|
17
|
+
## Source
|
|
18
|
+
|
|
19
|
+
Follows usr/docs/issues/20260909143500_occupied-window-current-origin.md. Specs: spec/partition_gardener/date_range_occupied_window_spec.rb, spec/partition_gardener/sliding_window_spec.rb, spec/integration/sliding_window_maintenance_spec.rb.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Tail slot RFC
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
RFC 0001 records the project RFC process. RFC 0002 specifies occupancy-first tail slot names. A date-named child that covers a range is that range. Current, open, and future fill only uncovered ranges. Future starts after premade months that occupy keys at or after the window end. Registry flags are out of this RFC. A later pass dropped ensure_current, ensure_future, and ensure_default from the draft.
|
|
10
|
+
|
|
11
|
+
## Effects
|
|
12
|
+
|
|
13
|
+
rfcs/0001-rfc-process.md and rfcs/0002-tail-slots.md are Proposed. Occupied-window naming (open after an occupying origin) matches RFC 0002 current-zone occupancy. A later pass put high-end occupancy for future into the planner. Registry flags are not in this RFC.
|
|
14
|
+
|
|
15
|
+
## Next
|
|
16
|
+
|
|
17
|
+
Discuss RFC 0002 unresolved questions. Implementation PRs cite RFC-0002.
|
|
18
|
+
|
|
19
|
+
## Source
|
|
20
|
+
|
|
21
|
+
Follows usr/docs/issues/20260909143500_occupied-window-current-origin.md and rfcs/0002-tail-slots.md.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# High-end occupancy for future
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
A date-named child that covers keys at or after the active window end stays in the plan. Future starts after the latest finite end among those occupants. Future is omitted when an occupant already ends at MAXVALUE. Holes between those occupants stay gaps. Default remains residual routing. No registry flags.
|
|
10
|
+
|
|
11
|
+
## Effects
|
|
12
|
+
|
|
13
|
+
The sliding-window planner keeps a premade month that begins at the window end and starts future from the next month. A child that straddles the window end pushes future to that child's end. Noncontiguous premade months past the window stay attached. Apply no longer attaches future over a month that already occupies that range.
|
|
14
|
+
|
|
15
|
+
## Next
|
|
16
|
+
|
|
17
|
+
Discuss RFC 0002 unresolved questions. Integer-window occupancy is unchanged.
|
|
18
|
+
|
|
19
|
+
## Source
|
|
20
|
+
|
|
21
|
+
Follows rfcs/0002-tail-slots.md and usr/docs/issues/20260909143500_occupied-window-current-origin.md.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Shipped contract RFCs
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Stable RFCs 0003 through 0015 specify contracts that already ship: sliding window layout, registry JSON, plan and audit reports, archive names, retention, CLI, maintenance backends, keyset rebalance, advisory lock and run records, hot switch, child column align, integer window, and hash list composite. RFC 0001 and RFC 0002 stay Proposed. Integer occupancy stays unresolved inside RFC 0014. Registry flags for ensure slots stay out.
|
|
10
|
+
|
|
11
|
+
## Effects
|
|
12
|
+
|
|
13
|
+
rfcs/ids/0003 through rfcs/ids/0015 are claimed. Matching rfcs/NNNN-slug.md files are Stable. rfcs/README.md Current set lists them. CHANGELOG 0.3.5 records the set. Operator guides gained Related links. docs/cli.md apply --all example now includes --confirm.
|
|
14
|
+
|
|
15
|
+
## Next
|
|
16
|
+
|
|
17
|
+
Implementation PRs cite RFC-NNNN. Discuss RFC 0002 unresolved questions. Integer occupancy is a follow-on, not this set.
|
|
18
|
+
|
|
19
|
+
## Source
|
|
20
|
+
|
|
21
|
+
Follows the RFC 0002 occupancy-first work and the request to specify remaining shipped contracts under rfcs/.
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
# Cut patch release 0.3.5
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Cut patch release 0.3.5 on main for occupancy-first sliding-window plans and RFCs 0001 through 0015, shipped after 0.3.4. Stay on main. No trunk branch.
|
|
10
|
+
|
|
11
|
+
## Effects
|
|
12
|
+
|
|
13
|
+
Moved Unreleased notes into CHANGELOG.md under 0.3.5. Bumped lib/partition_gardener/version.rb and path-gem lockfiles to 0.3.5.
|
|
14
|
+
|
|
15
|
+
## Next
|
|
16
|
+
|
|
17
|
+
Commit the version cut, then run make release.
|
|
18
|
+
|
|
19
|
+
## Source
|
|
20
|
+
|
|
21
|
+
Occupancy: usr/docs/changelogs/20260909143500_occupied-window-current-origin.md
|
|
22
|
+
High-end future: usr/docs/changelogs/20260909162554_high-end-future-occupancy.md
|
|
23
|
+
RFC 0002: usr/docs/changelogs/20260909154900_tail-slot-rfc.md
|
|
24
|
+
Stable RFCs: usr/docs/changelogs/20260909164000_shipped-contract-rfcs.md
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Public identifiers contract
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Host applications treat database uniqueness, public identifier, routing hint, and authorization as four contracts. When id is unique across children, keep a scalar application id and attach a routing hint on point lookups. When id is unique only inside a child, the public point-lookup identifier carries every uniqueness column as one opaque path segment.
|
|
10
|
+
|
|
11
|
+
Prefer Rails signed_id or SignedGlobalID when primary_key is the composite array. When primary_key stays :id, sign [id, partition_key] with a host MessageVerifier or use an unpartitioned mapping table. query_constraints still only helps writes. Do not use to_param underscore form or unsigned GlobalID for public point lookups. Fail closed after decode. Browse screens still show period or tenant. Gardener does not ship an encoder.
|
|
12
|
+
|
|
13
|
+
## Effects
|
|
14
|
+
|
|
15
|
+
docs/application_contract.md gained a Public identifiers section. docs/partition_landscape.md Rails application contract, routing hints, and UI surfaces point to it. README application_contract blurb names public identifiers.
|
|
16
|
+
|
|
17
|
+
## Next
|
|
18
|
+
|
|
19
|
+
No RFC unless public identifiers become a gem runtime contract.
|
|
20
|
+
|
|
21
|
+
## Source
|
|
22
|
+
|
|
23
|
+
usr/docs/issues/20260916113300_opaque-composite-public-ids.md
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Checkpoint table name
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
SqlRunRecordStore default table is partition_gardener_checkpoints. Hosts set run_record_table_name when they already own another relation. First use with the default name renames partition_gardener_runs or partition_gardener_run_records when that catalog relation exists and the default table does not.
|
|
10
|
+
|
|
11
|
+
RFC 0011 records the table, the config key, and the rename.
|
|
12
|
+
|
|
13
|
+
## Effects
|
|
14
|
+
|
|
15
|
+
lib/partition_gardener/sql_run_record_store.rb and configuration.rb. Docs in configuration, operations, and audit_reference. Specs in sql_run_record_store_spec.rb.
|
|
16
|
+
|
|
17
|
+
## Next
|
|
18
|
+
|
|
19
|
+
Hosts that already created partition_gardener_run_records keep resume rows after the first enabled run, or set run_record_table_name to keep the old name.
|
|
20
|
+
|
|
21
|
+
## Source
|
|
22
|
+
|
|
23
|
+
RFC 0011
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Cut patch release 0.3.6
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Cut patch release 0.3.6 on main for the default checkpoint table name, `run_record_table_name`, leftover table rename, and public identifier docs, shipped after 0.3.5. Stay on main. No trunk branch.
|
|
10
|
+
|
|
11
|
+
## Effects
|
|
12
|
+
|
|
13
|
+
Moved Unreleased notes into CHANGELOG.md under 0.3.6. Bumped lib/partition_gardener/version.rb and path-gem lockfiles to 0.3.6.
|
|
14
|
+
|
|
15
|
+
## Next
|
|
16
|
+
|
|
17
|
+
Run make release.
|
|
18
|
+
|
|
19
|
+
## Source
|
|
20
|
+
|
|
21
|
+
Checkpoint table: docs/changelogs/20261005135700_checkpoint-table-name.md
|
|
22
|
+
Public identifiers: docs/changelogs/20260916144600_public-identifiers-contract.md
|
|
23
|
+
RFC 0011
|
data/docs/configuration.md
CHANGED
|
@@ -47,6 +47,8 @@ When the gem loads inside Rails, the railtie sets `connection_resolver` and `tod
|
|
|
47
47
|
|
|
48
48
|
`run_record_enabled` (default: `true`) — persist phase checkpoints for resume (global default).
|
|
49
49
|
|
|
50
|
+
`run_record_table_name` (default: `partition_gardener_checkpoints`) — PostgreSQL table used by `SqlRunRecordStore`. On first use with the default name, the store renames `partition_gardener_runs` or `partition_gardener_run_records` when that relation exists and the default table does not.
|
|
51
|
+
|
|
50
52
|
`run_record_store` (default: `SqlRunRecordStore` when a database connection is available, else in-memory) — where rebalance checkpoints are stored.
|
|
51
53
|
|
|
52
54
|
`strict_maintenance_backend_validation` (default: `false`) — when `true`, raise `MaintenanceBackend::ValidationError` on register if `maintenance_backend` disagrees with `partman.parent_config`; when `false`, notify only.
|
|
@@ -67,7 +69,7 @@ Do not run two full `run!` jobs against the same table concurrently. Host apps s
|
|
|
67
69
|
|
|
68
70
|
### Run records
|
|
69
71
|
|
|
70
|
-
When a database connection is configured (Active Record or `DATABASE_URL`), `SqlRunRecordStore` creates `
|
|
72
|
+
When a database connection is configured (Active Record or `DATABASE_URL`), `SqlRunRecordStore` creates `partition_gardener_checkpoints` on first use. Checkpoints let incremental rebalance resume after partial failure. Disable with `config.run_record_enabled = false` or per-table `run_record_enabled: false`. Set `config.run_record_table_name` when the host already owns another table name.
|
|
71
73
|
|
|
72
74
|
## Per-table registration
|
|
73
75
|
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# json CVE-2026-71847
|
|
2
|
+
|
|
3
|
+
## Dependency
|
|
4
|
+
|
|
5
|
+
json, pulled in through the development graph (rubocop / activesupport). Root Gemfile.lock already had 2.21.2 after the Dependabot bundler group bump. Appraisal locks gemfiles/rails72.gemfile.lock, gemfiles/rails8ruby34.gemfile.lock, and gemfiles/rails8ruby4.gemfile.lock still resolved 2.20.0.
|
|
6
|
+
|
|
7
|
+
## Symptom
|
|
8
|
+
|
|
9
|
+
bundle-audit check reports GHSA-9hj4-r449-hfvc / CVE-2026-71847 on json 2.20.0 through 2.21.1. The published fix is >= 2.21.2. Dependency audit CI on the 0.3.3 merge failed against the root lock while it still had 2.21.1. After Dependabot updated only Gemfile.lock, later audit jobs on main passed while the appraisal graphs CI actually installs stayed on 2.20.0. The audit workflow find matched files named Gemfile.lock only, so gemfiles/*.gemfile.lock were never scanned.
|
|
10
|
+
|
|
11
|
+
Test CI on those same appraisal locks also failed in frozen mode because they still listed partition_gardener 0.3.2 after VERSION moved to 0.3.3.
|
|
12
|
+
|
|
13
|
+
## Evidence
|
|
14
|
+
|
|
15
|
+
Local bundle-audit check --gemfile-lock on each appraisal lock reported json 2.20.0 and the advisory. GitHub advisory: https://github.com/advisories/GHSA-9hj4-r449-hfvc. Upstream advisory: https://github.com/ruby/json/security/advisories/GHSA-9hj4-r449-hfvc. Failed Dependency audit run on the 0.3.3 merge: GitHub Actions run 31696494175.
|
|
16
|
+
|
|
17
|
+
## Suggested fix
|
|
18
|
+
|
|
19
|
+
Relock the three appraisal gemfiles so json is 2.21.2 and the path gem is 0.3.3. Teach .github/workflows/dependency-audit.yml to scan both Gemfile.lock and *.gemfile.lock.
|
|
20
|
+
|
|
21
|
+
## Next
|
|
22
|
+
|
|
23
|
+
Land the relock on main. Dependabot still watches only the root bundler directory, so appraisal graphs will need a manual relock when the next advisory lands unless that config grows.
|
|
24
|
+
|
|
25
|
+
## Source
|
|
26
|
+
|
|
27
|
+
Upstream: https://github.com/advisories/GHSA-9hj4-r449-hfvc and CVE-2026-71847. Downstream: gemfiles/*.gemfile.lock and .github/workflows/dependency-audit.yml on patch/relock-appraisal-advisory-json.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Occupied monthly children at the active window start
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
When attached monthly children already cover the start of the sliding-window active range, the plan keeps those children and fills the remainder as open, not current. Current is only the filler that starts at the active window origin. The planner reads occupancy from attached range bounds, not only from parsed monthly names.
|
|
10
|
+
|
|
11
|
+
Archive children whose range ends at or before the window start stay archive. Tail slot names current, open, and future are not occupancy. A first-month heat split that is not yet attached already named the later filler open; occupied prefix uses the same name.
|
|
12
|
+
|
|
13
|
+
RFC 0002 records occupancy-first placement. A date-named child that covers a range is that range. Current, open, and future fill only uncovered ranges. Default is residual routing. This patch does not add registry flags. A later RFC 0002 pass dropped ensure flags from the draft and added high-end occupancy for future versus premade months past the window.
|
|
14
|
+
|
|
15
|
+
## Effects
|
|
16
|
+
|
|
17
|
+
A catalog of monthly children through the current month, with default only and no current or future slot, plans open from the next month and attaches without overlapping an existing child. A premade month that begins at the window end stays attached. Future starts after that month.
|
|
18
|
+
|
|
19
|
+
## Next
|
|
20
|
+
|
|
21
|
+
RFC 0002 stays Proposed. Unresolved questions live in the RFC, including integer occupancy. Implementation PRs cite RFC-0002.
|
|
22
|
+
|
|
23
|
+
## Source
|
|
24
|
+
|
|
25
|
+
Requested as a sliding-window plan fix after 0.3.3 skipped only the reverse overlap, archive attach when current already covers that month. Naming follow-up: current must not start after the occupying prefix.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Specify shipped contracts as Stable RFCs
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Write Standards Track Stable RFCs for every shipped contract named in the RFC gap analysis, including layouts and execution paths that were previously operator-guide only. Skip whether-to-partition advice, engine recipes, monitoring SLOs, host CI, job wiring, and landscape UI notes. Skip registry ensure flags. Occupancy for integer catalogs stays an unresolved question, not a claimed walk.
|
|
10
|
+
|
|
11
|
+
RFC numbers: 0003 sliding-window-layout, 0004 registry-json, 0005 plan-audit-reports, 0006 archive-naming, 0007 retention, 0008 cli, 0009 maintenance-backends, 0010 keyset-rebalance, 0011 advisory-lock-run-records, 0012 hot-switch, 0013 child-column-align, 0014 integer-window, 0015 hash-list-composite.
|
|
12
|
+
|
|
13
|
+
Facts preserved: current is origin filler, not this month. Default is always created except hash. Apply always needs --confirm. Moves are insert then delete with ON CONFLICT. partition_key_column is trusted operator SQL.
|
|
14
|
+
|
|
15
|
+
## Effects
|
|
16
|
+
|
|
17
|
+
The RFC files and id claims exist on this branch. Indexes and Related links point at the set.
|
|
18
|
+
|
|
19
|
+
## Next
|
|
20
|
+
|
|
21
|
+
Review Proposed RFC 0002. Follow-on RFC if integer occupancy ships.
|
|
22
|
+
|
|
23
|
+
## Source
|
|
24
|
+
|
|
25
|
+
Requested after RFC 0001 and RFC 0002. Related usr/docs/changelogs/20260909164000_shipped-contract-rfcs.md.
|
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
# Opaque composite public identifiers
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
Andrei
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
Treat database uniqueness, public identifier, routing hint, and authorization as four separate contracts. PostgreSQL uniqueness on a partitioned parent is composite because the partition key must sit in every unique index. That does not by itself decide what a person sees in a URL or API.
|
|
10
|
+
|
|
11
|
+
When a single sequence or UUIDv7 keeps id unique across children, keep that logical id as the application identifier. Use self.primary_key = :id so to_param and JSON do not become Rails underscore composites such as 42_100. Still attach a routing hint on point lookups so SELECT can prune. Do not invent that hint from current month.
|
|
12
|
+
|
|
13
|
+
When id is unique only inside a child, the public identifier for point lookups must carry every uniqueness column. Prefer one opaque path segment over Rails extract_value underscore form or a separate query parameter the caller can edit independently.
|
|
14
|
+
|
|
15
|
+
If that segment is encoded, use unpadded base64url from RFC 4648 section 5, not standard base64. Standard base64 uses plus, slash, and padding equals, which are not URL-safe in a path. Encoding is packaging. Anyone can decode and re-encode another date or tenant. Tamper resistance needs an HMAC over a versioned payload, or a server-side mapping table. Encoding is not authorization. Lookups still go through an ownership set.
|
|
16
|
+
|
|
17
|
+
After decode, query with both columns and fail closed. Do not fall back to id-only scan. A bad or unsigned token is 404, not a bounded retry across children.
|
|
18
|
+
|
|
19
|
+
Do not hide browse context. List screens still show period, account, or branch in product language. Tenant and list keys for hot paths come from session or parent context, not from a tunable URL field. Point-lookup tokens may be opaque. Pagination cursors already belong in the same family as opaque tokens.
|
|
20
|
+
|
|
21
|
+
Do not ship an encoder in this gem until a second in-repo caller needs it. Host applications own secrets, token version, and decode. Gardener docs should name the contract.
|
|
22
|
+
|
|
23
|
+
Rails already covers the packaging the first pass sketched. Prefer those APIs over a homemade base64 string.
|
|
24
|
+
|
|
25
|
+
signed_id (Rails 7+) is an HMAC token over record.id, SHA256, JSON, url_safe: true. The docs say the payload is encoded not encrypted. find_signed fails closed on a bad signature. That is the closest out-of-box match to one opaque URL-safe path segment. It only carries both uniqueness columns when primary_key is the composite array. Scalar self.primary_key = :id plus query_constraints still signs and finds by id alone, so SELECT can scan every child. query_constraints never feeds SELECT.
|
|
26
|
+
|
|
27
|
+
to_param plus params.extract_value is the Rails CPK URL form. Default delimiter is underscore (4_2). Callers can edit either half. Dates and other values that contain underscore split wrong unless param_delimiter is changed. Use this for private admin paths, not for a public identifier that must not be tunable.
|
|
28
|
+
|
|
29
|
+
generates_token_for can embed extra JSON (password salt, or occurred_on) and re-check it after fetch. Lookup still yields only the primary key into find_by or find. Extra JSON is not a WHERE predicate and is plaintext in the token. CPK find_by_token_for was fixed in 2024 (Rails 7.2/8.0). find_signed CPK wrap landed later (Rails 8.1 / 2026-04). Gardener's development floor is ActiveRecord 7.1, so hosts on 7.1 cannot treat signed CPK find as solved.
|
|
30
|
+
|
|
31
|
+
has_secure_token stores a random Base58 string (URL-safe by alphabet). It is a mapping column, not an encoding of composite values. PostgreSQL still forbids a unique index on token alone on a partitioned parent unless the partition key is in that index. Lookup by token without the partition key then cannot prune, or the unique index cannot exist. Put the mapping on an unpartitioned table, or keep a globally unique logical id on the fact row.
|
|
32
|
+
|
|
33
|
+
GlobalID locates with Model.find(model_id). Composite keys become gid://app/Event/123/2026-03-15 (slash, CGI.escape, max 20 segments). Unsigned. to_param is unpadded base64url of that URI, still decodable. SignedGlobalID HMAC-wraps the same URI. Locator find prunes only when primary_key is the composite.
|
|
34
|
+
|
|
35
|
+
ActiveSupport::MessageVerifier is the primitive under signed_id, token_for, and SignedGlobalID. Default MessageVerifier is not URL-safe (plus, slash, padding). signed_id and GlobalID::Verifier opt into url_safe. MessageEncryptor exists; do not encrypt identifiers. CurrentAttributes can hold session tenant for list filters; that is not a public id.
|
|
36
|
+
|
|
37
|
+
## Effects
|
|
38
|
+
|
|
39
|
+
Added Public identifiers to docs/application_contract.md. Pointers from docs/partition_landscape.md Rails application contract, routing hints, and UI surfaces. README application_contract blurb names public identifiers. CHANGELOG Unreleased and usr/docs/changelogs/20260916144600_public-identifiers-contract.md record the doc contract.
|
|
40
|
+
|
|
41
|
+
## Next
|
|
42
|
+
|
|
43
|
+
Host docs only. No encoder in the gem. No RFC unless a later change makes public identifiers a user-facing gem contract.
|
|
44
|
+
|
|
45
|
+
## Source
|
|
46
|
+
|
|
47
|
+
Triggered by a review question: should composite ids for users be a single base64 string of composite values so callers cannot easily tune them and so the value stays URL-safe.
|
|
48
|
+
|
|
49
|
+
Later pass accepted the proposal as host documentation. Vendor product names stay out of the host contract; they remain a Source citation only.
|
|
50
|
+
|
|
51
|
+
Existing repo text: docs/partition_landscape.md Opaque tokens may encode (id, partition_key); Rails query_constraints; UI and product surfaces require visible period or tenant for browse. docs/application_contract.md query_constraints when logical id is not globally unique.
|
|
52
|
+
|
|
53
|
+
Materialized in docs/application_contract.md Public identifiers, docs/partition_landscape.md pointers, README application_contract blurb.
|
|
54
|
+
|
|
55
|
+
External: GitLab Database table partitioning, Rails Composite Primary Keys guide to_param 4_2 and extract_value, ActiveRecord::SignedId (url_safe MessageVerifier, find_by primary_key => [id] on main), ActiveRecord::TokenFor payload_for [id, block.as_json], has_secure_token SecureRandom.base58, URI::GID COMPOSITE_MODEL_ID_DELIMITER slash, GlobalID to_param unpadded base64url, Rails PRs 52798 and 57245, RFC 4648 section 5 base64url, GraphQL Relay global object identification, OWASP Insecure Direct Object Reference Prevention Cheat Sheet.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
CI quality failed on a trailing blank line in spec/support/junit_formatter.rb. Test CI also ran bundler-audit or osv-scanner, which fail whenever an advisory lands even when a separate dependency audit already exists.
|
|
2
|
+
|
|
3
|
+
## Participants
|
|
4
|
+
|
|
5
|
+
- amkisko
|
|
6
|
+
|
|
7
|
+
## Decisions
|
|
8
|
+
|
|
9
|
+
- Drop bundler-audit and security jobs from test.yml so tests no longer wait on advisory freshness.
|
|
10
|
+
- Keep dependency-audit.yml as workflow_dispatch only. Do not run it on push, pull request, or schedule.
|
|
11
|
+
- Disable osv-scanner in Trunk Check.
|
|
12
|
+
- Disable markdownlint MD013 in .markdownlint.yaml. Keep AGENTS.md in Trunk so other markdownlint rules still run.
|
|
13
|
+
- Keep lockfile and spec fixes that were already blocking quality or tests.
|
|
14
|
+
|
|
15
|
+
## Effects
|
|
16
|
+
|
|
17
|
+
- Test CI runs lint and specs without an advisory gate.
|
|
18
|
+
- Trunk Check no longer treats dependency CVEs as a required check.
|
|
19
|
+
- Manual dependency-audit remains available on workflow_dispatch.
|
|
20
|
+
|
|
21
|
+
## Next
|
|
22
|
+
|
|
23
|
+
- Run dependency audit on demand when a release or a known advisory needs it.
|
|
24
|
+
- Do not reintroduce advisory scanners into test.yml.
|
|
25
|
+
|
|
26
|
+
## Source
|
|
27
|
+
|
|
28
|
+
- GitHub Actions test and Trunk Check failures on 2026-09-17
|
data/docs/operations.md
CHANGED
|
@@ -83,7 +83,7 @@ table is not a partitioned table — registry typo or wrong connection. Action:
|
|
|
83
83
|
|
|
84
84
|
## Failed or partial runs
|
|
85
85
|
|
|
86
|
-
Gardener persists checkpoints in `
|
|
86
|
+
Gardener persists checkpoints in `partition_gardener_checkpoints` when `run_record_enabled` is true (default). Override the table with `run_record_table_name`.
|
|
87
87
|
|
|
88
88
|
1. Read `errors` and notifier context (`table_name`, `action`).
|
|
89
89
|
2. `audit TABLE` — confirm default count and gaps unchanged or worse.
|
data/docs/partition_landscape.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
Partition Gardener targets PostgreSQL native declarative partitioning with gardener-owned maintenance plans. This page maps industry patterns to what the gem implements, documents experimental layouts, and lists what belongs outside the gem.
|
|
4
4
|
|
|
5
|
-
Canonical home for routing (layers, recovery ladder, hints), pruning, UI scoping, and aggregate snapshots. Host query and
|
|
5
|
+
Canonical home for routing (layers, recovery ladder, hints), pruning, UI scoping, and aggregate snapshots. Host query, write, and public identifier obligations live in [application_contract.md](application_contract.md). Cross-engine portability lives in [partition_engines.md](partition_engines.md).
|
|
6
6
|
|
|
7
7
|
## Implemented templates
|
|
8
8
|
|
|
@@ -113,6 +113,8 @@ end
|
|
|
113
113
|
|
|
114
114
|
`query_constraints` (Rails 7.1+) adds listed columns to `UPDATE` and `DELETE` `WHERE` clauses so writes can prune. It does not add the partition key to `SELECT` lookups: `Event.find(id)` may still scan all partitions unless `id` is globally unique and the planner can prove a single child.
|
|
115
115
|
|
|
116
|
+
Public URLs and API ids are a separate contract from `query_constraints`. See [application_contract.md](application_contract.md#public-identifiers).
|
|
117
|
+
|
|
116
118
|
For reads, encode the contract in scopes used by hot paths:
|
|
117
119
|
|
|
118
120
|
```ruby
|
|
@@ -150,7 +152,7 @@ Recovery ladder (prefer earlier steps):
|
|
|
150
152
|
#### Exact key (preferred)
|
|
151
153
|
|
|
152
154
|
- Denormalize the parent business date (or tenant/branch) onto the child at insert time; filter and join on the child column.
|
|
153
|
-
- Store the partition key next to the logical id in API cursors, job args, outbox payloads, and admin deep links. Opaque tokens may encode `(id, partition_key)` so clients never send id-only.
|
|
155
|
+
- Store the partition key next to the logical id in API cursors, job args, outbox payloads, and admin deep links. Opaque tokens may encode `(id, partition_key)` so clients never send id-only ([application_contract.md](application_contract.md#public-identifiers)).
|
|
154
156
|
- Use a generated column when the key is a stable expression of an existing column the planner can see (`created_at::date` registered and filtered the same way).
|
|
155
157
|
|
|
156
158
|
#### Orientation / range from a related record
|
|
@@ -218,7 +220,7 @@ Registry `conflict_key` must match a parent unique index and should include the
|
|
|
218
220
|
|
|
219
221
|
### UI and product surfaces
|
|
220
222
|
|
|
221
|
-
Partition boundaries are an application contract, not only a database detail. Screens and APIs should make the same separation key visible to people so routine work stays inside one prune-friendly slice. User-facing copy should describe periods, accounts, or branches in product language, not partitions, shards, or child tables.
|
|
223
|
+
Partition boundaries are an application contract, not only a database detail. Screens and APIs should make the same separation key visible to people so routine work stays inside one prune-friendly slice. User-facing copy should describe periods, accounts, or branches in product language, not partitions, shards, or child tables. Point-lookup URLs may use an opaque token; list and filter screens still show the period or tenant ([application_contract.md](application_contract.md#public-identifiers)).
|
|
222
224
|
|
|
223
225
|
Time-partitioned data (monthly sliding window):
|
|
224
226
|
|
|
@@ -410,7 +412,7 @@ Operations and migration docs in this repository:
|
|
|
410
412
|
|
|
411
413
|
- [operations.md](operations.md) — runbook
|
|
412
414
|
- [cutover.md](cutover.md) — hot-switch playbook
|
|
413
|
-
- [application_contract.md](application_contract.md) — host app behavior
|
|
415
|
+
- [application_contract.md](application_contract.md) — host app behavior, public identifiers
|
|
414
416
|
- [monitoring.md](monitoring.md) — metrics and alerts
|
|
415
417
|
- [retention.md](retention.md) — archive detach and drop
|
|
416
418
|
- [audit_reference.md](audit_reference.md) — audit and plan catalog
|
|
@@ -15,7 +15,7 @@ module PartitionGardener
|
|
|
15
15
|
:strict_maintenance_backend_validation,
|
|
16
16
|
:current_run_metrics
|
|
17
17
|
|
|
18
|
-
attr_writer :run_record_store
|
|
18
|
+
attr_writer :run_record_store, :run_record_table_name
|
|
19
19
|
|
|
20
20
|
def initialize
|
|
21
21
|
@notifier = ->(_message_or_error, context: {}) {}
|
|
@@ -28,6 +28,7 @@ module PartitionGardener
|
|
|
28
28
|
@analyze_after_rebalance = false
|
|
29
29
|
@incremental_rebalance = true
|
|
30
30
|
@run_record_enabled = true
|
|
31
|
+
@run_record_table_name = SqlRunRecordStore::TABLE_NAME
|
|
31
32
|
@run_record_store = nil
|
|
32
33
|
@retention_detach_concurrently = false
|
|
33
34
|
@align_child_columns = true
|
|
@@ -39,6 +40,13 @@ module PartitionGardener
|
|
|
39
40
|
@run_record_store ||= default_run_record_store
|
|
40
41
|
end
|
|
41
42
|
|
|
43
|
+
def run_record_table_name
|
|
44
|
+
name = @run_record_table_name.to_s.strip
|
|
45
|
+
raise ArgumentError, "run_record_table_name is blank" if name.empty?
|
|
46
|
+
|
|
47
|
+
name
|
|
48
|
+
end
|
|
49
|
+
|
|
42
50
|
def connection
|
|
43
51
|
resolved = @connection_resolver.call
|
|
44
52
|
raise ArgumentError, "PartitionGardener connection is not configured" unless resolved
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
module PartitionGardener
|
|
2
2
|
class SqlRunRecordStore
|
|
3
|
-
TABLE_NAME = "
|
|
3
|
+
TABLE_NAME = "partition_gardener_checkpoints"
|
|
4
|
+
LEGACY_TABLE_NAMES = %w[partition_gardener_runs partition_gardener_run_records].freeze
|
|
4
5
|
|
|
5
6
|
def initialize
|
|
6
7
|
@schema_mutex = Mutex.new
|
|
@@ -40,6 +41,7 @@ module PartitionGardener
|
|
|
40
41
|
@schema_mutex.synchronize do
|
|
41
42
|
return if @schema_ready
|
|
42
43
|
|
|
44
|
+
migrate_legacy_table!
|
|
43
45
|
connection.execute(create_table_sql)
|
|
44
46
|
@schema_ready = true
|
|
45
47
|
end
|
|
@@ -55,17 +57,50 @@ module PartitionGardener
|
|
|
55
57
|
connection.quote_table_name(name)
|
|
56
58
|
end
|
|
57
59
|
|
|
60
|
+
def storage_table_name
|
|
61
|
+
PartitionGardener.configuration.run_record_table_name
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def migrate_legacy_table!
|
|
65
|
+
return unless storage_table_name == TABLE_NAME
|
|
66
|
+
return if catalog_relation_exists?(storage_table_name)
|
|
67
|
+
|
|
68
|
+
LEGACY_TABLE_NAMES.each do |legacy_name|
|
|
69
|
+
next if legacy_name == storage_table_name
|
|
70
|
+
next unless catalog_relation_exists?(legacy_name)
|
|
71
|
+
|
|
72
|
+
connection.execute(
|
|
73
|
+
"ALTER TABLE #{quoted_table(legacy_name)} RENAME TO #{quoted_table(storage_table_name)}"
|
|
74
|
+
)
|
|
75
|
+
break
|
|
76
|
+
end
|
|
77
|
+
end
|
|
78
|
+
|
|
79
|
+
def catalog_relation_exists?(name)
|
|
80
|
+
sql = <<~SQL
|
|
81
|
+
SELECT 1 AS present
|
|
82
|
+
FROM pg_catalog.pg_class AS classes
|
|
83
|
+
INNER JOIN pg_catalog.pg_namespace AS namespaces
|
|
84
|
+
ON namespaces.oid = classes.relnamespace
|
|
85
|
+
WHERE namespaces.nspname = #{connection.quote(PartitionGardener.configuration.schema_name)}
|
|
86
|
+
AND classes.relname = #{connection.quote(name)}
|
|
87
|
+
AND classes.relkind IN ('r', 'p')
|
|
88
|
+
SQL
|
|
89
|
+
|
|
90
|
+
Blank.present?(connection.execute(sql).first)
|
|
91
|
+
end
|
|
92
|
+
|
|
58
93
|
def load_sql(table_name)
|
|
59
94
|
<<~SQL
|
|
60
95
|
SELECT table_name, phase, plan_signature, staging_row_count
|
|
61
|
-
FROM #{quoted_table(
|
|
96
|
+
FROM #{quoted_table(storage_table_name)}
|
|
62
97
|
WHERE table_name = #{connection.quote(table_name)}
|
|
63
98
|
SQL
|
|
64
99
|
end
|
|
65
100
|
|
|
66
101
|
def save_sql(table_name, attributes)
|
|
67
102
|
<<~SQL
|
|
68
|
-
INSERT INTO #{quoted_table(
|
|
103
|
+
INSERT INTO #{quoted_table(storage_table_name)} (
|
|
69
104
|
table_name, phase, plan_signature, staging_row_count, updated_at
|
|
70
105
|
) VALUES (
|
|
71
106
|
#{connection.quote(table_name)},
|
|
@@ -84,14 +119,14 @@ module PartitionGardener
|
|
|
84
119
|
|
|
85
120
|
def clear_sql(table_name)
|
|
86
121
|
<<~SQL
|
|
87
|
-
DELETE FROM #{quoted_table(
|
|
122
|
+
DELETE FROM #{quoted_table(storage_table_name)}
|
|
88
123
|
WHERE table_name = #{connection.quote(table_name)}
|
|
89
124
|
SQL
|
|
90
125
|
end
|
|
91
126
|
|
|
92
127
|
def create_table_sql
|
|
93
128
|
<<~SQL
|
|
94
|
-
CREATE TABLE IF NOT EXISTS #{quoted_table(
|
|
129
|
+
CREATE TABLE IF NOT EXISTS #{quoted_table(storage_table_name)} (
|
|
95
130
|
table_name text PRIMARY KEY,
|
|
96
131
|
phase text NOT NULL,
|
|
97
132
|
plan_signature text NOT NULL,
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: partition_gardener
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.3.
|
|
4
|
+
version: 0.3.6
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Andrei Makarov
|
|
@@ -107,20 +107,6 @@ dependencies:
|
|
|
107
107
|
- - "~>"
|
|
108
108
|
- !ruby/object:Gem::Version
|
|
109
109
|
version: '2.0'
|
|
110
|
-
- !ruby/object:Gem::Dependency
|
|
111
|
-
name: rspec_junit_formatter
|
|
112
|
-
requirement: !ruby/object:Gem::Requirement
|
|
113
|
-
requirements:
|
|
114
|
-
- - "~>"
|
|
115
|
-
- !ruby/object:Gem::Version
|
|
116
|
-
version: '0.6'
|
|
117
|
-
type: :development
|
|
118
|
-
prerelease: false
|
|
119
|
-
version_requirements: !ruby/object:Gem::Requirement
|
|
120
|
-
requirements:
|
|
121
|
-
- - "~>"
|
|
122
|
-
- !ruby/object:Gem::Version
|
|
123
|
-
version: '0.6'
|
|
124
110
|
- !ruby/object:Gem::Dependency
|
|
125
111
|
name: standard
|
|
126
112
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -280,11 +266,28 @@ files:
|
|
|
280
266
|
- docs/audit_reference.md
|
|
281
267
|
- docs/background_job.md
|
|
282
268
|
- docs/changelogs/20260709120900_partition-engines-guide.md
|
|
269
|
+
- docs/changelogs/20260813142300_rebalance-before-archive-attach.md
|
|
270
|
+
- docs/changelogs/20260813144516_relock-appraisal-advisory-json.md
|
|
271
|
+
- docs/changelogs/20260902114500_align-child-columns-before-attach.md
|
|
272
|
+
- docs/changelogs/20260902124500_cut-patch-release-0-3-4.md
|
|
273
|
+
- docs/changelogs/20260909143500_occupied-window-current-origin.md
|
|
274
|
+
- docs/changelogs/20260909154900_tail-slot-rfc.md
|
|
275
|
+
- docs/changelogs/20260909162554_high-end-future-occupancy.md
|
|
276
|
+
- docs/changelogs/20260909164000_shipped-contract-rfcs.md
|
|
277
|
+
- docs/changelogs/20260909165500_cut-patch-release-0-3-5.md
|
|
278
|
+
- docs/changelogs/20260916144600_public-identifiers-contract.md
|
|
279
|
+
- docs/changelogs/20261005135700_checkpoint-table-name.md
|
|
280
|
+
- docs/changelogs/20261005141621_cut-patch-release-0-3-6.md
|
|
283
281
|
- docs/cli.md
|
|
284
282
|
- docs/configuration.md
|
|
285
283
|
- docs/cutover.md
|
|
286
284
|
- docs/decision_flow.md
|
|
285
|
+
- docs/dependencies/20260813144516_json-cve-2026-71847.md
|
|
287
286
|
- docs/host_testing.md
|
|
287
|
+
- docs/issues/20260909143500_occupied-window-current-origin.md
|
|
288
|
+
- docs/issues/20260909164000_shipped-contract-rfcs.md
|
|
289
|
+
- docs/issues/20260916113300_opaque-composite-public-ids.md
|
|
290
|
+
- docs/issues/20260917123253_ci-drop-automatic-dependency-audit.md
|
|
288
291
|
- docs/monitoring.md
|
|
289
292
|
- docs/naming.md
|
|
290
293
|
- docs/operations.md
|