sqlever 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +202 -0
- package/README.md +9 -0
- package/package.json +19 -0
- package/spec/SPEC-CHANGELOG.md +309 -0
- package/spec/SPEC.md +1521 -0
- package/src/cli.ts +41 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
package/README.md
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
# sqlever
|
|
2
|
+
|
|
3
|
+
A drop-in replacement for [Sqitch](https://sqitch.org/) — PostgreSQL-only, TypeScript/Bun, with static analysis and zero-downtime migration primitives.
|
|
4
|
+
|
|
5
|
+
**Status:** Early development. See [spec/SPEC.md](spec/SPEC.md) for the full design.
|
|
6
|
+
|
|
7
|
+
## License
|
|
8
|
+
|
|
9
|
+
Apache 2.0
|
package/package.json
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "sqlever",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Sqitch-compatible PostgreSQL migration tool",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"bin": {
|
|
7
|
+
"sqlever": "./src/cli.ts"
|
|
8
|
+
},
|
|
9
|
+
"scripts": {
|
|
10
|
+
"build": "bun build src/cli.ts --compile --outfile dist/sqlever",
|
|
11
|
+
"test": "bun test"
|
|
12
|
+
},
|
|
13
|
+
"devDependencies": {
|
|
14
|
+
"bun-types": "latest"
|
|
15
|
+
},
|
|
16
|
+
"dependencies": {
|
|
17
|
+
"pg": "^8.11.0"
|
|
18
|
+
}
|
|
19
|
+
}
|
|
@@ -0,0 +1,309 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to the sqlever spec and codebase will be documented here.
|
|
4
|
+
|
|
5
|
+
## [Unreleased]
|
|
6
|
+
|
|
7
|
+
## [SPEC 0.9] — 2026-03-20
|
|
8
|
+
|
|
9
|
+
- Renamed project from sqevo to sqlever.
|
|
10
|
+
|
|
11
|
+
## [SPEC 0.8] — 2026-03-20
|
|
12
|
+
|
|
13
|
+
- Renamed project from stitch to sqevo.
|
|
14
|
+
|
|
15
|
+
## [SPEC 0.7.1] — 2026-03-20
|
|
16
|
+
|
|
17
|
+
Customer zero validation: PostgresAI Console (`postgres-ai/platform-ui/db`).
|
|
18
|
+
|
|
19
|
+
- Added `postgres-ai-console/` to test fixture corpus as named customer zero fixture
|
|
20
|
+
- Added edge case fixtures: `planner-edge-cases/`, `missing-verify/`, `non-revertable/`, `heavy-includes/`
|
|
21
|
+
- Specified missing verify script behavior: `--verify` skips gracefully when verify file absent
|
|
22
|
+
- Specified non-revertable migration handling: log failure, record `fail` event
|
|
23
|
+
- Strengthened Problem 5 with real-world example (130+ shared files included via `\i`)
|
|
24
|
+
|
|
25
|
+
## [SPEC 0.7] — 2026-03-20
|
|
26
|
+
|
|
27
|
+
- **DD12 RESOLVED: Shell out to psql.** 100% psql metacommand compatibility — no subset, no reimplementation. `\i`, `\ir`, `\set`, `\copy`, `\if` all work exactly as in Sqitch. `node-postgres` used only for sqlever's own DB operations (tracking tables, advisory locks, introspection, batch DML). This was the last major OPEN blocking implementation.
|
|
28
|
+
|
|
29
|
+
## [SPEC 0.6.2] — 2026-03-20
|
|
30
|
+
|
|
31
|
+
- **DD9: 3-partition queue WITH SKIP LOCKED, not vs.** These are complementary, not alternatives. Partition rotation solves bloat (TRUNCATE vs DELETE). SKIP LOCKED solves worker concurrency (lock-free dequeue). Removed the OPEN marker — the design is sound when both are used together as PGQ intended.
|
|
32
|
+
|
|
33
|
+
## [SPEC 0.6.1] — 2026-03-20
|
|
34
|
+
|
|
35
|
+
Round 5 convergence check. All four expert domains confirmed CONVERGED.
|
|
36
|
+
|
|
37
|
+
- Removed incorrect `dependencies_check` CHECK constraint from `sqitch.dependencies` DDL — actual Sqitch has no CHECK constraint on this table. `dependency_id` can be NULL for cross-project dependencies regardless of type, and NOT NULL for conflicts referencing known changes.
|
|
38
|
+
|
|
39
|
+
## [SPEC 0.6] — 2026-03-20
|
|
40
|
+
|
|
41
|
+
Round 4 expert review findings addressed (convergence check). Same four reviewers. Surgical fixes — the spec is nearly converged. All reviewers confirmed zero critical behavioral issues with PG claims, no contradictions, and all OPEN markers remain appropriate.
|
|
42
|
+
|
|
43
|
+
### Critical fixes
|
|
44
|
+
|
|
45
|
+
- **Change ID `parent` line corrected:** The `parent` line in the change ID info string applies to ALL changes that have a preceding change in the plan file, not just reworked changes. Every change except the first has a parent (the immediately preceding change). The v0.5 spec incorrectly stated "only if this is a reworked change" — this would have produced incorrect change IDs for every change after the first in any plan with 2+ changes.
|
|
46
|
+
|
|
47
|
+
- **`dependencies.change_id` FK: added `ON DELETE CASCADE`:** Without this, reverting a change (which deletes from `sqitch.changes`) fails with a foreign key violation on the dependencies table.
|
|
48
|
+
|
|
49
|
+
### Important fixes
|
|
50
|
+
|
|
51
|
+
- **`dependencies.dependency_id` FK added:** The `dependency_id` column now has `REFERENCES sqitch.changes(change_id) ON UPDATE CASCADE`, matching Sqitch's actual DDL. Without this, a `require` dependency could reference a non-existent change ID. The CHECK constraint is now named `dependencies_check`.
|
|
52
|
+
|
|
53
|
+
- **Tag info string `@` prefix clarified:** Tag `format_name` prepends `@` to the tag name. The info string line is `tag @v1.0`, not `tag v1.0`. Incorrect tag IDs would result from omitting the prefix.
|
|
54
|
+
|
|
55
|
+
- **SA016 lock level corrected (again):** `ADD CHECK` takes `ShareLock` on PG < 16, NOT `AccessExclusiveLock`. The v0.5 spec overcorrected from v0.4. `ShareLock` still blocks writes, so the rule recommendation (use `NOT VALID`) and severity (`error`) remain correct.
|
|
56
|
+
|
|
57
|
+
- **Advisory lock unlock on all exit paths:** Added explicit note that `pg_advisory_unlock` must be called on ALL exit paths (success, failure, analysis abort), not just the happy path. Disconnect-based release is the safety net for crashes, not the primary unlock mechanism.
|
|
58
|
+
|
|
59
|
+
- **`sqlever.pending_changes` protected by deploy advisory lock:** Added explicit note linking the session-level advisory lock to `sqlever.pending_changes` — the lock prevents concurrent access to pending records.
|
|
60
|
+
|
|
61
|
+
### Minor fixes
|
|
62
|
+
|
|
63
|
+
- SA003: clarified that `USING` clause presence always triggers SA003 regardless of the safe cast allowlist
|
|
64
|
+
- SA001: confirmed it does NOT fire when a `DEFAULT` is present (that case is SA002/SA002b territory)
|
|
65
|
+
- `sqlever.*` schema creation: clarified that the `sqlever` schema is created on first non-transactional deploy (not just expand/contract and batched DML)
|
|
66
|
+
- Version bumped to 0.6
|
|
67
|
+
|
|
68
|
+
## [SPEC 0.5] — 2026-03-20
|
|
69
|
+
|
|
70
|
+
Round 3 expert review findings addressed. Same four reviewers: PG internals expert, Sqitch power user, production SRE, static analysis engineer. Additional research: pg_index_pilot architecture analysis. Focus on correcting ID algorithms verified against Sqitch source, fixing tracking schema to match actual Sqitch DDL, and resolving advisory lock API mismatch.
|
|
71
|
+
|
|
72
|
+
### Critical fixes
|
|
73
|
+
|
|
74
|
+
- **Change ID algorithm corrected:** The v0.4 format was wrong. Fixed to match `App::Sqitch::Plan::Change->info` exactly: `uri` line conditional on `%uri` pragma, `parent` line for reworked changes, requires/conflicts use section headers with indented ` + dep` / ` - dep` entries (not `require <dep>` per line), note is raw text after a blank line (not `note <text>`). Every field format difference produces a different SHA-1.
|
|
75
|
+
|
|
76
|
+
- **Tag ID algorithm corrected:** Fixed to match `App::Sqitch::Plan::Tag->info`: added conditional `uri` line and conditional note (raw text after blank line).
|
|
77
|
+
|
|
78
|
+
- **script_hash is plain SHA-1, NOT git-style:** Sqitch does `SHA1(raw_file_content)` with NO `"blob <size>\0"` prefix. The v0.4 spec incorrectly claimed git-style blob hashing. Fixed. Clarified that for reworked changes, the hash is computed from the file at deploy time.
|
|
79
|
+
|
|
80
|
+
### Important fixes
|
|
81
|
+
|
|
82
|
+
- **Tracking schema corrections:** Added missing `sqitch.releases` table (registry versioning). Fixed `projects.uri` to include `UNIQUE` constraint. Changed all default timestamps from `NOW()` to `clock_timestamp()` (wall-clock time, advances within transaction). Added `ON UPDATE CASCADE` on all foreign key references. Added `UNIQUE(project, script_hash)` on changes table. Fixed events table — it DOES have `PRIMARY KEY (change_id, committed_at)`. Added `merge` to events type CHECK constraint. Added missing CHECK constraint on dependencies table. Reordered DDL to show projects first (referenced by other tables).
|
|
83
|
+
|
|
84
|
+
- **`pg_advisory_lock` replaced with `pg_try_advisory_lock`:** Default mode is now non-blocking — `pg_try_advisory_lock()` returns false immediately if lock held, matching "exit 4" behavior. Added configurable `advisory_lock_timeout` (default 30s) for CI wait mode using `pg_advisory_lock` with `SET lock_timeout`. Default: try, fail fast.
|
|
85
|
+
|
|
86
|
+
- **`hashtext()` instability resolved:** Replaced `hashtext('sqlever_deploy_' || project_name)` with application-level hash. Lock key uses two-argument form `pg_advisory_lock(constant_namespace, project_hash)` with a stable application-computed hash. `hashtext()` output is not guaranteed stable across PG major versions.
|
|
87
|
+
|
|
88
|
+
- **SET LOCAL trigger guard replaced:** `SET LOCAL sqlever.syncing = 'true'` remains true for the entire transaction, suppressing ALL subsequent trigger fires. Replaced with `pg_trigger_depth()` scoped to sqlever triggers via `TG_NAME LIKE 'sqlever_sync_%'`. All sqlever-generated sync triggers must use the `sqlever_sync_` name prefix.
|
|
89
|
+
|
|
90
|
+
- **SA016 lock level corrected:** Reverted PG < 16 lock from `ShareLock` to `AccessExclusiveLock`. `ADD CONSTRAINT ... CHECK` (with immediate validation) takes `AccessExclusiveLock` on PG < 16, `ShareUpdateExclusiveLock` on PG 16+.
|
|
91
|
+
|
|
92
|
+
- **`--mode all` is NOT a single transaction in Sqitch:** Sqitch's `_deploy_all` uses per-change transactions with explicit revert on failure, NOT a single wrapping transaction. Documented accurately. sqlever's true single-transaction `--mode all` is a sqlever improvement.
|
|
93
|
+
|
|
94
|
+
- **`-- sqitch-no-transaction` does NOT exist in Sqitch:** No evidence found in Sqitch source. Changed to `-- sqlever:no-transaction` as a sqlever-only convention. SA020 reference updated.
|
|
95
|
+
|
|
96
|
+
- **Sqitch uses `LOCK TABLE changes IN EXCLUSIVE MODE`, not advisory locks:** Documented that sqlever's advisory lock approach is a sqlever improvement providing stronger coordination (spans full deploy session vs. per-transaction table lock).
|
|
97
|
+
|
|
98
|
+
- **`sqlever.pending_changes` schema defined:** DDL specified: `change_id`, `change_name`, `project`, `script_path`, `started_at`, `status` (pending/complete/failed), `error_message`.
|
|
99
|
+
|
|
100
|
+
- **Non-transactional verify logic specified:** For index operations, check `pg_index.indisvalid`. For other DDL, run the change's verify script. Documented that automated verification only works for known DDL patterns.
|
|
101
|
+
|
|
102
|
+
- **Batch worker heartbeat added:** `heartbeat_at` column updated each batch. Configurable staleness threshold (default 5 minutes). Dead workers detected and jobs marked failed.
|
|
103
|
+
|
|
104
|
+
- **pg_index_pilot added to prior art:** Key patterns: write-ahead tracking (`in_progress` → `completed` | `failed`) for crash recovery, advisory lock using `pg_try_advisory_lock`, invalid index cleanup via `pg_index.indisvalid`. Added to Section 1 and prior art summary table.
|
|
105
|
+
|
|
106
|
+
- **Hybrid rule interface convention documented:** Hybrid rules check `context.db !== undefined` internally. Suppression filtering happens in analyzer entry point after rules return findings. Rules may produce multiple findings from one statement.
|
|
107
|
+
|
|
108
|
+
- **`--mode all` + non-transactional partial state documented:** Non-transactional changes that committed before a later failure remain deployed. `sqlever status` reports partial state correctly.
|
|
109
|
+
|
|
110
|
+
- **`--strict` and `error_on_warn` relationship documented:** `--strict` is the CLI equivalent of `error_on_warn = true` in config.
|
|
111
|
+
|
|
112
|
+
### Minor fixes
|
|
113
|
+
|
|
114
|
+
- SA009: corrected lock on referenced table to "brief" (still blocks concurrent DDL)
|
|
115
|
+
- script_hash for reworked changes: clarified it's computed from the file at deploy time
|
|
116
|
+
- `SHOW pool_mode`: documented as best-effort detection, `connection_type` config is the reliable mechanism
|
|
117
|
+
- GitLab Code Quality severity mapping specified: `error` → `critical`, `warn` → `major`, `info` → `minor`
|
|
118
|
+
- GitLab fingerprint specified: SHA-1 of `(ruleId, filePath, line)`
|
|
119
|
+
- Unused suppression warnings: `-- sqlever:disable` matching no finding produces a warning
|
|
120
|
+
- SA001: removed confusing parenthetical about PG < 11 defaults (that case is SA002b's territory)
|
|
121
|
+
- Change ID requires/conflicts: documented they preserve declaration order (not sorted)
|
|
122
|
+
|
|
123
|
+
### Other changes
|
|
124
|
+
|
|
125
|
+
- Version bumped to 0.5
|
|
126
|
+
- Remaining OPEN markers (4): SA003 safe-cast list (needs `pg_cast` audit), logical replication + expand/contract, PGQ vs SKIP LOCKED, DD12 psql vs node-postgres
|
|
127
|
+
|
|
128
|
+
## [SPEC 0.4] — 2026-03-20
|
|
129
|
+
|
|
130
|
+
Round 2 expert review findings addressed. Four reviewers: PG internals expert, Sqitch power user, production SRE, static analysis engineer. Focus on resolving contradictions, closing OPEN markers, and correcting factual errors.
|
|
131
|
+
|
|
132
|
+
### Critical fixes
|
|
133
|
+
|
|
134
|
+
- **Advisory lock design resolved:** Resolved xact vs session lock contradiction. Use `pg_advisory_lock` (session-level) as default — only option that works across multi-transaction deploys (`--mode change`) and non-transactional changes. Lock key: `pg_advisory_lock(hashtext('sqlever_deploy_' || project_name))`. Require direct connections for deploy (not PgBouncer in transaction mode). Apply to revert/rebase/checkout too, not just deploy.
|
|
135
|
+
|
|
136
|
+
- **`now()` is STABLE, not VOLATILE:** Removed `now()` from SA002 volatile examples — `now()` returns transaction start time and is classified STABLE. Corrected Problem 1 example text. Correct volatile examples: `random()`, `gen_random_uuid()`, `clock_timestamp()`, `txid_current()`. Updated SA002 test fixtures accordingly.
|
|
137
|
+
|
|
138
|
+
- **Change ID algorithm documented:** Closed OPEN marker. Algorithm: SHA-1 of `"change <length>\0project <project>\nchange <name>\nnote <note>\nplanner <planner> <<email>>\ndate <date>\nrequire <dep>\nconflict <dep>\n\n"`. Source: `App::Sqitch::Plan::Change->id`.
|
|
139
|
+
|
|
140
|
+
- **Tag ID computation added:** New addition. SHA-1 of `"tag <length>\0project <project>\ntag <tag_name>\nchange <change_id>\nplanner <planner> <<email>>\ndate <date>\n\n"`.
|
|
141
|
+
|
|
142
|
+
- **script_hash algorithm documented:** Closed OPEN marker. Uses git-style blob hashing: `SHA-1("blob <size>\0<content>")`. Raw file bytes, no line-ending normalization.
|
|
143
|
+
|
|
144
|
+
### Important fixes
|
|
145
|
+
|
|
146
|
+
- **SA003 USING clause corrected:** Changed from "always requires a rewrite" to "PostgreSQL rewrites the table to evaluate the expression, even when types are binary-compatible." Added missing safe casts: `char(N)` to `varchar`/`text`, `numeric(P,S)` to unconstrained `numeric`. Documented that `int` to `bigint` is NOT safe (rewrite required).
|
|
147
|
+
|
|
148
|
+
- **SA016 lock level corrected:** Fixed from `AccessExclusiveLock` to `ShareLock` (PG < 16) / `ShareUpdateExclusiveLock` (PG 16+).
|
|
149
|
+
|
|
150
|
+
- **Hybrid rule classification introduced:** New `type: "hybrid"` for rules with both static and connected concerns. SA009 (static: NOT VALID detection; connected: index check), SA017 (static: fire on SET NOT NULL; connected: check for CHECK constraint), SA018 (static: fire on ADD PRIMARY KEY; connected: check for pre-existing index).
|
|
151
|
+
|
|
152
|
+
- **`--mode all` + non-transactional behavior specified:** Non-transactional changes break the transaction (COMMIT before, execute, BEGIN after). Warning emitted when `--mode all` used with non-transactional changes.
|
|
153
|
+
|
|
154
|
+
- **Non-transactional write-ahead tracking:** Before executing non-transactional DDL, write "pending" record to `sqlever.pending_changes`. After success, update to "complete" and write sqitch tracking. On next deploy, check for pending non-transactional changes and verify state.
|
|
155
|
+
|
|
156
|
+
- **`--no-transaction` is a script comment in Sqitch:** Fixed — Sqitch uses `-- sqitch-no-transaction` comment in deploy script first line. sqlever supports both the script comment (Sqitch compat) and plan file pragma.
|
|
157
|
+
|
|
158
|
+
- **Inline suppression scoping specified:** Unclosed block extends to EOF with warning, single-line comment attaches to preceding statement, comma-separated rule IDs supported, unknown rules produce warning, `all` not supported.
|
|
159
|
+
|
|
160
|
+
- **SA020 expanded scope:** Now covers `DROP INDEX CONCURRENTLY` and `REINDEX CONCURRENTLY` in addition to `CREATE INDEX CONCURRENTLY`. Standalone mode behavior specified: warn on any CONCURRENTLY usage.
|
|
161
|
+
|
|
162
|
+
- **Reporter format schemas defined:** JSON output schema (envelope with metadata, findings, summary), GitHub annotations (`::error file=...` format), GitLab Code Quality JSON schema documented. `--exit-code` renamed to `--strict`.
|
|
163
|
+
|
|
164
|
+
- **Tracking schema DDL completeness:** Added `PRIMARY KEY (change_id, dependency)` to `sqitch.dependencies`. Added `CHECK` constraint and note about no PK on `sqitch.events`. Added `UNIQUE (project, tag)` to `sqitch.tags`. Added `ON DELETE CASCADE` where Sqitch uses it.
|
|
165
|
+
|
|
166
|
+
- **ALTER TYPE ADD VALUE PG 12+ gotcha documented:** Even in PG 12+ where it can run in a transaction, the new enum value is not usable within the same transaction.
|
|
167
|
+
|
|
168
|
+
- **Trigger recursion guard changed:** Replaced `pg_trigger_depth() < 2` with session variable approach: `SET LOCAL sqlever.syncing = 'true'` / `current_setting('sqlever.syncing', true)`. More robust in environments with existing triggers.
|
|
169
|
+
|
|
170
|
+
- **search_path OPEN resolved:** Use database/role default (Sqitch-compatible). Override available via `sqlever.toml`.
|
|
171
|
+
|
|
172
|
+
- **application_name added:** Set `application_name = 'sqlever/<command>/<project>'` on deploy/batch connections for production debugging.
|
|
173
|
+
|
|
174
|
+
- **Advisory locks for revert/rebase/checkout:** Not just deploy — any command modifying tracking state or executing DDL.
|
|
175
|
+
|
|
176
|
+
- **idle_in_transaction_session_timeout:** Changed from 0 (unlimited) to configurable generous value (default 10 minutes).
|
|
177
|
+
|
|
178
|
+
- **Non-transactional statement_timeout:** Separate configurable timeout (default 4 hours) for non-transactional DDL.
|
|
179
|
+
|
|
180
|
+
- **Lock retry for CI:** Added `--lock-retries N` with exponential backoff (default 0 = no retry).
|
|
181
|
+
|
|
182
|
+
- **PgBouncer detection improved:** Use `SHOW pool_mode` (PgBouncer-specific). Added `connection_type` config option for non-PgBouncer poolers.
|
|
183
|
+
|
|
184
|
+
- **PG 13 partition discussion simplified:** Since test matrix is PG 14+, trigger inheritance is always available. Removed per-partition installation discussion.
|
|
185
|
+
|
|
186
|
+
- **`--force-rule SA003` added:** Per-rule deploy-time override alongside blanket `--force`.
|
|
187
|
+
|
|
188
|
+
- **VACUUM pressure threshold defined:** Ratio `n_dead_tup / (n_live_tup + n_dead_tup)` exceeding configurable percentage (default 10%).
|
|
189
|
+
|
|
190
|
+
### Other changes
|
|
191
|
+
|
|
192
|
+
- Version bumped to 0.4
|
|
193
|
+
- Architecture tree fixed: `SPEC.md` path updated to `spec/SPEC.md`
|
|
194
|
+
- `sqitch.conf` `[deploy]` section documented (`verify`, `mode` defaults)
|
|
195
|
+
- SA002 test fixtures updated: `now()` moved from trigger/ to no_trigger/, `clock_timestamp()` added to trigger/
|
|
196
|
+
- Remaining OPEN markers (4): SA003 safe-cast list (needs `pg_cast` audit), logical replication + expand/contract, PGQ vs SKIP LOCKED, DD12 psql vs node-postgres
|
|
197
|
+
|
|
198
|
+
## [SPEC 0.3] — 2026-03-20
|
|
199
|
+
|
|
200
|
+
Comprehensive update based on expert review from four specialists: PG internals expert, Sqitch power user, production SRE, and static analysis engineer. All critical and important findings addressed.
|
|
201
|
+
|
|
202
|
+
### Critical fixes
|
|
203
|
+
|
|
204
|
+
- **Non-transactional DDL support (C1):** Added `--no-transaction` flag to `sqlever add` and plan file pragma. Deploy data flow updated to execute non-transactional changes without `BEGIN`/`COMMIT` wrapper. Tracking updates happen in a separate transaction. Covers `CREATE INDEX CONCURRENTLY`, `DROP INDEX CONCURRENTLY`, `ALTER TYPE ADD VALUE` (PG < 12), `REINDEX CONCURRENTLY`. Added SA020 rule to detect `CONCURRENTLY` inside transactional deploys.
|
|
205
|
+
|
|
206
|
+
- **Advisory locks for deploy coordination (C2):** Deploy data flow now acquires `pg_advisory_xact_lock` (or `pg_advisory_lock`) before executing changes. Second concurrent deploy exits with code 4. Crash recovery: PG auto-releases advisory locks on disconnect. Added integration tests for concurrent deploy scenarios.
|
|
207
|
+
|
|
208
|
+
- **SA002 volatile defaults (C3):** Split into SA002 (volatile defaults cause rewrite on ALL PG versions, promoted to `error`) and SA002b (non-volatile defaults cause rewrite only on PG < 11, `warn`). Fixed Problem 1 example to use a non-volatile default to accurately illustrate the PG < 11 behavior.
|
|
209
|
+
|
|
210
|
+
- **SA003 safe cast allowlist (C4):** Defined "non-trivial cast" explicitly with a safe-cast allowlist (varchar widening, varchar→text, numeric precision widening). Everything else flags. Reference to `pg_cast` for connected analysis. OPEN marker for comprehensive allowlist audit.
|
|
211
|
+
|
|
212
|
+
- **Missing Sqitch commands (C5):** Added `rework`, `rebase`, `bundle`, `checkout`, `show`, `plan`, `upgrade` to R1 command table. Added `rework.ts`, `rebase.ts`, `bundle.ts`, `checkout.ts`, `show.ts`, `plan.ts`, `upgrade.ts` to architecture. Documented rework semantics including `@tag` syntax and plan file format. Added `reworked/` test fixture.
|
|
213
|
+
|
|
214
|
+
- **Tracking schema corrected (C6):** R3 now includes full DDL for all five tables (`sqitch.changes`, `sqitch.dependencies`, `sqitch.events`, `sqitch.tags`, `sqitch.projects`). Fixed column names: `committer_name`/`committer_email` (not `deployed_by`), `planner_name`/`planner_email`, `planned_at`, `note`. Added `sqitch.dependencies` table. Oracle comparison table updated to match actual Sqitch schema.
|
|
215
|
+
|
|
216
|
+
- **psql vs node-postgres (C7):** Added DD12 documenting the fundamental architecture decision. Three options presented (shell to psql, pre-processing layer, both modes). Marked as OPEN — must be resolved before Sprint 2. Added impact analysis on data flow, `--mode` semantics, `ON_ERROR_STOP`, `--set` variables, and snapshot includes.
|
|
217
|
+
|
|
218
|
+
- **Change ID computation (C8):** Documented that change IDs are SHA-1 hashes computed from change content. Added OPEN marker for exact algorithm specification. Added change ID verification tests to unit test plan and oracle comparison.
|
|
219
|
+
|
|
220
|
+
- **PgBouncer compatibility (C9):** Added DD13 covering PgBouncer detection, `pg_advisory_xact_lock` preference, SET re-issue per transaction, and recommendation for direct connections during deploy/batch operations.
|
|
221
|
+
|
|
222
|
+
- **pgsql-parser + bun build --compile validation (C10):** Added Phase 0 validation spike for native C addon bundling. Evaluate WASM alternatives if bundling fails. Marked as go/no-go for architecture.
|
|
223
|
+
|
|
224
|
+
- **Inline suppression for analysis (C11):** Added `-- sqlever:disable SA010` comment syntax and per-file overrides in `sqlever.toml`. Documented in Section 5.1 and included in unit test plan.
|
|
225
|
+
|
|
226
|
+
- **Expand/contract trigger edge cases (C12):** Added subsection covering: infinite recursion (`pg_trigger_depth()` guard), logical replication (triggers don't fire on subscribers), partitioned tables (PG 13+ for trigger inheritance), COPY performance, trigger installation lock (`AccessExclusiveLock`), concurrency control via advisory locks.
|
|
227
|
+
|
|
228
|
+
### Important fixes
|
|
229
|
+
|
|
230
|
+
- **SA001 description corrected (I1):** Changed from "Takes AccessExclusiveLock" to "Fails outright on populated tables." The issue is a DDL error, not a lock concern.
|
|
231
|
+
|
|
232
|
+
- **New analysis rules (I2):** Added SA016 (`ADD CONSTRAINT CHECK` without `NOT VALID`), SA017 (`SET NOT NULL` on existing column), SA018 (`ADD PRIMARY KEY` without pre-existing index), SA019 (`REINDEX` without `CONCURRENTLY`), SA020 (`CREATE INDEX CONCURRENTLY` in transaction), SA021 (explicit `LOCK TABLE`).
|
|
233
|
+
|
|
234
|
+
- **Static vs connected rules (I3):** Added rule type classification. SA009 and SA011 marked as "connected" (require DB context). Static rules work in standalone linter mode. Connected rules silently skipped when no connection is available.
|
|
235
|
+
|
|
236
|
+
- **PL/pgSQL body exclusion (I4):** DML inside `CREATE FUNCTION`, `CREATE PROCEDURE`, and `DO` blocks excluded from SA010/SA011/SA008. Analysis rules operate on top-level statements only.
|
|
237
|
+
|
|
238
|
+
- **sqitch.conf format documented (I5):** Added Git-style INI format description with subsections (`[engine "pg"]`), `db:` URI scheme, config precedence hierarchy (system < user < project < sqlever.toml < env < flags).
|
|
239
|
+
|
|
240
|
+
- **Missing flags added (I6):** `--set`/`-s` (template variables), `--log-only` (adopt existing schemas), `--target`, `--no-verify`, `--verify` added to R1 flags list with descriptions.
|
|
241
|
+
|
|
242
|
+
- **Session settings (I7):** Added DD14 covering `statement_timeout=0` and `idle_in_transaction_session_timeout=0` for deploy connections.
|
|
243
|
+
|
|
244
|
+
- **Lock timeout guard moved to v1.0 (I8):** Moved from v1.1 to v1.0. Core safety infrastructure, not optional.
|
|
245
|
+
|
|
246
|
+
- **Batch job dead state recovery (I9):** Added `sqlever batch retry` command. Dead jobs can be manually retried. Last processed PK tracked so retried jobs resume from where they stopped.
|
|
247
|
+
|
|
248
|
+
- **Replication lag monitoring (I10):** Added to batched DML features. Query `pg_stat_replication.replay_lag`, pause when lag exceeds configurable threshold (default 10s).
|
|
249
|
+
|
|
250
|
+
- **Reverse handoff test (I11):** Added sqlever→Sqitch compatibility test: deploy with sqlever, verify Sqitch reads tracking tables correctly, add/revert changes with Sqitch.
|
|
251
|
+
|
|
252
|
+
- **sqlever analyze composability (I12):** `sqlever analyze file.sql` works with zero config. No `sqitch.plan` required. Standalone linter mode for teams using other migration tools. Added `--changed` flag for CI.
|
|
253
|
+
|
|
254
|
+
- **Plan file entry format documented (I13):** `change_name [deps] YYYY-MM-DDTHH:MM:SSZ planner_name <planner_email> # note`.
|
|
255
|
+
|
|
256
|
+
- **Plan file pragmas documented (I14):** `%syntax-version`, `%project`, `%uri` — all documented with descriptions.
|
|
257
|
+
|
|
258
|
+
- **Cross-project dependencies (I15):** `project:change` syntax documented in R2, test cases added.
|
|
259
|
+
|
|
260
|
+
- **SA010 downgraded to warn (I16):** Full-table DML is often intentional in migrations. Use inline suppression for acknowledged cases.
|
|
261
|
+
|
|
262
|
+
- **SA015 expanded and downgraded (I17):** Now covers both table and column renames. Downgraded to `warn` until expand/contract (v2.0) exists, since there is no way to satisfy the rule before then.
|
|
263
|
+
|
|
264
|
+
- **Rule interface contract (I18):** Defined `Rule`, `AnalysisContext`, and `Finding` interfaces in Section 5.1.
|
|
265
|
+
|
|
266
|
+
- **--mode transaction semantics (I19):** Documented `all`/`change`/`tag` transaction scope differences. Non-transactional changes always execute outside any transaction regardless of mode.
|
|
267
|
+
|
|
268
|
+
- **Exit code 127 replaced (I20):** Changed "database unreachable" from 127 to 10. Added exit codes 4 (concurrent deploy) and 5 (lock timeout). Exit code table added to R6.
|
|
269
|
+
|
|
270
|
+
- **sqlever.* schema documented (I21):** Clarified in DD3 that `sqlever.*` schema is created only when sqlever-specific features are used. Independent of `sqitch.*`. Can be safely dropped if reverting to Sqitch.
|
|
271
|
+
|
|
272
|
+
- **Batch worker connections (I22):** Documented that batch worker requires direct PostgreSQL connection (not PgBouncer in transaction mode). SET statements re-issued per batch transaction.
|
|
273
|
+
|
|
274
|
+
- **VACUUM pressure and bloat (I23):** Added dead tuple monitoring (`pg_stat_user_tables.n_dead_tup`) to batched DML features. Pause if accumulation exceeds threshold.
|
|
275
|
+
|
|
276
|
+
- **SA009 expanded (I24):** Now also flags `ADD FOREIGN KEY` without `NOT VALID` as the primary concern (lock during validation). Recommends two-step `NOT VALID` + `VALIDATE CONSTRAINT` pattern.
|
|
277
|
+
|
|
278
|
+
- **script_hash computation (I25):** Documented in R3. OPEN marker for exact algorithm and interaction with snapshot includes.
|
|
279
|
+
|
|
280
|
+
- **SA014 expanded (I26):** Now also covers `CLUSTER` (same lock + rewrite behavior as `VACUUM FULL`).
|
|
281
|
+
|
|
282
|
+
- **search_path handling (I27):** Added as OPEN in DD14. Options documented.
|
|
283
|
+
|
|
284
|
+
- **Problem 1 example fixed (I28):** Changed from `DEFAULT now()` (volatile, rewrites on ALL versions) to `DEFAULT '2024-01-01'::timestamptz` (non-volatile, correctly illustrates PG < 11 rewrite behavior).
|
|
285
|
+
|
|
286
|
+
### Other changes
|
|
287
|
+
|
|
288
|
+
- Version bumped to 0.3
|
|
289
|
+
- Static analysis moved from v1.1 to v1.0 (core safety infrastructure)
|
|
290
|
+
- CI integration moved from v1.1+ to v1.0+ (aligns with analysis move)
|
|
291
|
+
- Conflict dependency semantics documented in DD6
|
|
292
|
+
- `sqlever analyze` scope defined (file, directory, pending, --all, --changed)
|
|
293
|
+
- Registry schema creation specified (IF NOT EXISTS + advisory lock for concurrent first-deploy)
|
|
294
|
+
- Added OPEN markers for: change ID algorithm, script_hash computation, search_path handling, logical replication + expand/contract, PGQ vs SKIP LOCKED queue design, comprehensive SA003 safe-cast list
|
|
295
|
+
- Added `preprocess.ts` to architecture for psql metacommand handling
|
|
296
|
+
- Test fixtures expanded: `reworked/`, `cross-project/`, `conflicts/`, `non-transactional/`
|
|
297
|
+
- Dry-run mode: explicitly documented what it does and does NOT guarantee
|
|
298
|
+
|
|
299
|
+
## [SPEC 0.2] — 2026-03-20
|
|
300
|
+
|
|
301
|
+
- Full rewrite of testing strategy (section 8): unit, integration, Sqitch oracle/compat, analysis fixture corpus, performance tests, CI configuration, local dev workflow
|
|
302
|
+
- Added DD11: Sqitch as oracle for compatibility testing
|
|
303
|
+
- Added links to GitLab migration_helpers.rb, batched background migrations docs, migration style guide, migration pipeline docs
|
|
304
|
+
- Fixed header metadata rendering
|
|
305
|
+
|
|
306
|
+
## [SPEC 0.1] — 2026-03-20
|
|
307
|
+
|
|
308
|
+
- Initial spec: problem statement, goals, requirements, feature ideas (10 features), design decisions (DD1–DD10), architecture, basic testing notes, phased implementation plan (7 phases, 16 sprints)
|
|
309
|
+
- Repo scaffolded: README, package.json, src/cli.ts skeleton, LICENSE (Apache 2.0)
|