bern 2026.11.19__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- bern-2026.11.19.dist-info/METADATA +93 -0
- bern-2026.11.19.dist-info/RECORD +9 -0
- bern-2026.11.19.dist-info/WHEEL +4 -0
- bern-2026.11.19.dist-info/entry_points.txt +3 -0
- bern-2026.11.19.dist-info/licenses/LICENSE +21 -0
- bernard/__init__.py +0 -0
- bernard/animation.py +39 -0
- bernard/cli.py +144 -0
- bernard/core.py +357 -0
|
@@ -0,0 +1,93 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: bern
|
|
3
|
+
Version: 2026.11.19
|
|
4
|
+
Summary: Swap a Django swappable model mid-project: the hermit crab changes shells, the crab stays the same.
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
License-File: LICENSE
|
|
7
|
+
Requires-Dist: django>=4.2
|
|
8
|
+
Requires-Python: >=3.10
|
|
9
|
+
Description-Content-Type: text/markdown
|
|
10
|
+
|
|
11
|
+
# Bernard
|
|
12
|
+
|
|
13
|
+
```text
|
|
14
|
+
.-~~~~-.
|
|
15
|
+
.-' .--. '-.
|
|
16
|
+
/ .' __ '. \
|
|
17
|
+
| / ( ) \ |
|
|
18
|
+
| \ '' / |
|
|
19
|
+
o o \ '.____.' /
|
|
20
|
+
\ / '-. .-'
|
|
21
|
+
,-( ^ )-. __/'~~~~~~'
|
|
22
|
+
(_/ \_/ \_)--'
|
|
23
|
+
/| |\
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
*Bernard l'ermite* is the French hermit crab: when it outgrows its shell, it moves into a new one and stays the same crab.
|
|
27
|
+
|
|
28
|
+
`bern` does that for Django swappable models. You set `AUTH_USER_MODEL`, `WAGTAIL_PAGE_MODEL`, or any `Meta.swappable` setting partway through a project, and Bernard moves the existing rows into the new model. It copies nothing.
|
|
29
|
+
|
|
30
|
+
Run it from your project, next to `manage.py`:
|
|
31
|
+
|
|
32
|
+
```console
|
|
33
|
+
$ uvx bern # plan: reads only
|
|
34
|
+
bern · plan · default (postgresql, shop)
|
|
35
|
+
|
|
36
|
+
auth.User → accounts.User (AUTH_USER_MODEL, 1,204 rows)
|
|
37
|
+
rename auth_user → accounts_user
|
|
38
|
+
rename auth_user_groups → accounts_user_groups
|
|
39
|
+
rename auth_user_user_permissions → accounts_user_user_permissions
|
|
40
|
+
create accounts_profile
|
|
41
|
+
alter accounts_user.id integer → bigint
|
|
42
|
+
add column accounts_user.nickname
|
|
43
|
+
relabel auth.user → accounts.user
|
|
44
|
+
record accounts.0001_initial as applied
|
|
45
|
+
|
|
46
|
+
$ uvx bern apply # asks you to type the database name
|
|
47
|
+
$ python manage.py migrate
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
`uvx` runs bern in its own environment, which lacks your project's packages. So bern finds your project's virtualenv (`$VIRTUAL_ENV`, or a `.venv` in the current folder or above) and runs again with its Python. Your project's Django and apps win. Without a virtualenv to find, run `uv run --with bern bern` or install `bern` in the project.
|
|
51
|
+
|
|
52
|
+
## What it does
|
|
53
|
+
|
|
54
|
+
Bernard finds the swaps by itself. It loads your settings with `--settings`, `$DJANGO_SETTINGS_MODULE`, or the module your `manage.py` sets, then lists every model whose `_meta.swapped` is set. For each swap:
|
|
55
|
+
|
|
56
|
+
1. It finds the migration that creates the new model. A database that predates the swap has never recorded that migration, which is why `migrate` fails with `InconsistentMigrationHistory`.
|
|
57
|
+
2. It renames the legacy tables to the new names: the model's own table, its many-to-many tables, and companion models of the same name. Foreign keys pointing at them follow automatically.
|
|
58
|
+
3. It creates the tables that have no legacy counterpart. It adds the columns, indexes and constraints the migration declares and the legacy table lacks. It changes column types that differ, such as `AutoField` to `BigAutoField`.
|
|
59
|
+
4. It relabels the content types. Everything keyed on a content type follows: permissions, the admin log, Wagtail pages, revisions, and the search and reference indexes.
|
|
60
|
+
5. It records the migration as applied, so `migrate` carries on from there.
|
|
61
|
+
|
|
62
|
+
This is safe because Django skips every migration operation on a swapped-out model, so nothing will touch its tables again.
|
|
63
|
+
|
|
64
|
+
## Safety
|
|
65
|
+
|
|
66
|
+
- **Plan first.** `bern` with no command only reads. `apply` asks you to type the database name. In a deploy job with no terminal, it refuses unless you pass `--yes`.
|
|
67
|
+
- **All or nothing.** The whole move runs in a single transaction. If any step fails, or the process is killed, the database is left exactly as it was. That only works where DDL can be rolled back, so Bernard refuses MySQL and Oracle.
|
|
68
|
+
- **No surprises.** `apply` computes the plan again inside the transaction and stops if it differs from the one you confirmed.
|
|
69
|
+
- **Fails fast, doesn't queue.** On PostgreSQL, an advisory lock keeps two runs from overlapping. A 10 s `lock_timeout` makes Bernard give up instead of queueing behind a long transaction and blocking your site.
|
|
70
|
+
- **Refuses what it can't do right**:
|
|
71
|
+
- errors from Django's system checks;
|
|
72
|
+
- a creating migration that also does something else, like `RunPython`;
|
|
73
|
+
- legacy columns the new model would drop;
|
|
74
|
+
- tables that both hold rows;
|
|
75
|
+
- content types or permissions already in use under the new label.
|
|
76
|
+
- **Idempotent.** Once the move is done, `bern apply --yes` does nothing. You can keep it before `migrate` in your deploy job.
|
|
77
|
+
|
|
78
|
+
If an earlier `migrate` already created the new tables empty, Bernard drops them before moving the legacy ones in. It only does that while they are still empty and nothing else points at them.
|
|
79
|
+
|
|
80
|
+
## Deploying
|
|
81
|
+
|
|
82
|
+
Run it where you run `migrate`, between stopping the old code and starting the new one:
|
|
83
|
+
|
|
84
|
+
```sh
|
|
85
|
+
bern apply --yes && python manage.py migrate
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
After the move, the old code would query tables that no longer exist.
|
|
89
|
+
|
|
90
|
+
## Not covered
|
|
91
|
+
|
|
92
|
+
- **Wagtail images and documents** (`WAGTAILIMAGES_IMAGE_MODEL`, `WAGTAILDOCS_DOCUMENT_MODEL`) are not Django-swappable. `wagtailimages.Image` stays a live model with its own migrations, so its table can't be taken. They need their rows copied instead.
|
|
93
|
+
- Index and constraint names inherited from the legacy tables keep their old names. Django finds them by introspection when it later alters them.
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
bernard/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
2
|
+
bernard/animation.py,sha256=FY15IWRR8g2DqkDupsooXvo_2rLVeAPRsDR00iN0NQ0,1377
|
|
3
|
+
bernard/cli.py,sha256=uYpvQK24-8Q9qE3EjK32d3o2VnYmieDMe7vlqJQv5jM,5821
|
|
4
|
+
bernard/core.py,sha256=hHaqSbbqh_aY_Ws57ISCu3dXd3NFE_SHsdPn4EqRnBw,16274
|
|
5
|
+
bern-2026.11.19.dist-info/licenses/LICENSE,sha256=O4dN0_hQXjF4UaustszMKDqUNbE-HivrO8WSUF7QAaI,1074
|
|
6
|
+
bern-2026.11.19.dist-info/WHEEL,sha256=G_-rrEibZr7OEM7zdH1dSO6s-VdjfnS5PA4CLAdI9Vw,80
|
|
7
|
+
bern-2026.11.19.dist-info/entry_points.txt,sha256=Hm7_B5NKkn49qhncPL1odP3ctApK36pDa4YjOMQxw30,43
|
|
8
|
+
bern-2026.11.19.dist-info/METADATA,sha256=XZPUNU4bFwNQ18TkVAh-aAF1uFIw7ZJo2JptbLB6KRQ,5226
|
|
9
|
+
bern-2026.11.19.dist-info/RECORD,,
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Fabien Le Frapper
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
bernard/__init__.py
ADDED
|
File without changes
|
bernard/animation.py
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
"""Bernard l'ermite leaves the old shell (the swapped-out model) and walks into the new one.
|
|
2
|
+
|
|
3
|
+
Same crab (the rows), new shell (the model): nothing is copied, only the label changes.
|
|
4
|
+
"""
|
|
5
|
+
|
|
6
|
+
import os
|
|
7
|
+
import sys
|
|
8
|
+
import time
|
|
9
|
+
|
|
10
|
+
CRAB, SHELL = "🦀", "🐚" # 2 columns each
|
|
11
|
+
GAP = 24
|
|
12
|
+
FRAME = 0.06
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
def frames(old, new):
|
|
16
|
+
labels = f"{old[:GAP]:<{GAP + 4}}{new}"
|
|
17
|
+
yield [f"{SHELL}{CRAB}{' ' * GAP}{SHELL}", labels] # out of the shell, looking around
|
|
18
|
+
for pos in range(2, GAP + 1, 2):
|
|
19
|
+
legs = " " if pos % 4 else "' " # a little scuttle
|
|
20
|
+
trail = (". " * (pos // 2))[: pos - 2] + legs
|
|
21
|
+
yield [f"{SHELL}{trail}{CRAB}{' ' * (GAP - pos)}{SHELL}", labels]
|
|
22
|
+
yield [f"{' '}{' ' * GAP}{SHELL}{CRAB}", f"{'':<{GAP + 4}}{new} ✓"] # moved in
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
def walk(old, new, stream=sys.stdout):
|
|
26
|
+
if not stream.isatty() or os.environ.get("TERM") == "dumb" or os.environ.get("CI"):
|
|
27
|
+
return
|
|
28
|
+
stream.write("\x1b[?25l") # hide the cursor
|
|
29
|
+
try:
|
|
30
|
+
for i, frame in enumerate(frames(old, new)):
|
|
31
|
+
if i:
|
|
32
|
+
stream.write(f"\x1b[{len(frame)}F") # back to the first line of the frame
|
|
33
|
+
stream.write("".join(f"\x1b[2K {line}\n" for line in frame))
|
|
34
|
+
stream.flush()
|
|
35
|
+
time.sleep(FRAME * (8 if i == 0 else 1))
|
|
36
|
+
stream.write("\n")
|
|
37
|
+
finally:
|
|
38
|
+
stream.write("\x1b[?25h")
|
|
39
|
+
stream.flush()
|
bernard/cli.py
ADDED
|
@@ -0,0 +1,144 @@
|
|
|
1
|
+
"""bern: swap a Django swappable model in the middle of a project.
|
|
2
|
+
|
|
3
|
+
uvx bern show what would move (reads only)
|
|
4
|
+
uvx bern apply move it, in one transaction, after you type the database name
|
|
5
|
+
"""
|
|
6
|
+
|
|
7
|
+
import argparse
|
|
8
|
+
import os
|
|
9
|
+
import re
|
|
10
|
+
import sys
|
|
11
|
+
from pathlib import Path
|
|
12
|
+
|
|
13
|
+
from bernard import animation, core
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
def main(argv=None):
|
|
17
|
+
argv = sys.argv[1:] if argv is None else argv
|
|
18
|
+
_run_in_project(argv)
|
|
19
|
+
parser = argparse.ArgumentParser(
|
|
20
|
+
prog="bern",
|
|
21
|
+
description="Move the rows of swapped-out Django models into the models that swap "
|
|
22
|
+
"them in: a new shell, the same crab. Nothing is copied, and nothing changes unless everything does.",
|
|
23
|
+
)
|
|
24
|
+
parser.add_argument("command", nargs="?", choices=["plan", "apply"], default="plan",
|
|
25
|
+
help="plan (default) only reads; apply moves")
|
|
26
|
+
parser.add_argument("--settings", help="settings module (default: $DJANGO_SETTINGS_MODULE, "
|
|
27
|
+
"then the one manage.py sets)")
|
|
28
|
+
parser.add_argument("--database", default="default", help="database alias (default: default)")
|
|
29
|
+
parser.add_argument("--yes", action="store_true", help="apply without asking, for deploy jobs")
|
|
30
|
+
parser.add_argument("--no-fun", action="store_true", help="skip the animation")
|
|
31
|
+
args = parser.parse_args(argv)
|
|
32
|
+
_setup(args.settings)
|
|
33
|
+
return _run(args)
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
def _run_in_project(argv):
|
|
37
|
+
"""Under uvx, bern runs in its own environment, without the project's apps: run again
|
|
38
|
+
with the project's Python, bern's own code last on the path so the project's Django wins."""
|
|
39
|
+
python = _project_python()
|
|
40
|
+
if not python or os.environ.get("BERN_IN_PROJECT"):
|
|
41
|
+
return
|
|
42
|
+
os.environ["BERN_IN_PROJECT"] = "1"
|
|
43
|
+
code = "import sys; sys.path.append(sys.argv.pop(1)); from bernard.cli import main; sys.exit(main(sys.argv[1:]))"
|
|
44
|
+
os.execv(python, [str(python), "-c", code, str(Path(__file__).resolve().parent.parent), *argv])
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
def _project_python():
|
|
48
|
+
"""The project's virtualenv Python, unless bern already runs in it."""
|
|
49
|
+
here = Path.cwd()
|
|
50
|
+
candidates = [os.environ.get("VIRTUAL_ENV"), *(d / ".venv" for d in [here, *here.parents])]
|
|
51
|
+
for venv in filter(None, candidates):
|
|
52
|
+
for python in (Path(venv) / "bin" / "python", Path(venv) / "Scripts" / "python.exe"):
|
|
53
|
+
if python.exists():
|
|
54
|
+
return None if Path(venv).resolve() == Path(sys.prefix).resolve() else python
|
|
55
|
+
return None
|
|
56
|
+
|
|
57
|
+
|
|
58
|
+
def _setup(settings):
|
|
59
|
+
sys.path.insert(0, os.getcwd())
|
|
60
|
+
settings = settings or os.environ.get("DJANGO_SETTINGS_MODULE") or _from_manage_py()
|
|
61
|
+
if not settings:
|
|
62
|
+
sys.exit("bern: no Django settings found. Pass --settings, set "
|
|
63
|
+
"DJANGO_SETTINGS_MODULE, or run bern next to manage.py.")
|
|
64
|
+
os.environ["DJANGO_SETTINGS_MODULE"] = settings
|
|
65
|
+
import django
|
|
66
|
+
|
|
67
|
+
try:
|
|
68
|
+
django.setup()
|
|
69
|
+
except ModuleNotFoundError as e:
|
|
70
|
+
sys.exit(f"bern: {e}. Run bern with your project's packages: from its virtualenv, "
|
|
71
|
+
"or with `uv run --with bern bern`.")
|
|
72
|
+
|
|
73
|
+
|
|
74
|
+
def _from_manage_py():
|
|
75
|
+
manage = Path("manage.py")
|
|
76
|
+
if manage.exists():
|
|
77
|
+
found = re.search(r"DJANGO_SETTINGS_MODULE['\"]\s*,\s*['\"]([\w.]+)", manage.read_text())
|
|
78
|
+
return found and found[1]
|
|
79
|
+
|
|
80
|
+
|
|
81
|
+
def _run(args):
|
|
82
|
+
from django.db import connections
|
|
83
|
+
|
|
84
|
+
connection = connections[args.database]
|
|
85
|
+
db_name = os.path.basename(str(connection.settings_dict["NAME"]))
|
|
86
|
+
print(f"bern · {args.command} · {args.database} ({connection.vendor}, {db_name})\n")
|
|
87
|
+
moves, problems = core.plan(args.database)
|
|
88
|
+
for move in moves:
|
|
89
|
+
print(describe(move))
|
|
90
|
+
if problems:
|
|
91
|
+
print("Bernard stays in the old shell:", *(f" - {p}" for p in problems), sep="\n")
|
|
92
|
+
return 1
|
|
93
|
+
if not moves:
|
|
94
|
+
print("Nothing to move: every swapped-in model already has its tables.")
|
|
95
|
+
return 0
|
|
96
|
+
if args.command == "plan":
|
|
97
|
+
print("No row is copied, and it all runs in one transaction: if any step fails, "
|
|
98
|
+
"nothing changes.\nRun `bern apply` to move.")
|
|
99
|
+
return 0
|
|
100
|
+
if not args.yes and not _confirmed(db_name):
|
|
101
|
+
print("Not confirmed: nothing changed.")
|
|
102
|
+
return 1
|
|
103
|
+
try:
|
|
104
|
+
core.apply(moves, args.database)
|
|
105
|
+
except core.Refused as e:
|
|
106
|
+
print(f"Nothing changed: {e}")
|
|
107
|
+
return 1
|
|
108
|
+
except BaseException:
|
|
109
|
+
print("Nothing changed: the transaction was rolled back.", file=sys.stderr)
|
|
110
|
+
raise
|
|
111
|
+
rows = sum(m.rows for m in moves)
|
|
112
|
+
if not args.no_fun:
|
|
113
|
+
animation.walk(", ".join(m.old for m in moves), ", ".join(m.new for m in moves))
|
|
114
|
+
print(f"Done: {', '.join(m.new for m in moves)} moved in ({_rows(rows)}, none copied). "
|
|
115
|
+
"Run `migrate` next.")
|
|
116
|
+
return 0
|
|
117
|
+
|
|
118
|
+
|
|
119
|
+
def _confirmed(db_name):
|
|
120
|
+
if not sys.stdin.isatty():
|
|
121
|
+
print("No terminal to confirm in: pass --yes to apply.")
|
|
122
|
+
return False
|
|
123
|
+
return input(f"Type the database name ({db_name}) to move: ").strip() == db_name
|
|
124
|
+
|
|
125
|
+
|
|
126
|
+
def _rows(n):
|
|
127
|
+
return f"{n:,} row{'s' if n != 1 else ''}"
|
|
128
|
+
|
|
129
|
+
|
|
130
|
+
def describe(move):
|
|
131
|
+
width = max([len(a) for a, _ in move.renames] or [0])
|
|
132
|
+
lines = [f" {move.old} → {move.new} ({move.setting}, {_rows(move.rows)})"]
|
|
133
|
+
lines += [f" drop {t} (empty, made by an earlier migrate)" for t in move.drops]
|
|
134
|
+
lines += [f" rename {a:<{width}} → {b}" for a, b in move.renames]
|
|
135
|
+
lines += [f" create {t}" for t in move.creates]
|
|
136
|
+
lines += [f" alter {t}.{c} {a} → {b}" for t, c, a, b in move.alters]
|
|
137
|
+
lines += [f" add {kind} {t}.{name}" for t, kind, name in move.adds]
|
|
138
|
+
lines += [f" relabel {oa}.{om} → {na}.{nm}" for oa, om, na, nm in move.relabels]
|
|
139
|
+
lines.append(f" record {'.'.join(move.migration)} as applied")
|
|
140
|
+
return "\n".join(lines) + "\n"
|
|
141
|
+
|
|
142
|
+
|
|
143
|
+
if __name__ == "__main__":
|
|
144
|
+
sys.exit(main())
|
bernard/core.py
ADDED
|
@@ -0,0 +1,357 @@
|
|
|
1
|
+
"""Bernard (as in bernard-l'ermite, the hermit crab) moves the rows of a swapped-out model
|
|
2
|
+
into the model that swaps it in: a new shell, the same crab.
|
|
3
|
+
|
|
4
|
+
Django skips every migration operation on a swapped-out model (``Options.can_migrate``),
|
|
5
|
+
so its tables are free to take. For each swap, Bernard renames the legacy tables to the
|
|
6
|
+
new model's names, creates what has no legacy counterpart, adds the columns, indexes and
|
|
7
|
+
constraints the new model's creating migration declares, relabels the content types,
|
|
8
|
+
and records that migration as applied. No row is copied: foreign keys follow the renamed
|
|
9
|
+
tables, and everything keyed on a content type (permissions, admin log, Wagtail pages,
|
|
10
|
+
revisions, search index) follows the relabelled one. The whole move runs in one
|
|
11
|
+
transaction, so a failure anywhere leaves the database untouched.
|
|
12
|
+
"""
|
|
13
|
+
|
|
14
|
+
from dataclasses import dataclass, field
|
|
15
|
+
|
|
16
|
+
from django.apps import apps
|
|
17
|
+
from django.core import checks
|
|
18
|
+
from django.core.exceptions import FieldDoesNotExist
|
|
19
|
+
from django.db import connections
|
|
20
|
+
from django.db.migrations.loader import MigrationLoader
|
|
21
|
+
from django.db.migrations.operations import CreateModel
|
|
22
|
+
from django.db.migrations.recorder import MigrationRecorder
|
|
23
|
+
|
|
24
|
+
ADVISORY_LOCK = 0xBE4A4D # PostgreSQL: one Bernard per database at a time
|
|
25
|
+
# ponytail: fixed; make it a flag if a busy table needs longer.
|
|
26
|
+
LOCK_TIMEOUT = "10s"
|
|
27
|
+
|
|
28
|
+
|
|
29
|
+
class Refused(Exception):
|
|
30
|
+
pass
|
|
31
|
+
|
|
32
|
+
|
|
33
|
+
@dataclass(frozen=True)
|
|
34
|
+
class Move:
|
|
35
|
+
old: str # "auth.User"
|
|
36
|
+
new: str # "accounts.User"
|
|
37
|
+
setting: str # "AUTH_USER_MODEL"
|
|
38
|
+
migration: tuple # the new model's creating migration, recorded as applied
|
|
39
|
+
records: tuple # that migration and the ones it replaces
|
|
40
|
+
drops: tuple # empty tables a previous `migrate` created for the new model
|
|
41
|
+
renames: tuple # ((legacy_table, table), ...)
|
|
42
|
+
creates: tuple # tables with no legacy counterpart
|
|
43
|
+
alters: tuple # ((table, column, old_type, new_type), ...)
|
|
44
|
+
adds: tuple # ((table, "column" | "index" | "constraint", name), ...)
|
|
45
|
+
relabels: tuple # ((old_app, old_model, new_app, new_model), ...)
|
|
46
|
+
rows: int = field(compare=False)
|
|
47
|
+
models: dict = field(compare=False, repr=False) # table -> historical model
|
|
48
|
+
|
|
49
|
+
|
|
50
|
+
def label(model):
|
|
51
|
+
return model._meta.label
|
|
52
|
+
|
|
53
|
+
|
|
54
|
+
def swaps():
|
|
55
|
+
for model in apps.get_models(include_swapped=True):
|
|
56
|
+
if model._meta.swapped:
|
|
57
|
+
yield model, apps.get_model(model._meta.swapped)
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def plan(using="default"):
|
|
61
|
+
"""Return (moves, problems), reading only."""
|
|
62
|
+
connection = connections[using]
|
|
63
|
+
problems = [str(m) for m in checks.run_checks() if m.is_serious()]
|
|
64
|
+
if not connection.features.can_rollback_ddl:
|
|
65
|
+
problems.append(
|
|
66
|
+
f"{connection.vendor} cannot roll back schema changes, so a failed move would "
|
|
67
|
+
"leave a half-moved database: Bernard only runs on PostgreSQL and SQLite."
|
|
68
|
+
)
|
|
69
|
+
loader = MigrationLoader(connection, ignore_no_migrations=True)
|
|
70
|
+
with connection.cursor() as cursor:
|
|
71
|
+
tables = set(connection.introspection.table_names(cursor))
|
|
72
|
+
moves = []
|
|
73
|
+
for old, new in swaps():
|
|
74
|
+
try:
|
|
75
|
+
move = _plan_one(old, new, loader, connection, tables)
|
|
76
|
+
except Refused as e:
|
|
77
|
+
problems.append(f"{label(old)} → {label(new)}: {e}")
|
|
78
|
+
continue
|
|
79
|
+
if move:
|
|
80
|
+
moves.append(move)
|
|
81
|
+
moves.sort(key=lambda m: len(loader.graph.forwards_plan(m.migration)))
|
|
82
|
+
pending = {m.migration for m in moves} | loader.applied_migrations.keys()
|
|
83
|
+
for move in moves:
|
|
84
|
+
missing = loader.graph.node_map[move.migration].parents
|
|
85
|
+
missing = sorted(p.key for p in missing if p.key not in pending)
|
|
86
|
+
if missing:
|
|
87
|
+
problems.append(f"{move.new}: apply {missing} first, {move.migration} depends on them.")
|
|
88
|
+
return moves, problems
|
|
89
|
+
|
|
90
|
+
|
|
91
|
+
def _creating_migration(loader, model):
|
|
92
|
+
app, name = model._meta.app_label, model._meta.model_name
|
|
93
|
+
for key, migration in sorted(loader.graph.nodes.items()):
|
|
94
|
+
if key[0] == app and any(
|
|
95
|
+
isinstance(op, CreateModel) and op.name_lower == name for op in migration.operations
|
|
96
|
+
):
|
|
97
|
+
return key, migration
|
|
98
|
+
raise Refused(f"no migration of {app} creates {name}: run `makemigrations {app}` first.")
|
|
99
|
+
|
|
100
|
+
|
|
101
|
+
def _count(connection, table):
|
|
102
|
+
with connection.cursor() as cursor:
|
|
103
|
+
cursor.execute(f"SELECT COUNT(*) FROM {connection.ops.quote_name(table)}")
|
|
104
|
+
return cursor.fetchone()[0]
|
|
105
|
+
|
|
106
|
+
|
|
107
|
+
def _auto_throughs(model):
|
|
108
|
+
return [
|
|
109
|
+
f.remote_field.through
|
|
110
|
+
for f in model._meta.local_many_to_many
|
|
111
|
+
if f.remote_field.through._meta.auto_created
|
|
112
|
+
]
|
|
113
|
+
|
|
114
|
+
|
|
115
|
+
def _plan_one(old, new, loader, connection, tables):
|
|
116
|
+
key, migration = _creating_migration(loader, new)
|
|
117
|
+
old_table, new_table = old._meta.db_table, new._meta.db_table
|
|
118
|
+
applied = key in loader.applied_migrations
|
|
119
|
+
if old_table not in tables:
|
|
120
|
+
return None # fresh database, or already moved
|
|
121
|
+
if applied and new_table not in tables:
|
|
122
|
+
raise Refused(f"{key} is recorded as applied but {new_table} is missing.")
|
|
123
|
+
|
|
124
|
+
created_names = {op.name_lower for op in migration.operations if isinstance(op, CreateModel)}
|
|
125
|
+
for op in migration.operations:
|
|
126
|
+
target = getattr(op, "model_name_lower", None) or getattr(op, "name_lower", None)
|
|
127
|
+
if target not in created_names:
|
|
128
|
+
raise Refused(
|
|
129
|
+
f"{key} also does `{op.describe()}`, which Bernard would skip when recording "
|
|
130
|
+
"it: move that operation to a later migration."
|
|
131
|
+
)
|
|
132
|
+
# What the database holds once the move is recorded: every applied migration, plus this one.
|
|
133
|
+
nodes = [n for n in loader.applied_migrations if n in loader.graph.nodes]
|
|
134
|
+
state = loader.project_state([*nodes, key], at_end=True).apps
|
|
135
|
+
created = [state.get_model(key[0], name) for name in created_names]
|
|
136
|
+
models = {t._meta.db_table: t for m in created for t in [m, *_auto_throughs(m)]}
|
|
137
|
+
|
|
138
|
+
drops = ()
|
|
139
|
+
if new_table in tables:
|
|
140
|
+
if not _count(connection, old_table):
|
|
141
|
+
return None # the legacy table was left behind empty: nothing to move
|
|
142
|
+
drops = _droppable(models, key, loader, connection, tables)
|
|
143
|
+
|
|
144
|
+
live = {m._meta.db_table for m in apps.get_models(include_auto_created=True)}
|
|
145
|
+
renames, creates, alters, adds, relabels = [], [], [], [], []
|
|
146
|
+
for model in created:
|
|
147
|
+
is_main = model._meta.model_name == new._meta.model_name
|
|
148
|
+
table = model._meta.db_table
|
|
149
|
+
legacy = old_table if is_main else _swap_prefix(table, old, new)
|
|
150
|
+
if legacy not in tables or (not is_main and legacy in live):
|
|
151
|
+
creates.append(table)
|
|
152
|
+
continue
|
|
153
|
+
relabels.append(
|
|
154
|
+
(old._meta.app_label, old._meta.model_name if is_main else model._meta.model_name,
|
|
155
|
+
model._meta.app_label, model._meta.model_name)
|
|
156
|
+
)
|
|
157
|
+
for hm in [model, *_auto_throughs(model)]:
|
|
158
|
+
hm_legacy = legacy + hm._meta.db_table[len(table):]
|
|
159
|
+
if hm_legacy not in tables:
|
|
160
|
+
creates.append(hm._meta.db_table)
|
|
161
|
+
continue
|
|
162
|
+
renames.append((hm_legacy, hm._meta.db_table))
|
|
163
|
+
adds += _missing(hm, hm_legacy, connection)
|
|
164
|
+
if is_main:
|
|
165
|
+
alters += _type_changes(old, model, connection)
|
|
166
|
+
|
|
167
|
+
return Move(
|
|
168
|
+
old=label(old), new=label(new), setting=old._meta.swappable, migration=key,
|
|
169
|
+
records=(key, *migration.replaces), drops=drops, renames=tuple(renames),
|
|
170
|
+
creates=tuple(creates), alters=tuple(alters), adds=tuple(adds),
|
|
171
|
+
relabels=tuple(relabels), rows=_count(connection, old_table), models=models,
|
|
172
|
+
)
|
|
173
|
+
|
|
174
|
+
|
|
175
|
+
def _swap_prefix(table, old, new):
|
|
176
|
+
prefix = new._meta.app_label + "_"
|
|
177
|
+
return old._meta.app_label + "_" + table[len(prefix):] if table.startswith(prefix) else None
|
|
178
|
+
|
|
179
|
+
|
|
180
|
+
def _droppable(models, key, loader, connection, tables):
|
|
181
|
+
"""A `migrate` already created the new tables, empty: they can go if nothing built on them."""
|
|
182
|
+
existing = tuple(t for t in models if t in tables)
|
|
183
|
+
if any(_count(connection, t) for t in existing):
|
|
184
|
+
raise Refused(f"both the legacy and the new tables hold rows ({', '.join(existing)}): "
|
|
185
|
+
"merging them is not supported.")
|
|
186
|
+
later = sorted(c.key for c in loader.graph.node_map[key].children
|
|
187
|
+
if c.key[0] == key[0] and c.key in loader.applied_migrations)
|
|
188
|
+
if later:
|
|
189
|
+
raise Refused(f"the new tables are empty but {later} already changed them: "
|
|
190
|
+
f"`migrate {key[0]} {key[1]}` back first.")
|
|
191
|
+
with connection.cursor() as cursor:
|
|
192
|
+
for table in tables - set(existing):
|
|
193
|
+
for name, c in connection.introspection.get_constraints(cursor, table).items():
|
|
194
|
+
if c["foreign_key"] and c["foreign_key"][0] in existing:
|
|
195
|
+
raise Refused(f"{table}.{name} points at the empty {c['foreign_key'][0]}, "
|
|
196
|
+
"dropping it would lose that constraint.")
|
|
197
|
+
return existing
|
|
198
|
+
|
|
199
|
+
|
|
200
|
+
def _missing(model, legacy, connection):
|
|
201
|
+
with connection.cursor() as cursor:
|
|
202
|
+
columns = {c.name for c in connection.introspection.get_table_description(cursor, legacy)}
|
|
203
|
+
names = set(connection.introspection.get_constraints(cursor, legacy))
|
|
204
|
+
expected = {f.column: f for f in model._meta.local_concrete_fields}
|
|
205
|
+
orphans = sorted(columns - set(expected))
|
|
206
|
+
if orphans:
|
|
207
|
+
raise Refused(f"{legacy} has columns {orphans} that {label(model)} lacks: "
|
|
208
|
+
"their data would be left behind. Add them to the new model.")
|
|
209
|
+
table = model._meta.db_table
|
|
210
|
+
return [
|
|
211
|
+
*[(table, "column", f.name) for c, f in expected.items() if c not in columns],
|
|
212
|
+
*[(table, "index", i.name) for i in model._meta.indexes if i.name not in names],
|
|
213
|
+
*[(table, "constraint", c.name) for c in model._meta.constraints if c.name not in names],
|
|
214
|
+
]
|
|
215
|
+
|
|
216
|
+
|
|
217
|
+
def _type_changes(old, model, connection):
|
|
218
|
+
changes = []
|
|
219
|
+
for f in model._meta.local_concrete_fields:
|
|
220
|
+
try:
|
|
221
|
+
before = old._meta.get_field(f.name).db_type(connection)
|
|
222
|
+
except FieldDoesNotExist:
|
|
223
|
+
continue
|
|
224
|
+
if before != f.db_type(connection):
|
|
225
|
+
changes.append((model._meta.db_table, f.name, before, f.db_type(connection)))
|
|
226
|
+
return changes
|
|
227
|
+
|
|
228
|
+
|
|
229
|
+
def apply(confirmed, using="default"):
|
|
230
|
+
"""Run the confirmed moves in one transaction, after checking the plan still holds."""
|
|
231
|
+
connection = connections[using]
|
|
232
|
+
with connection.schema_editor(atomic=True) as editor:
|
|
233
|
+
_lock(connection)
|
|
234
|
+
moves, problems = plan(using)
|
|
235
|
+
if problems:
|
|
236
|
+
raise Refused("\n".join(problems))
|
|
237
|
+
if moves != confirmed:
|
|
238
|
+
raise Refused("the database changed since the plan was shown: run bern again.")
|
|
239
|
+
for move in moves:
|
|
240
|
+
_apply(move, editor, using)
|
|
241
|
+
if apps.is_installed("django.contrib.contenttypes"):
|
|
242
|
+
from django.contrib.contenttypes.models import ContentType
|
|
243
|
+
|
|
244
|
+
ContentType.objects.clear_cache()
|
|
245
|
+
return moves
|
|
246
|
+
|
|
247
|
+
|
|
248
|
+
def _lock(connection):
|
|
249
|
+
if connection.vendor != "postgresql":
|
|
250
|
+
return
|
|
251
|
+
with connection.cursor() as cursor:
|
|
252
|
+
cursor.execute(f"SET LOCAL lock_timeout = '{LOCK_TIMEOUT}'")
|
|
253
|
+
cursor.execute("SELECT pg_try_advisory_xact_lock(%s)", [ADVISORY_LOCK])
|
|
254
|
+
if not cursor.fetchone()[0]:
|
|
255
|
+
raise Refused("another bern is running on this database.")
|
|
256
|
+
|
|
257
|
+
|
|
258
|
+
def _apply(move, editor, using):
|
|
259
|
+
connection, models = editor.connection, move.models
|
|
260
|
+
for table in move.drops:
|
|
261
|
+
if connection.vendor == "postgresql":
|
|
262
|
+
editor.execute(f"LOCK TABLE {editor.quote_name(table)} IN ACCESS EXCLUSIVE MODE")
|
|
263
|
+
if _count(connection, table):
|
|
264
|
+
raise Refused(f"{table} received rows while Bernard was running.")
|
|
265
|
+
for table in move.drops:
|
|
266
|
+
editor.execute(editor.sql_delete_table % {"table": editor.quote_name(table)})
|
|
267
|
+
for legacy, table in move.renames:
|
|
268
|
+
editor.alter_db_table(models[table], legacy, table)
|
|
269
|
+
_rename_sequence(editor, models[table], legacy)
|
|
270
|
+
for table in move.creates:
|
|
271
|
+
editor.create_model(models[table])
|
|
272
|
+
for sql in editor.deferred_sql: # their foreign keys, before an alter recreates them
|
|
273
|
+
editor.execute(sql)
|
|
274
|
+
editor.deferred_sql.clear()
|
|
275
|
+
old = apps.get_model(move.old)
|
|
276
|
+
for table, name, _, _ in move.alters:
|
|
277
|
+
model = models[table]
|
|
278
|
+
before = old._meta.get_field(name).clone()
|
|
279
|
+
before.set_attributes_from_name(name)
|
|
280
|
+
before.model = model
|
|
281
|
+
editor.alter_field(model, before, model._meta.get_field(name))
|
|
282
|
+
for table, kind, name in move.adds:
|
|
283
|
+
model = models[table]
|
|
284
|
+
if kind == "column":
|
|
285
|
+
editor.add_field(model, model._meta.get_field(name))
|
|
286
|
+
elif name not in _constraint_names(connection, table): # SQLite's table rebuild adds them
|
|
287
|
+
items = model._meta.indexes if kind == "index" else model._meta.constraints
|
|
288
|
+
item = next(i for i in items if i.name == name)
|
|
289
|
+
(editor.add_index if kind == "index" else editor.add_constraint)(model, item)
|
|
290
|
+
for relabel in move.relabels:
|
|
291
|
+
_relabel(using, *relabel)
|
|
292
|
+
recorder = MigrationRecorder(connection)
|
|
293
|
+
applied = recorder.applied_migrations()
|
|
294
|
+
for app, name in move.records:
|
|
295
|
+
if (app, name) not in applied:
|
|
296
|
+
recorder.record_applied(app, name)
|
|
297
|
+
|
|
298
|
+
|
|
299
|
+
def _constraint_names(connection, table):
|
|
300
|
+
with connection.cursor() as cursor:
|
|
301
|
+
return set(connection.introspection.get_constraints(cursor, table))
|
|
302
|
+
|
|
303
|
+
|
|
304
|
+
def _rename_sequence(editor, model, legacy):
|
|
305
|
+
"""PostgreSQL keeps `<legacy>_id_seq` on a renamed table: give it the name Django expects."""
|
|
306
|
+
pk = model._meta.pk
|
|
307
|
+
if editor.connection.vendor != "postgresql" or pk.model is not model:
|
|
308
|
+
return
|
|
309
|
+
with editor.connection.cursor() as cursor:
|
|
310
|
+
cursor.execute("SELECT pg_get_serial_sequence(%s, %s)",
|
|
311
|
+
[editor.quote_name(model._meta.db_table), pk.column])
|
|
312
|
+
sequence = cursor.fetchone()[0]
|
|
313
|
+
if sequence and sequence.rsplit(".", 1)[-1].strip('"') == f"{legacy}_{pk.column}_seq":
|
|
314
|
+
new_name = editor.quote_name(f"{model._meta.db_table}_{pk.column}_seq")
|
|
315
|
+
editor.execute(f"ALTER SEQUENCE {sequence} RENAME TO {new_name}")
|
|
316
|
+
|
|
317
|
+
|
|
318
|
+
def _relabel(using, old_app, old_model, new_app, new_model):
|
|
319
|
+
if not apps.is_installed("django.contrib.contenttypes"):
|
|
320
|
+
return
|
|
321
|
+
from django.contrib.contenttypes.models import ContentType
|
|
322
|
+
|
|
323
|
+
types = ContentType.objects.db_manager(using)
|
|
324
|
+
stray = types.filter(app_label=new_app, model=new_model).first()
|
|
325
|
+
if stray: # created lazily by code that ran on the new settings: it must hold nothing
|
|
326
|
+
_delete_unreferenced(stray, using)
|
|
327
|
+
types.filter(app_label=old_app, model=old_model).update(app_label=new_app, model=new_model)
|
|
328
|
+
if old_model != new_model and apps.is_installed("django.contrib.auth"):
|
|
329
|
+
from django.contrib.auth.models import Permission
|
|
330
|
+
|
|
331
|
+
perms = Permission.objects.db_manager(using).filter(
|
|
332
|
+
content_type__app_label=new_app, content_type__model=new_model
|
|
333
|
+
)
|
|
334
|
+
for action in ("add", "change", "delete", "view"):
|
|
335
|
+
perms.filter(codename=f"{action}_{old_model}").update(codename=f"{action}_{new_model}")
|
|
336
|
+
|
|
337
|
+
|
|
338
|
+
def _delete_unreferenced(content_type, using):
|
|
339
|
+
"""Delete a content type and its permissions, unless any row points at them."""
|
|
340
|
+
connection = connections[using]
|
|
341
|
+
with connection.cursor() as cursor:
|
|
342
|
+
tables = set(connection.introspection.table_names(cursor))
|
|
343
|
+
perms = []
|
|
344
|
+
if apps.is_installed("django.contrib.auth"):
|
|
345
|
+
from django.contrib.auth.models import Permission
|
|
346
|
+
|
|
347
|
+
perms = list(Permission.objects.db_manager(using).filter(content_type=content_type))
|
|
348
|
+
for item in [*perms, content_type]:
|
|
349
|
+
for rel in item._meta.related_objects:
|
|
350
|
+
related = rel.related_model
|
|
351
|
+
if (perms and related is type(perms[0])) or related._meta.db_table not in tables:
|
|
352
|
+
continue # its own permissions are checked one by one; no table, no rows
|
|
353
|
+
if related._base_manager.db_manager(using).filter(**{rel.field.name: item}).exists():
|
|
354
|
+
raise Refused(f"{content_type} is already used by {label(related)}: "
|
|
355
|
+
"Bernard cannot tell which rows to keep.")
|
|
356
|
+
for item in [*perms, content_type]:
|
|
357
|
+
item.delete(using=using)
|