django-data-shape 0.14.0__tar.gz → 0.16.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/.pre-commit-config.yaml +12 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/CHANGELOG.md +86 -1
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/PKG-INFO +1 -1
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/__init__.py +2 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/keys/md5_keys.py +19 -10
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/resolve_fan_out.py +46 -0
- django_data_shape-0.16.0/django_data_shape/shape_from_factory.py +252 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/version.py +1 -1
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/index.md +71 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/reference.md +2 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/relations.md +10 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/pyproject.toml +1 -1
- django_data_shape-0.16.0/scripts/check-changelog +148 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/keys/test_md5_keys.py +46 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_build_graph.py +42 -1
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_build_projection.py +41 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_resolve_fan_out.py +57 -0
- django_data_shape-0.16.0/tests/test_shape_from_factory.py +248 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/testapp/models.py +18 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/.github/CODEOWNERS +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/.github/SECURITY.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/.github/dependabot.yml +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/.github/workflows/release.yml +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/.github/workflows/tests.yml +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/.github/workflows/upstream-drift.yml +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/.gitignore +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/CLAUDE.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/LICENSE +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/Makefile +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/README.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/apply_statistics_targets.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/build.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/build_result.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/canonical.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/check_constraints.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/check_invariants.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/children_per_parent.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/clone_database.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivation_queried_database.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivations/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivations/after.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivations/aligned.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivations/derivation.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivations/derived.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivations/given.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivations/per_parent.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/derivations/scope.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/ascending.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/bounded.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/categorical.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/constant.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/distinct.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/distribution.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/sequential.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/skew.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/uniform.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/distributions/zipf.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/drop_database.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/fan_out.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/fan_out_plan.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/fan_out_sizes.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/fixtures/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/fixtures/scale_fixture.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/fixtures/shape_fixture.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/fixtures/skip_unless_postgres.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/generate_rows.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/infer_key_strategy.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/invalid_shape.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/invariant.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/invariant_violated.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/keys/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/keys/disjoint.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/keys/key_function.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/keys/key_strategy.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/keys/sequential_keys.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/keys/sql_keys.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/keys/uuid_keys.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/order_derivations.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/order_tables.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/paired.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/paired_plan.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/projection.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/py.typed +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/refuse_queries.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/require_clone_strategy.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/require_postgres.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/resolve_paired.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/scale_protocol.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/scaled_shape.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/scaled_world.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/shape.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/shape_digest.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/shape_not_empty.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/table.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/table_result.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/template_database.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/unhashable_shape.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/unsupported_backend.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/utils.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/django_data_shape/world_changed.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/derivations.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/invariants.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/keys.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/projections.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/pytest.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/docs/statistics.md +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/mkdocs.yml +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/scripts/release-publish.sh +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/conftest.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/conftest_settings.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/derivations/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/derivations/test_after.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/derivations/test_aligned.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/derivations/test_derived.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/derivations/test_given.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/derivations/test_per_parent.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/distributions/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/distributions/test_constant.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/distributions/test_sequential.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/distributions/test_skew.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/distributions/test_uniform.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/distributions/test_zipf.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/keys/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/keys/test_key_function.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/keys/test_sequential_keys.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/keys/test_uuid_keys.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/scale_protocol_consumers.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/scale_protocol_impostors.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_apply_statistics_targets.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_build.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_build_derivations.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_build_invariants.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_build_keys.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_build_m2m.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_build_portable.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_check_constraints.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_children_per_parent.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_documentation.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_fan_out.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_fan_out_sizes.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_fixtures.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_generate_rows.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_infer_key_strategy.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_invariant.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_order_derivations.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_order_tables.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_paired.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_paired_plan.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_projection.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_reference_documents_the_public_surface.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_refuse_queries.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_require_clone_strategy.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_require_postgres.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_resolve_paired.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_scale_protocol.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_scaled_shape.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_scaled_world.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_shape.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_shape_digest.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_table.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_template_database.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_utils.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/test_version.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/tests/testapp/__init__.py +0 -0
- {django_data_shape-0.14.0 → django_data_shape-0.16.0}/uv.lock +0 -0
|
@@ -65,3 +65,15 @@ repos:
|
|
|
65
65
|
language: pygrep
|
|
66
66
|
types: [text]
|
|
67
67
|
exclude: '^\.pre-commit-config\.yaml$'
|
|
68
|
+
|
|
69
|
+
# Structural rather than a pattern, so it is a script where the guards
|
|
70
|
+
# above are pygrep. A merge or a rebase can rearrange this file without
|
|
71
|
+
# conflicting: a released heading and an unreleased entry insert after the
|
|
72
|
+
# same line, git resolves them silently, and shipped work ends up filed
|
|
73
|
+
# under a version already published. It has happened four times across two
|
|
74
|
+
# repositories and every catch was a habit rather than a gate.
|
|
75
|
+
- id: check-changelog
|
|
76
|
+
name: "check CHANGELOG structure (merge and rebase leave it silently wrong)"
|
|
77
|
+
entry: scripts/check-changelog
|
|
78
|
+
language: script
|
|
79
|
+
files: '^CHANGELOG\.md$'
|
|
@@ -7,6 +7,89 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.16.0] — 2026-09-04
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
- **A `FanOut(parents=[...])` partition no longer depends on what the keys are.**
|
|
14
|
+
Keys are read back ordered by primary key and the sizes are assigned by
|
|
15
|
+
position, so the weights followed the *sort order of the values* rather than
|
|
16
|
+
the declaration. With integer keys that is merely surprising; with the UUID
|
|
17
|
+
primary keys a factory row has on a modern schema it means **the same shape
|
|
18
|
+
builds differently every run** -- measured, one declaration gave the
|
|
19
|
+
first-named parent 5, 11 or 79 rows across twelve builds. That is the promise
|
|
20
|
+
the package rests on, and the same reason `UuidKeys` derives rather than
|
|
21
|
+
draws, so a narrowing that broke it was a defect and not a preference. Named
|
|
22
|
+
parents are put back in the order they were named before anything is weighed.
|
|
23
|
+
- **Reversing the list is now a different declaration**, which is what a
|
|
24
|
+
reader writing one would expect and was the symptom reported.
|
|
25
|
+
- **Weights are still scattered across positions**, so the reasoning that
|
|
26
|
+
argued for scattering survives: a caller's *order* decides which key lands
|
|
27
|
+
where, and nothing correlates a parent's key with its child count.
|
|
28
|
+
- Worth knowing for anyone testing near this: a test written over
|
|
29
|
+
**integer** parents passes against the broken behaviour, because the
|
|
30
|
+
database's sort order matches insertion order there. Only UUID parents show
|
|
31
|
+
it.
|
|
32
|
+
- **`shape_from_factory` takes `defaults=`, and names a factory that cannot be
|
|
33
|
+
called.** Most factories in a real codebase take arguments and need them; a
|
|
34
|
+
`TeamFactory` wanting a `permission_role`, called with nothing, left a
|
|
35
|
+
required column empty and failed as a raw `IntegrityError` naming a
|
|
36
|
+
constraint. That said nothing about what the caller ran, which is the one
|
|
37
|
+
thing every other refusal here avoids. Anything the factory raises is now
|
|
38
|
+
named with the call number, because "it failed" and "it failed after three"
|
|
39
|
+
are different bugs.
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
## [0.15.0] — 2026-09-04
|
|
43
|
+
|
|
44
|
+
### Added
|
|
45
|
+
- **`shape_from_factory` runs a factory you already have and returns source, not
|
|
46
|
+
a `Shape`.** That is the design rather than a limitation of it: a shape this
|
|
47
|
+
package builds is *declared*, which is what makes it reviewable and
|
|
48
|
+
assertable, and a shape learned from a sample is neither -- one used directly
|
|
49
|
+
would change whenever the factory did, silently. So it returns text, and a
|
|
50
|
+
person decides what of it to keep.
|
|
51
|
+
- **What it finds matters more than what it writes**, because a faithful
|
|
52
|
+
reading of a typical factory produces the flat world this package exists to
|
|
53
|
+
argue against. Factories are written for single-object tests, so they fix
|
|
54
|
+
values and reach foreign keys in the two most unrealistic ways there are.
|
|
55
|
+
Every such column is a **finding**, and the report leads with them.
|
|
56
|
+
- **A sub-factory is the sharpest case and the reason to run it.**
|
|
57
|
+
`company = SubFactory(CompanyFactory)` creates one parent per child, which is
|
|
58
|
+
a fan-out of degree one: every parent has exactly one row, the average is the
|
|
59
|
+
truth, and a join over it cannot be misestimated. It is invisible in the
|
|
60
|
+
factory's own source, so it is detected by watching which other tables grew
|
|
61
|
+
and by how much. A round-robin over four parents is the same defect with a
|
|
62
|
+
different number, and is reported too.
|
|
63
|
+
- A relation is rendered as a `FanOut` and never as a value distribution,
|
|
64
|
+
because `Table` refuses the second -- emitting it would hand back source that
|
|
65
|
+
cannot be built, which is worse than a wrong number. A factory that already
|
|
66
|
+
varies everything gets a declaration and **no findings**: if the quiet case
|
|
67
|
+
were not quiet, the loud one would stop meaning anything.
|
|
68
|
+
- Nothing is left behind. The calls run inside a transaction that is rolled
|
|
69
|
+
back, so it can be pointed at a development database without writing to one.
|
|
70
|
+
The sample size is stated in the output, because the answer moves with it.
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
### Fixed
|
|
74
|
+
- **`Md5Keys` could not fill a projection on about half of all tables.**
|
|
75
|
+
`key_sql` wrote `to_hex(<stream>::bigint)`, asking PostgreSQL to re-derive a
|
|
76
|
+
number Python produced as **unsigned** -- and `bigint` is signed, so any
|
|
77
|
+
stream above 2^63 raised `NumericValueOutOfRange` before a row was written.
|
|
78
|
+
The stream is a hash of the table and field names, so it is a coin flip per
|
|
79
|
+
table rather than anything about a schema or a seed: measured, 49% of table
|
|
80
|
+
names land above the limit. The stream is a constant by the time the statement
|
|
81
|
+
is built, so the sixteen hex digits are embedded and nothing is re-derived.
|
|
82
|
+
- **Reported by a consumer, and it had never executed for them.** In 0.13.0
|
|
83
|
+
the join-ambiguity refusal answered first, so `key_sql` was unreachable;
|
|
84
|
+
fixing that in 0.14.0 with `through=` is what exposed a feature 0.13.0 had
|
|
85
|
+
shipped and nobody could run.
|
|
86
|
+
- **The test that was supposed to prove the two halves agree used
|
|
87
|
+
`stream=12345`** -- a number chosen for a test rather than one the producer
|
|
88
|
+
makes. It now uses a pair `field_stream` actually produced, one either side
|
|
89
|
+
of the limit, and asserts that the pair straddles it so a later change to
|
|
90
|
+
how streams are derived cannot quietly make both tests vacuous.
|
|
91
|
+
|
|
92
|
+
|
|
10
93
|
## [0.14.0] — 2026-09-03
|
|
11
94
|
|
|
12
95
|
### Added
|
|
@@ -967,7 +1050,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
967
1050
|
into `COPY FROM STDIN`, which psycopg 2 cannot do without materialising them
|
|
968
1051
|
first.
|
|
969
1052
|
|
|
970
|
-
[Unreleased]: https://github.com/Artui/django-data-shape/compare/v0.
|
|
1053
|
+
[Unreleased]: https://github.com/Artui/django-data-shape/compare/v0.16.0...HEAD
|
|
1054
|
+
[0.16.0]: https://github.com/Artui/django-data-shape/compare/v0.15.0...v0.16.0
|
|
1055
|
+
[0.15.0]: https://github.com/Artui/django-data-shape/compare/v0.14.0...v0.15.0
|
|
971
1056
|
[0.14.0]: https://github.com/Artui/django-data-shape/compare/v0.13.0...v0.14.0
|
|
972
1057
|
[0.13.0]: https://github.com/Artui/django-data-shape/compare/v0.12.0...v0.13.0
|
|
973
1058
|
[0.12.0]: https://github.com/Artui/django-data-shape/compare/v0.11.0...v0.12.0
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: django-data-shape
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.16.0
|
|
4
4
|
Summary: A realistically shaped test database from Django models: declare cardinality, skew and fan-out, load by COPY, and make the query planner believe it.
|
|
5
5
|
Project-URL: Homepage, https://github.com/Artui/django-data-shape
|
|
6
6
|
Project-URL: Repository, https://github.com/Artui/django-data-shape
|
|
@@ -47,6 +47,7 @@ from django_data_shape.scaled_shape import scaled_shape
|
|
|
47
47
|
from django_data_shape.scaled_world import scaled_world
|
|
48
48
|
from django_data_shape.shape import Shape
|
|
49
49
|
from django_data_shape.shape_digest import shape_digest
|
|
50
|
+
from django_data_shape.shape_from_factory import shape_from_factory
|
|
50
51
|
from django_data_shape.shape_not_empty import ShapeNotEmpty
|
|
51
52
|
from django_data_shape.table import Table
|
|
52
53
|
from django_data_shape.table_result import TableResult
|
|
@@ -108,6 +109,7 @@ __all__ = [
|
|
|
108
109
|
"fan_out_sizes",
|
|
109
110
|
"require_clone_strategy",
|
|
110
111
|
"scaled_shape",
|
|
112
|
+
"shape_from_factory",
|
|
111
113
|
"scaled_world",
|
|
112
114
|
"shape_digest",
|
|
113
115
|
"template_database",
|
|
@@ -47,17 +47,26 @@ class Md5Keys:
|
|
|
47
47
|
a test computes both for the same rows and compares them, which is the
|
|
48
48
|
only form of agreement that means anything here.
|
|
49
49
|
|
|
50
|
-
``md5`` takes the same eight-byte big-endian pair Python hashes,
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
50
|
+
``md5`` takes the same eight-byte big-endian pair Python hashes, turned
|
|
51
|
+
back into bytes by ``decode``, so neither side is hashing a rendering of
|
|
52
|
+
the other's input. The stamp is two ``overlay`` calls on the hex digest:
|
|
53
|
+
nibble 13 is the version and is always ``4``, and nibble 17 is the
|
|
54
|
+
variant, which keeps its low two bits and takes ``8`` in the high two.
|
|
55
|
+
|
|
56
|
+
**The stream is embedded as hex rather than converted by the server**,
|
|
57
|
+
and that is a fix rather than a preference. It used to be written
|
|
58
|
+
``to_hex(<stream>::bigint)``, which asks PostgreSQL to re-derive a
|
|
59
|
+
number Python produced as *unsigned* -- and ``bigint`` is signed, so any
|
|
60
|
+
stream above 2^63 raised ``NumericValueOutOfRange`` before a row was
|
|
61
|
+
written. That is a coin flip per table, not a property of any schema:
|
|
62
|
+
the stream is a hash of the table and field names, so roughly half of
|
|
63
|
+
all table names land above the limit. It is a constant by the time this
|
|
64
|
+
statement is built, so nothing about it needs deriving at all.
|
|
65
|
+
|
|
66
|
+
``row`` stays a conversion because it is genuinely an expression the
|
|
67
|
+
server evaluates, and a row index is far below the limit.
|
|
56
68
|
"""
|
|
57
|
-
digest = (
|
|
58
|
-
f"md5(decode(lpad(to_hex({stream}::bigint), 16, '0') || "
|
|
59
|
-
f"lpad(to_hex(({row})::bigint), 16, '0'), 'hex'))"
|
|
60
|
-
)
|
|
69
|
+
digest = f"md5(decode('{stream:016x}' || lpad(to_hex(({row})::bigint), 16, '0'), 'hex'))"
|
|
61
70
|
variant = f"to_hex((('x' || substr({digest}, 17, 1))::bit(4)::int & 3) | 8)"
|
|
62
71
|
return (
|
|
63
72
|
f"(overlay(overlay({digest} placing '4' from 13 for 1) "
|
|
@@ -45,6 +45,7 @@ def resolve_fan_out(
|
|
|
45
45
|
"""
|
|
46
46
|
keys, parent_values = _read_parents(parent, parent_fields, connection, fan_out.parents)
|
|
47
47
|
_require_every_named_parent(fan_out.parents, keys, parent, table, field)
|
|
48
|
+
keys, parent_values = _in_declared_order(fan_out.parents, keys, parent_values)
|
|
48
49
|
|
|
49
50
|
if not keys and rows:
|
|
50
51
|
raise InvalidShape(
|
|
@@ -71,6 +72,51 @@ def resolve_fan_out(
|
|
|
71
72
|
)
|
|
72
73
|
|
|
73
74
|
|
|
75
|
+
def _in_declared_order(
|
|
76
|
+
named: tuple[object, ...] | None,
|
|
77
|
+
keys: list[int],
|
|
78
|
+
parent_values: dict[str, list[object]],
|
|
79
|
+
) -> tuple[list[int], dict[str, list[object]]]:
|
|
80
|
+
"""Put named parents back in the order the declaration named them.
|
|
81
|
+
|
|
82
|
+
**The partition has to be a function of the declaration, and this is what
|
|
83
|
+
makes it one.** Keys arrive ordered by primary key because that is how they
|
|
84
|
+
are read, and the sizes below are assigned by *position* -- so without this
|
|
85
|
+
the weights followed the sort order of values nobody wrote down. With
|
|
86
|
+
integer keys that is merely surprising; with the UUID keys a factory row has
|
|
87
|
+
on a modern schema it means **the same shape builds differently every run**.
|
|
88
|
+
Measured before this existed: one declaration gave the first-named parent 5,
|
|
89
|
+
11 or 79 rows across twelve builds, because the sort order moved with the
|
|
90
|
+
keys.
|
|
91
|
+
|
|
92
|
+
That is the promise the whole package rests on -- two builds of one shape
|
|
93
|
+
agree, which is why ``UuidKeys`` derives rather than draws -- so a narrowing
|
|
94
|
+
that broke it was a defect rather than a preference.
|
|
95
|
+
|
|
96
|
+
**The weights are still scattered across positions**, and the reasoning for
|
|
97
|
+
that survives unchanged: it is a caller's *order* that decides which key
|
|
98
|
+
lands where, not their key values, so nothing here correlates a parent's key
|
|
99
|
+
with its child count. What it does mean is that reversing the list is a
|
|
100
|
+
different declaration, which is what a reader writing one would expect.
|
|
101
|
+
|
|
102
|
+
Parent values move with their keys, or a derivation would read the right
|
|
103
|
+
column off the wrong row.
|
|
104
|
+
"""
|
|
105
|
+
if named is None:
|
|
106
|
+
return keys, parent_values
|
|
107
|
+
# `dict[object, int]` rather than the `list[int]` this module annotates its
|
|
108
|
+
# keys with, because that annotation is not true: a primary key is whatever
|
|
109
|
+
# the model declares, and the UUID case is exactly the one this function
|
|
110
|
+
# exists for. Widening it everywhere is a change of its own; narrowing here
|
|
111
|
+
# would be a lie that type-checks.
|
|
112
|
+
position: dict[object, int] = {key: index for index, key in enumerate(keys)}
|
|
113
|
+
order = [position[key] for key in named]
|
|
114
|
+
return (
|
|
115
|
+
[keys[index] for index in order],
|
|
116
|
+
{name: [values[index] for index in order] for name, values in parent_values.items()},
|
|
117
|
+
)
|
|
118
|
+
|
|
119
|
+
|
|
74
120
|
def _require_every_named_parent(
|
|
75
121
|
named: tuple[object, ...] | None,
|
|
76
122
|
keys: list[int],
|
|
@@ -0,0 +1,252 @@
|
|
|
1
|
+
"""Read what a factory actually produces, and say what it does not."""
|
|
2
|
+
|
|
3
|
+
from __future__ import annotations
|
|
4
|
+
|
|
5
|
+
from collections import Counter
|
|
6
|
+
from collections.abc import Callable, Mapping
|
|
7
|
+
from typing import Any
|
|
8
|
+
|
|
9
|
+
from django.apps import apps
|
|
10
|
+
from django.db import DEFAULT_DB_ALIAS, transaction
|
|
11
|
+
from django.db.models import Field, Model
|
|
12
|
+
|
|
13
|
+
from django_data_shape.invalid_shape import InvalidShape
|
|
14
|
+
|
|
15
|
+
_MANY_VALUES = 12
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
def shape_from_factory(
|
|
19
|
+
factory: Callable[..., Model],
|
|
20
|
+
*,
|
|
21
|
+
samples: int = 200,
|
|
22
|
+
defaults: Mapping[str, object] | None = None,
|
|
23
|
+
using: str = DEFAULT_DB_ALIAS,
|
|
24
|
+
) -> str:
|
|
25
|
+
"""Run a factory, measure what it made, and return a declaration **as source**.
|
|
26
|
+
|
|
27
|
+
**Source to read, edit and check in -- never a ``Shape`` to build from**, and
|
|
28
|
+
that is the whole design rather than a limitation of it. A shape this package
|
|
29
|
+
builds is *declared*, which is what makes it reviewable and assertable; a
|
|
30
|
+
shape learned from a sample is neither, and one used directly would change
|
|
31
|
+
whenever the factory did, silently. So what comes back is text, and a person
|
|
32
|
+
decides what of it to keep.
|
|
33
|
+
|
|
34
|
+
**Because the thing it usually finds is that the factory is flat.** Factories
|
|
35
|
+
are written for single-object tests, so they fix values: one status, one
|
|
36
|
+
parent, one count. Measured on exactly that shape, faithful inference emits
|
|
37
|
+
``status=Constant('ACTIVE'), company=Constant(1)`` -- the uniform world this
|
|
38
|
+
package exists to argue against, now with a declaration blessing it. So every
|
|
39
|
+
such column is reported as a **finding** rather than quietly written down,
|
|
40
|
+
and the report leads with them.
|
|
41
|
+
|
|
42
|
+
**A sub-factory is the sharpest case and the one worth running this for.**
|
|
43
|
+
``company = SubFactory(CompanyFactory)`` creates one parent per child, which
|
|
44
|
+
is a fan-out of degree one: every parent has exactly one row, the average is
|
|
45
|
+
the truth, and the join estimate cannot miss. It is the single most
|
|
46
|
+
unrealistic thing a fixture can do and it is invisible in the factory's own
|
|
47
|
+
source, so it is detected by watching which other tables grew and by how
|
|
48
|
+
much.
|
|
49
|
+
|
|
50
|
+
``defaults`` is passed to every call, because **most factories in a mature
|
|
51
|
+
codebase need arguments**. A ``TeamFactory`` wanting a ``permission_role``,
|
|
52
|
+
called with nothing, leaves a required column empty and fails as a raw
|
|
53
|
+
``IntegrityError`` naming a constraint -- which says nothing about what the
|
|
54
|
+
caller did. Anything the factory raises is caught and named here instead,
|
|
55
|
+
with the call number, because "it failed" and "it failed after three" are
|
|
56
|
+
different bugs.
|
|
57
|
+
|
|
58
|
+
Nothing is left behind: the calls run inside a transaction that is rolled
|
|
59
|
+
back, so this can be pointed at a development database without writing to it.
|
|
60
|
+
|
|
61
|
+
``samples`` decides how much of the tail is seen, and the answer moves with
|
|
62
|
+
it -- fifty runs saw nineteen distinct parents where a thousand saw
|
|
63
|
+
forty-nine. That is stated in the output rather than hidden, because two
|
|
64
|
+
people running this at different sizes should not be surprised by two
|
|
65
|
+
different declarations.
|
|
66
|
+
"""
|
|
67
|
+
if samples < 2:
|
|
68
|
+
raise InvalidShape(
|
|
69
|
+
f"shape_from_factory needs at least two samples to see a shape, got {samples}. "
|
|
70
|
+
"One row cannot tell a constant from a distribution."
|
|
71
|
+
)
|
|
72
|
+
before = _counts(using)
|
|
73
|
+
made: list[Model] = []
|
|
74
|
+
arguments = dict(defaults or {})
|
|
75
|
+
with transaction.atomic(using=using):
|
|
76
|
+
for index in range(samples):
|
|
77
|
+
try:
|
|
78
|
+
row = factory(**arguments)
|
|
79
|
+
except Exception as error:
|
|
80
|
+
# Re-raised rather than propagated, because the failure a caller
|
|
81
|
+
# sees otherwise is a database constraint naming a column, which
|
|
82
|
+
# describes neither what they ran nor what to do about it. The
|
|
83
|
+
# driver's own words are kept: unlike a refusal about a
|
|
84
|
+
# declaration, the column their factory left empty is the whole
|
|
85
|
+
# of the information, and nothing here retains a generated value.
|
|
86
|
+
raise InvalidShape(
|
|
87
|
+
f"the factory could not be called on call {index + 1} of {samples}: "
|
|
88
|
+
f"{type(error).__name__}: {error}. Most factories in a real codebase take "
|
|
89
|
+
"arguments and need them -- pass them with defaults={...}, which is handed "
|
|
90
|
+
"to every call."
|
|
91
|
+
) from error
|
|
92
|
+
if not isinstance(row, Model):
|
|
93
|
+
raise InvalidShape(
|
|
94
|
+
f"shape_from_factory expects the factory to return a model instance, got "
|
|
95
|
+
f"{type(row).__name__}. A factory that returns nothing cannot be measured, "
|
|
96
|
+
"because what it made is what is being read."
|
|
97
|
+
)
|
|
98
|
+
made.append(row)
|
|
99
|
+
after = _counts(using)
|
|
100
|
+
transaction.set_rollback(True, using=using)
|
|
101
|
+
return _render(made[0].__class__, made, _grown(before, after, made[0].__class__), samples)
|
|
102
|
+
|
|
103
|
+
|
|
104
|
+
def _counts(using: str) -> dict[type[Model], int]:
|
|
105
|
+
"""How many rows every model holds, so growth elsewhere can be seen."""
|
|
106
|
+
return {
|
|
107
|
+
model: model._base_manager.using(using).count()
|
|
108
|
+
for config in apps.get_app_configs()
|
|
109
|
+
for model in config.get_models()
|
|
110
|
+
if not model._meta.proxy
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
|
|
114
|
+
def _grown(
|
|
115
|
+
before: dict[type[Model], int], after: dict[type[Model], int], made: type[Model]
|
|
116
|
+
) -> dict[type[Model], int]:
|
|
117
|
+
return {
|
|
118
|
+
model: after[model] - before[model]
|
|
119
|
+
for model in after
|
|
120
|
+
if model is not made and after[model] != before.get(model, 0)
|
|
121
|
+
}
|
|
122
|
+
|
|
123
|
+
|
|
124
|
+
def _describe(values: list[object], rows: int) -> tuple[str, str | None]:
|
|
125
|
+
"""A declaration for one column, and the finding it deserves if any."""
|
|
126
|
+
counts = Counter(values)
|
|
127
|
+
if len(counts) == 1:
|
|
128
|
+
only = next(iter(counts))
|
|
129
|
+
return (
|
|
130
|
+
f"Constant({only!r})",
|
|
131
|
+
f"every row got {only!r}. The factory fixes this column, so the declaration above "
|
|
132
|
+
"says the planner will see one value -- decide what the real spread is.",
|
|
133
|
+
)
|
|
134
|
+
if len(counts) == rows:
|
|
135
|
+
return (
|
|
136
|
+
"Sequential(...) # every value distinct",
|
|
137
|
+
"every value was distinct. Sequential fills that cheaply; a unique short code needs "
|
|
138
|
+
"a generator of its own.",
|
|
139
|
+
)
|
|
140
|
+
if len(counts) <= _MANY_VALUES:
|
|
141
|
+
shares = ", ".join(
|
|
142
|
+
f"{value!r}: {count / rows:.2f}" for value, count in counts.most_common()
|
|
143
|
+
)
|
|
144
|
+
return f"Skew({{{shares}}})", None
|
|
145
|
+
return (
|
|
146
|
+
f"Skew({{...}}) # {len(counts)} distinct values, "
|
|
147
|
+
f"top share {counts.most_common(1)[0][1] / rows:.0%}",
|
|
148
|
+
f"{len(counts)} distinct values is more than a Skew wants written out. If they are "
|
|
149
|
+
"evenly spread, Uniform says it in one line; if they are not, keep the head and let the "
|
|
150
|
+
"tail share a weight.",
|
|
151
|
+
)
|
|
152
|
+
|
|
153
|
+
|
|
154
|
+
def _relation(values: list[object], rows: int) -> tuple[str, str | None]:
|
|
155
|
+
"""A declaration for a foreign key, which is a fan-out and never a draw.
|
|
156
|
+
|
|
157
|
+
Its own function rather than a branch in :func:`_describe`, because the
|
|
158
|
+
answer is a different *kind* of thing. ``Table`` refuses a value
|
|
159
|
+
distribution on a relation -- it would emit keys drawn from nothing -- so
|
|
160
|
+
emitting one here would hand back source that cannot be built, which is a
|
|
161
|
+
worse failure than a wrong number: the reader would take it for a starting
|
|
162
|
+
point and it is not even a valid one.
|
|
163
|
+
|
|
164
|
+
The finding is what matters anyway. A factory reaches a foreign key in one
|
|
165
|
+
of two unrealistic ways and both come out here: a sub-factory gives every
|
|
166
|
+
parent exactly one child, and a round-robin gives every parent the same
|
|
167
|
+
number. Either way the average is the truth and a join over that column
|
|
168
|
+
cannot be misestimated, which is the defect this package exists to make
|
|
169
|
+
reproducible.
|
|
170
|
+
"""
|
|
171
|
+
counts = Counter(values)
|
|
172
|
+
spread = sorted(counts.values(), reverse=True)
|
|
173
|
+
parents = len(counts)
|
|
174
|
+
if parents == 1:
|
|
175
|
+
return (
|
|
176
|
+
"FanOut(Zipf()), # every row pointed at one parent",
|
|
177
|
+
"every row pointed at the same parent. That is not a fan-out at all -- the column has "
|
|
178
|
+
"one value, so the planner sees no distribution across the join.",
|
|
179
|
+
)
|
|
180
|
+
if spread[0] == spread[-1]:
|
|
181
|
+
return (
|
|
182
|
+
f"FanOut(Zipf()), # was flat: {parents} parents, {spread[0]} row(s) each",
|
|
183
|
+
f"every one of the {parents} parents got exactly {spread[0]} row(s). A flat fan-out "
|
|
184
|
+
"is the one shape in which a join estimate cannot miss, because the average is the "
|
|
185
|
+
"truth -- declare the tail you actually have, and childless= for the parents nobody "
|
|
186
|
+
"references.",
|
|
187
|
+
)
|
|
188
|
+
return (
|
|
189
|
+
f"FanOut(Zipf()), # {parents} parents, {spread[0]} at the head and {spread[-1]} at the tail",
|
|
190
|
+
None,
|
|
191
|
+
)
|
|
192
|
+
|
|
193
|
+
|
|
194
|
+
def _render(
|
|
195
|
+
model: type[Model], made: list[Model], grown: dict[type[Model], int], samples: int
|
|
196
|
+
) -> str:
|
|
197
|
+
lines = [
|
|
198
|
+
f"# Measured from {samples} calls. This is source to read and edit, not a shape to",
|
|
199
|
+
"# build from: what it found is what your factory does, which is not the same",
|
|
200
|
+
"# question as what your production data looks like.",
|
|
201
|
+
"",
|
|
202
|
+
]
|
|
203
|
+
findings: list[str] = []
|
|
204
|
+
for model_grown, delta in sorted(grown.items(), key=lambda pair: pair[0].__name__):
|
|
205
|
+
if delta == samples:
|
|
206
|
+
findings.append(
|
|
207
|
+
f"{model_grown.__name__} grew by exactly one per call -- a sub-factory. That is a "
|
|
208
|
+
"fan-out of degree one: every parent has exactly one child, so the average is the "
|
|
209
|
+
"truth and a join over it cannot be misestimated. It is the most unrealistic "
|
|
210
|
+
"thing a fixture can do, and nothing in the factory's source says so."
|
|
211
|
+
)
|
|
212
|
+
else:
|
|
213
|
+
findings.append(
|
|
214
|
+
f"{model_grown.__name__} grew by {delta} over {samples} calls, about "
|
|
215
|
+
f"{samples / delta:.1f} children per parent."
|
|
216
|
+
)
|
|
217
|
+
|
|
218
|
+
body = [f"Table({model.__name__}, rows=...,"]
|
|
219
|
+
for field in model._meta.concrete_fields:
|
|
220
|
+
if field.primary_key:
|
|
221
|
+
continue
|
|
222
|
+
values = [getattr(row, _read(field)) for row in made]
|
|
223
|
+
if field.is_relation:
|
|
224
|
+
declaration, finding = _relation(values, len(made))
|
|
225
|
+
else:
|
|
226
|
+
declaration, finding = _describe(values, len(made))
|
|
227
|
+
body.append(f" {field.name}={declaration},")
|
|
228
|
+
if finding:
|
|
229
|
+
findings.append(f"{model.__name__}.{field.name}: {finding}")
|
|
230
|
+
body.append(")")
|
|
231
|
+
|
|
232
|
+
if findings:
|
|
233
|
+
lines.append("# What your factory does not vary, which is what this is for:")
|
|
234
|
+
lines.extend(f"# - {finding}" for finding in findings)
|
|
235
|
+
lines.append("")
|
|
236
|
+
lines.extend(body)
|
|
237
|
+
lines.append("")
|
|
238
|
+
lines.append(
|
|
239
|
+
f"# Read at {samples} samples. A larger run sees more of the tail, so the numbers above "
|
|
240
|
+
"move with it."
|
|
241
|
+
)
|
|
242
|
+
return "\n".join(lines)
|
|
243
|
+
|
|
244
|
+
|
|
245
|
+
def _read(field: Field[Any, Any]) -> str:
|
|
246
|
+
"""The attribute holding this column's value on an unsaved-or-saved instance.
|
|
247
|
+
|
|
248
|
+
``attname`` rather than ``name`` for a relation, so a foreign key is read as
|
|
249
|
+
the key it stores rather than by fetching the parent -- which would be a
|
|
250
|
+
query per row and would describe the parent instead of this column.
|
|
251
|
+
"""
|
|
252
|
+
return field.attname
|
|
@@ -79,6 +79,11 @@ print(result.rows)
|
|
|
79
79
|
stand in for a fan-out the way it stands in for a scalar. A required relation
|
|
80
80
|
carrying one is refused by name, like any other undeclared required relation.
|
|
81
81
|
A real `db_default` is DDL, so a column carrying one is left to the database.
|
|
82
|
+
- **A factory you already have can be measured, but never learned from.**
|
|
83
|
+
`shape_from_factory` runs one and returns **source to read and edit** — see
|
|
84
|
+
[Reading a factory](#reading-a-factory-you-already-have). It never returns a
|
|
85
|
+
`Shape`, because a shape this package builds is declared, and a declaration
|
|
86
|
+
learned from a sample is neither reviewable nor stable.
|
|
82
87
|
- **A callable default is refused.** `uuid4` varies per row and `dict` does not,
|
|
83
88
|
and nothing on the field distinguishes them. Declare a distribution instead.
|
|
84
89
|
- **A declaration is read-only once built.** Every rule runs in the constructor,
|
|
@@ -171,3 +176,69 @@ declaration from a database or a factory that already exists. The first two are
|
|
|
171
176
|
for a through table points at the escape hatch that does build one today: a
|
|
172
177
|
`Projection` with your own `sql=`, which can select the pairs already
|
|
173
178
|
deduplicated.
|
|
179
|
+
|
|
180
|
+
## Reading a factory you already have
|
|
181
|
+
|
|
182
|
+
```python
|
|
183
|
+
from django_data_shape import shape_from_factory
|
|
184
|
+
|
|
185
|
+
print(shape_from_factory(ProjectFactory, samples=200))
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
**Most factories in a real codebase take arguments and need them.** `defaults=`
|
|
189
|
+
is handed to every call:
|
|
190
|
+
|
|
191
|
+
```python
|
|
192
|
+
print(shape_from_factory(TeamFactory, samples=200, defaults={"permission_role": role}))
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
Called without them, a factory leaves a required column empty and the database
|
|
196
|
+
answers with a constraint name. That failure is caught and named here instead,
|
|
197
|
+
with the call number — because "it failed" and "it failed after three" are
|
|
198
|
+
different bugs.
|
|
199
|
+
|
|
200
|
+
It runs the factory, measures what the calls wrote, rolls the transaction back
|
|
201
|
+
and returns **source**. Not a `Shape` — text, for you to read, edit and check
|
|
202
|
+
in.
|
|
203
|
+
|
|
204
|
+
That is the design and not a limitation of it. A shape this package builds is
|
|
205
|
+
*declared*, which is what makes it reviewable and assertable. A shape learned
|
|
206
|
+
from a sample is neither, and one used directly would change whenever the
|
|
207
|
+
factory did, silently.
|
|
208
|
+
|
|
209
|
+
### The point is what it finds, not what it writes
|
|
210
|
+
|
|
211
|
+
Factories are written for single-object tests, so they fix values and they reach
|
|
212
|
+
foreign keys in the two most unrealistic ways there are. Measured faithfully,
|
|
213
|
+
inference would emit `status=Constant('ACTIVE'), company=FanOut(...)` over one
|
|
214
|
+
parent per child — the uniform world this library exists to argue against, now
|
|
215
|
+
with a declaration blessing it. So those are reported as **findings** and the
|
|
216
|
+
report leads with them:
|
|
217
|
+
|
|
218
|
+
```text
|
|
219
|
+
# What your factory does not vary, which is what this is for:
|
|
220
|
+
# - Company grew by exactly one per call -- a sub-factory. That is a fan-out of
|
|
221
|
+
# degree one: every parent has exactly one child, so the average is the truth
|
|
222
|
+
# and a join over it cannot be misestimated.
|
|
223
|
+
# - Session.label: every row got 's'. The factory fixes this column, so the
|
|
224
|
+
# declaration above says the planner will see one value -- decide what the
|
|
225
|
+
# real spread is.
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
**A sub-factory is the sharpest case and the reason to run this.**
|
|
229
|
+
`company = SubFactory(CompanyFactory)` creates one parent per child, which is
|
|
230
|
+
invisible in the factory's own source and is the single most unrealistic thing a
|
|
231
|
+
fixture can do. A round-robin over four parents is the same defect wearing a
|
|
232
|
+
different number: every parent gets the same count, so the average is still the
|
|
233
|
+
truth.
|
|
234
|
+
|
|
235
|
+
A factory that already varies everything gets a declaration and **no findings** —
|
|
236
|
+
if the quiet case were not quiet, the loud one would stop meaning anything.
|
|
237
|
+
|
|
238
|
+
### What it cannot tell you
|
|
239
|
+
|
|
240
|
+
The sample size decides how much of the tail is seen, and the answer moves with
|
|
241
|
+
it: fifty runs saw nineteen distinct parents where a thousand saw forty-nine.
|
|
242
|
+
The report says which size produced it. And a column with many distinct values is
|
|
243
|
+
reported as a count and a head share rather than written out — a `Skew` listing
|
|
244
|
+
sixty entries is not a declaration anybody keeps.
|
|
@@ -73,6 +73,16 @@ Table(Session, rows=50, company=FanOut(Constant(1), parents=[tenant.pk]), label=
|
|
|
73
73
|
|
|
74
74
|
Real keys, read out of the parent table like any other fan-out — a key that
|
|
75
75
|
matches no row is refused by name rather than quietly taking a smaller share.
|
|
76
|
+
|
|
77
|
+
**The order you name them in is part of the declaration.** Keys come back from
|
|
78
|
+
the database sorted by primary key, and the weights are assigned by position, so
|
|
79
|
+
they are put back into the order you wrote before anything is weighed. Without
|
|
80
|
+
that the partition followed the *values* of the keys: with the UUID primary keys
|
|
81
|
+
a factory row has, one declaration gave the first-named parent 5, 11 or 79 rows
|
|
82
|
+
across twelve builds. Two builds of one shape have to agree, which is the same
|
|
83
|
+
reason `UuidKeys` derives rather than draws. Reversing the list is therefore a
|
|
84
|
+
different declaration, and weights are still scattered across positions — so
|
|
85
|
+
nothing correlates a parent's key with its child count.
|
|
76
86
|
A **list of keys, never a queryset**: a declaration needs no connection, and
|
|
77
87
|
evaluating a queryset here would give it one. Build the list yourself, which is
|
|
78
88
|
the query you were going to run anyway.
|