openstatspec 0.2.0__tar.gz → 0.4.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (67) hide show
  1. {openstatspec-0.2.0 → openstatspec-0.4.0}/.github/workflows/ci.yml +23 -10
  2. {openstatspec-0.2.0 → openstatspec-0.4.0}/.github/workflows/release.yml +1 -1
  3. openstatspec-0.4.0/CHANGELOG.md +123 -0
  4. {openstatspec-0.2.0 → openstatspec-0.4.0}/PKG-INFO +52 -5
  5. {openstatspec-0.2.0 → openstatspec-0.4.0}/README.md +51 -4
  6. openstatspec-0.4.0/docs/in-place-transformation.md +9 -0
  7. openstatspec-0.4.0/docs/release-readiness.md +110 -0
  8. {openstatspec-0.2.0 → openstatspec-0.4.0}/docs/sav-profile.md +21 -6
  9. openstatspec-0.4.0/docs/transformations.md +204 -0
  10. {openstatspec-0.2.0 → openstatspec-0.4.0}/pyproject.toml +1 -1
  11. openstatspec-0.4.0/src/openstatspec/__init__.py +40 -0
  12. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/api.py +49 -1
  13. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/cli.py +63 -2
  14. openstatspec-0.4.0/src/openstatspec/frontends/__init__.py +1 -0
  15. openstatspec-0.4.0/src/openstatspec/frontends/sas/.gitkeep +0 -0
  16. openstatspec-0.4.0/src/openstatspec/frontends/spss/__init__.py +26 -0
  17. openstatspec-0.4.0/src/openstatspec/frontends/spss/binding.py +265 -0
  18. openstatspec-0.4.0/src/openstatspec/frontends/spss/compiler.py +51 -0
  19. openstatspec-0.4.0/src/openstatspec/frontends/spss/execution.py +48 -0
  20. openstatspec-0.4.0/src/openstatspec/frontends/spss/syntax.py +478 -0
  21. openstatspec-0.4.0/src/openstatspec/frontends/stata/.gitkeep +0 -0
  22. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/sql/capabilities.py +42 -16
  23. openstatspec-0.4.0/src/openstatspec/sql/inplace_transform.py +682 -0
  24. openstatspec-0.4.0/src/openstatspec/transform/__init__.py +52 -0
  25. openstatspec-0.4.0/src/openstatspec/transform/errors.py +59 -0
  26. openstatspec-0.4.0/src/openstatspec/transform/plan.py +415 -0
  27. openstatspec-0.4.0/src/openstatspec/transform/schema.py +46 -0
  28. openstatspec-0.4.0/src/openstatspec/transform/validation.py +178 -0
  29. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_cli.py +66 -3
  30. openstatspec-0.4.0/tests/test_inplace_transform.py +508 -0
  31. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_sql_profiles.py +122 -25
  32. openstatspec-0.4.0/tests/test_transform_frontend.py +367 -0
  33. openstatspec-0.2.0/CHANGELOG.md +0 -59
  34. openstatspec-0.2.0/docs/release-readiness.md +0 -70
  35. openstatspec-0.2.0/src/openstatspec/__init__.py +0 -22
  36. {openstatspec-0.2.0 → openstatspec-0.4.0}/.github/verify_release_ref.py +0 -0
  37. {openstatspec-0.2.0 → openstatspec-0.4.0}/.gitignore +0 -0
  38. {openstatspec-0.2.0 → openstatspec-0.4.0}/THIRD_PARTY_NOTICES.md +0 -0
  39. {openstatspec-0.2.0 → openstatspec-0.4.0}/docs/architecture.md +0 -0
  40. {openstatspec-0.2.0 → openstatspec-0.4.0}/docs/pyspssio-spike.md +0 -0
  41. {openstatspec-0.2.0 → openstatspec-0.4.0}/docs/sql-transformation-workflow.md +0 -0
  42. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/core/__init__.py +0 -0
  43. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/core/results.py +0 -0
  44. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/spss/__init__.py +0 -0
  45. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/spss/dictionary.py +0 -0
  46. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/spss/raw_dictionary.py +0 -0
  47. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/spss/sav.py +0 -0
  48. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/sql/__init__.py +0 -0
  49. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/sql/catalog_api.py +0 -0
  50. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/sql/normative.py +0 -0
  51. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/sql/profiles.py +0 -0
  52. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/sql/wide.py +0 -0
  53. {openstatspec-0.2.0 → openstatspec-0.4.0}/src/openstatspec/sql/workflow.py +0 -0
  54. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/conformance.py +0 -0
  55. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_atomic_import.py +0 -0
  56. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_attribute_catalog.py +0 -0
  57. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_document_round_trip.py +0 -0
  58. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_loss_reports.py +0 -0
  59. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_normative_catalog.py +0 -0
  60. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_official_conformance.py +0 -0
  61. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_pyspssio_catalog_authority.py +0 -0
  62. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_release_ref.py +0 -0
  63. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_sav_sqlite.py +0 -0
  64. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_spss_catalog_preflight.py +0 -0
  65. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_sql_services.py +0 -0
  66. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_sql_workflow.py +0 -0
  67. {openstatspec-0.2.0 → openstatspec-0.4.0}/tests/test_vls_compatible_names.py +0 -0
@@ -20,7 +20,7 @@ jobs:
20
20
  uses: actions/checkout@v7
21
21
  with: &specification-checkout
22
22
  repository: OpenStatSpec/specification
23
- ref: fef0dc6f4b17ff7141dad3f49d0524c63efbfed5
23
+ ref: 79339ec3d8f8aa81789b7e85f6b8afa6f1374e50
24
24
  path: openstatspec-specification
25
25
  - name: Checkout required SPSS engine
26
26
  uses: actions/checkout@v7
@@ -63,7 +63,7 @@ jobs:
63
63
  strategy:
64
64
  fail-fast: false
65
65
  matrix:
66
- postgres: ["17", "18"]
66
+ postgres: ["17.10", "18.4"]
67
67
  services:
68
68
  postgres:
69
69
  image: postgres:${{ matrix.postgres }}
@@ -77,6 +77,7 @@ jobs:
77
77
  --health-interval 10s --health-timeout 5s --health-retries 5
78
78
  env:
79
79
  OPENSTATSPEC_POSTGRES_URL: postgresql+psycopg://openstatspec:openstatspec@localhost:5432/openstatspec
80
+ OPENSTATSPEC_EXPECTED_POSTGRES_VERSION: ${{ matrix.postgres }}
80
81
  steps: &sql-test-steps
81
82
  - uses: actions/checkout@v7
82
83
  - name: Checkout OpenStatSpec specification fixtures
@@ -99,7 +100,7 @@ jobs:
99
100
  strategy:
100
101
  fail-fast: false
101
102
  matrix:
102
- mysql: ["8.4", "9.7"]
103
+ mysql: ["8.4.11", "9.7.2"]
103
104
  services:
104
105
  mysql:
105
106
  image: mysql:${{ matrix.mysql }}
@@ -114,6 +115,7 @@ jobs:
114
115
  --health-interval 10s --health-timeout 5s --health-retries 10
115
116
  env:
116
117
  OPENSTATSPEC_MYSQL_URL: mysql+pymysql://openstatspec:openstatspec@localhost:3306/openstatspec
118
+ OPENSTATSPEC_EXPECTED_MYSQL_VERSION: ${{ matrix.mysql }}
117
119
  steps: *sql-test-steps
118
120
 
119
121
  mariadb-integration:
@@ -122,7 +124,7 @@ jobs:
122
124
  strategy:
123
125
  fail-fast: false
124
126
  matrix:
125
- mariadb: ["11.4", "11.8", "12.3"]
127
+ mariadb: ["11.4.12", "11.8.8", "12.3.2"]
126
128
  services:
127
129
  mariadb:
128
130
  image: mariadb:${{ matrix.mariadb }}
@@ -137,13 +139,24 @@ jobs:
137
139
  --health-interval 10s --health-timeout 5s --health-retries 10
138
140
  env:
139
141
  OPENSTATSPEC_MARIADB_URL: mysql+pymysql://openstatspec:openstatspec@localhost:3306/openstatspec
142
+ OPENSTATSPEC_EXPECTED_MARIADB_VERSION: ${{ matrix.mariadb }}
140
143
  steps: *sql-test-steps
141
144
 
142
145
  dolt-integration:
143
- name: Dolt 2.2.2 integration
146
+ name: Dolt ${{ matrix.dolt }} integration
144
147
  runs-on: ubuntu-latest
148
+ strategy:
149
+ fail-fast: false
150
+ matrix:
151
+ include:
152
+ - dolt: "2.2.2"
153
+ image: dolthub/dolt:2.2.2@sha256:ae85afdd95317f0a982cab87fa3092a5a2e6689bc5de822ad7ae4f65b86d39a1
154
+ - dolt: "2.2.3"
155
+ image: dolthub/dolt:2.2.3@sha256:1a651e738a2e7275b9c69fec8c5f42c6e23954314405c412f4dc9287ee7c0080
145
156
  env:
146
157
  OPENSTATSPEC_DOLT_URL: mysql+pymysql://openstatspec:openstatspec@127.0.0.1:13308/openstatspec
158
+ OPENSTATSPEC_EXPECTED_DOLT_VERSION: ${{ matrix.dolt }}
159
+ OPENSTATSPEC_DOLT_IMAGE: ${{ matrix.image }}
147
160
  steps:
148
161
  - uses: actions/checkout@v7
149
162
  - name: Checkout OpenStatSpec specification fixtures
@@ -155,29 +168,29 @@ jobs:
155
168
  - uses: actions/setup-python@v7
156
169
  with:
157
170
  python-version: "3.12"
158
- - name: Initialize Dolt 2.2.2
171
+ - name: Initialize Dolt ${{ matrix.dolt }}
159
172
  run: |
160
173
  mkdir -p "$RUNNER_TEMP/openstatspec-dolt"
161
174
  docker run --rm \
162
175
  --volume "$RUNNER_TEMP/openstatspec-dolt:/var/lib/dolt" \
163
176
  --workdir /var/lib/dolt \
164
- dolthub/dolt:2.2.2 init \
177
+ "$OPENSTATSPEC_DOLT_IMAGE" init \
165
178
  --name "OpenStatSpec CI" --email "ci@openstatspec.org"
166
179
  docker run --rm \
167
180
  --volume "$RUNNER_TEMP/openstatspec-dolt:/var/lib/dolt" \
168
181
  --workdir /var/lib/dolt \
169
- dolthub/dolt:2.2.2 sql \
182
+ "$OPENSTATSPEC_DOLT_IMAGE" sql \
170
183
  -q "CREATE DATABASE openstatspec"
171
184
  docker run --rm \
172
185
  --volume "$RUNNER_TEMP/openstatspec-dolt:/var/lib/dolt" \
173
186
  --workdir /var/lib/dolt \
174
- dolthub/dolt:2.2.2 sql \
187
+ "$OPENSTATSPEC_DOLT_IMAGE" sql \
175
188
  -q "CREATE USER 'openstatspec'@'%' IDENTIFIED BY 'openstatspec'; GRANT ALL PRIVILEGES ON openstatspec.* TO 'openstatspec'@'%'"
176
189
  docker run --detach --name openstatspec-dolt \
177
190
  --publish 127.0.0.1:13308:3306 \
178
191
  --volume "$RUNNER_TEMP/openstatspec-dolt:/var/lib/dolt" \
179
192
  --workdir /var/lib/dolt \
180
- dolthub/dolt:2.2.2 sql-server -H 0.0.0.0 -l warning
193
+ "$OPENSTATSPEC_DOLT_IMAGE" sql-server -H 0.0.0.0 -l warning
181
194
  - run: python -m pip install --upgrade pip
182
195
  - run: python -m pip install ./openstatspec-pyspssio
183
196
  - run: python -m pip install -e ".[dev,sql]"
@@ -47,7 +47,7 @@ jobs:
47
47
  uses: actions/checkout@v7
48
48
  with:
49
49
  repository: OpenStatSpec/specification
50
- ref: fef0dc6f4b17ff7141dad3f49d0524c63efbfed5
50
+ ref: 79339ec3d8f8aa81789b7e85f6b8afa6f1374e50
51
51
  path: openstatspec-specification
52
52
  - name: Checkout required SPSS engine
53
53
  uses: actions/checkout@v7
@@ -0,0 +1,123 @@
1
+ # Changelog
2
+
3
+ All notable changes to this reference implementation are documented here.
4
+
5
+ ## 0.4.0 — 2026-07-31
6
+
7
+ ### Added
8
+
9
+ - Added a public canonical transformation-plan in-place API accepting either
10
+ typed plan objects or strictly validated mappings.
11
+ - Added apply-plan and install-in-place-schema CLI workflows alongside the
12
+ compatible apply-spss command.
13
+ - Added explicit audit provenance distinguishing canonical plan documents from
14
+ SPSS syntax sources.
15
+ - Added a transformation manual covering the canonical core, frontend boundary,
16
+ execution invariants, CLI/API workflows, and extension rules.
17
+
18
+ ### Changed
19
+
20
+ - Moved the SPSS parser, binder, compiler, and convenience execution adapter to
21
+ the dedicated openstatspec.frontends.spss package.
22
+ - Kept openstatspec.transform language-neutral with canonical plan, schema, and
23
+ live-schema validation modules while preserving previous SPSS import paths.
24
+ - Reserved empty Stata and SAS frontend directories without exposing parsers,
25
+ capabilities, CLI choices, or support claims.
26
+ - Strengthened pre-mutation target-type checks and target-scoped physical table
27
+ identity guards. The executor continues to modify the same dataset and table
28
+ without creating OpenStatSpec rollback, copy, snapshot, or history layers.
29
+
30
+ ### Specification basis
31
+
32
+ - Conformance and release validation continue to use OpenStatSpec specification
33
+ release v0.2.0 at exact commit
34
+ 79339ec3d8f8aa81789b7e85f6b8afa6f1374e50.
35
+
36
+ ## 0.3.0 — 2026-07-31
37
+
38
+ ### Added
39
+
40
+ - Added a bounded SPSS-like `RECODE`, `VARIABLE LABELS`, and `VALUE LABELS`
41
+ parser with stable diagnostics and conformance fixtures.
42
+ - Added canonical Transformation Plan serialization, RFC 8785 hashes, schema
43
+ binding, deterministic typed operations, and an agent-facing CLI/API.
44
+ - Added product-neutral in-place execution with same dataset/table identity,
45
+ direct data and metadata mutation, and compact operation audit. Dolt adds
46
+ expected branch/HEAD and clean-working-set checks.
47
+
48
+ ### Changed
49
+
50
+ - Version history, diff, rollback, restoration, and commit remain database
51
+ responsibilities; the transformer creates no derived/copy/snapshot/recovery
52
+ layer and performs no `DOLT_COMMIT`.
53
+ - SQL server support now distinguishes conservative family claims from exact
54
+ CI evidence: PostgreSQL 17.x/18.x at 17.10/18.4, MySQL 8.4.x/9.7.x at
55
+ 8.4.11/9.7.2, and MariaDB 11.4.x/11.8.x/12.3.x at
56
+ 11.4.12/11.8.8/12.3.2. Each service job verifies the live normalized version.
57
+ - Dolt remains an independent core profile, now claiming canonical stable
58
+ versions `>=2.2.2,<2.3.0` and running its complete service suite at exact
59
+ versions 2.2.2 and 2.2.3 from immutable container-image pins.
60
+ SQLite retains its core `>=3.24.0,<4.0.0` and optional workflow
61
+ `>=3.35.0,<4.0.0` tiers. MSSQL remains unsupported and roadmap-only.
62
+
63
+ ### Specification basis
64
+
65
+ - CI, release validation, and machine-readable capabilities use OpenStatSpec
66
+ specification release `v0.2.0` at exact commit
67
+ `79339ec3d8f8aa81789b7e85f6b8afa6f1374e50`.
68
+
69
+ ## 0.2.0 — 2026-07-30
70
+
71
+ This release extends the reference adapter while keeping imported source
72
+ datasets immutable and all unsupported paths fail-closed.
73
+
74
+ ### Added
75
+
76
+ - An optional SQLite-only Transformation Workflow for immutable, versioned,
77
+ parameterized SELECT definitions; materialized derived datasets; lineage and
78
+ weights; public catalog APIs; and CLI operations.
79
+ - An independent Dolt core SQL profile for exact product version 2.2.2, with
80
+ service-backed coverage, conservative adapter limits, complete compensating
81
+ cleanup, and pre-DDL rejection of unsupported values. Transformation Workflow
82
+ support on Dolt is not claimed.
83
+
84
+ ### Changed
85
+
86
+ - SAV and ZSAV validation now rewrites legacy compatible names consistently
87
+ across type-2 records and subtype-13/subtype-14 long-name metadata.
88
+ - Transformation publication, interrupted-run reconciliation, retirement, and
89
+ physical-removal recovery now preserve auditable state and fail closed on
90
+ catalog or relation drift.
91
+ - Release automation builds distributions before test-only source checkouts and
92
+ supports idempotent rebuilds of an existing release tag.
93
+
94
+ ### Specification basis
95
+
96
+ - CI and release validation are pinned to OpenStatSpec specification release
97
+ `v1.0.0-rc.1` at exact commit
98
+ `fef0dc6f4b17ff7141dad3f49d0524c63efbfed5`.
99
+
100
+ ## 0.1.0 — 2026-07-29
101
+
102
+ First public reference implementation of the OpenStatSpec strict wide-table
103
+ SPSS profile.
104
+
105
+ ### Included
106
+
107
+ - Import of unencrypted SAV and ZSAV sources into one dedicated wide SQL table
108
+ and its metadata catalog.
109
+ - Export of supported dataset semantics to SAV and ZSAV.
110
+ - SQLite, PostgreSQL, MySQL, and MariaDB profiles, including service-backed
111
+ PostgreSQL 17/18, MySQL 8.4/9.7, and MariaDB 11.4/11.8/12.3 CI coverage.
112
+ - Preflight checks for target profile limits, atomic imports, validation, a
113
+ command-line interface, and machine-readable capability and loss reports.
114
+
115
+ ### Important boundaries
116
+
117
+ - This release promises semantic round-trip equivalence only for supported
118
+ features; it never promises byte-identical SAV/ZSAV output.
119
+ - The adapter requires explicit `allow_loss` consent before exporting known
120
+ unsupported or lossy dictionary semantics.
121
+ - Multiple-response sets, variable alignment, variable sets, and custom
122
+ attributes have explicit capability diagnostics; see the SAV profile.
123
+ - Encrypted SPSS files are not supported.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: openstatspec
3
- Version: 0.2.0
3
+ Version: 0.4.0
4
4
  Summary: Reference adapter for the OpenStatSpec relational contract
5
5
  Author: OpenStatSpec
6
6
  License: Apache-2.0
@@ -45,6 +45,10 @@ import.
45
45
  capabilities, and loss reports.
46
46
  - `openstatspec.sql`: database connection and wide-table/catalog operations.
47
47
  - `openstatspec.spss`: SAV/ZSAV adapter boundary.
48
+ - `openstatspec.transform`: canonical plans, frontend-neutral schema concepts,
49
+ and plan validation.
50
+ - `openstatspec.frontends.spss`: the SPSS-like syntax frontend and convenience
51
+ execution adapter.
48
52
 
49
53
  ## Intended workflow
50
54
 
@@ -69,12 +73,32 @@ derived datasets through a public catalog API. It uses a separate profile
69
73
  catalog and never presents SQL output as an imported source dataset.
70
74
  Workflow operations support SQLite only in this milestone and fail closed on
71
75
  PostgreSQL/MySQL/MariaDB; core import/export database support is unchanged.
76
+ The core SQLite import/export profile accepts SQLite `>=3.24.0,<4.0.0`; the
77
+ optional transformation workflow deliberately has the narrower
78
+ `>=3.35.0,<4.0.0` runtime preflight. These independent tiers do not change the
79
+ server-profile matrix. Microsoft SQL Server is not supported; its future
80
+ dialect is scoped only in the specification's
81
+ [MSSQL roadmap](https://github.com/OpenStatSpec/specification/blob/main/docs/mssql-dialect-roadmap.md).
72
82
 
73
83
  See [the SQL transformation workflow](docs/sql-transformation-workflow.md) for
74
84
  Python and CLI examples, migration behavior, hashing, atomicity, and the exact
75
85
  implemented capability boundary.
76
86
 
77
- ## 0.2.0 support status
87
+ ## SPSS-like transformation frontend
88
+
89
+ The SPSS-like frontend lowers supported `RECODE`, `VARIABLE LABELS`, and
90
+ `VALUE LABELS` syntax into a language-neutral canonical plan. The in-place
91
+ path applies it to the same logical dataset, physical wide table, and metadata
92
+ catalog without a derived dataset, copied table, snapshot, or separate
93
+ rollback/history layer. Dolt remains the sole versioning layer for Dolt-backed
94
+ edits, and the transformer never calls `DOLT_COMMIT`.
95
+
96
+ See the [dataset transformations manual](docs/transformations.md) for schema
97
+ installation, Python and CLI surfaces, database invariants, audit provenance,
98
+ package layout, and extension guidance. Stata and SAS are unimplemented
99
+ placeholders.
100
+
101
+ ## Current support status
78
102
 
79
103
  The adapter requires `openstatspec-pyspssio==0.5.1.post2` as its sole SPSS
80
104
  engine. Its import module remains `pyspssio`; the exact source commit is recorded
@@ -82,13 +106,36 @@ in operation metadata. There is no fallback reader or writer. It supports unencr
82
106
  SAV/ZSAV export for the semantics exposed by that engine. SQLite is the local
83
107
  reference path.
84
108
  PostgreSQL, MySQL, MariaDB, and Dolt are each covered by separate service-backed CI
85
- conformance checks. Dolt support is an independent core profile pinned to 2.2.2;
86
- other Dolt versions and unknown MySQL-wire products fail closed. Use these explicit SQLAlchemy URLs:
109
+ conformance checks. Dolt support is an independent core profile for the
110
+ canonical stable range `>=2.2.2,<2.3.0`; earlier patches, other families,
111
+ noncanonical versions, and unknown MySQL-wire products fail closed.
112
+
113
+ The supported family claims are broader than the deliberately exact CI
114
+ evidence points: PostgreSQL 17.x/18.x is exercised at 17.10/18.4, MySQL
115
+ 8.4.x/9.7.x at 8.4.11/9.7.2, and MariaDB 11.4.x/11.8.x/12.3.x at
116
+ 11.4.12/11.8.8/12.3.2. Each service job checks the normalized live server
117
+ version against its exact matrix entry before that run can count as evidence.
118
+ Dolt claims the conservative 2.2.x range `>=2.2.2,<2.3.0`; its full service
119
+ suite is exercised independently at exact versions 2.2.2 and 2.2.3 using
120
+ immutable container-image digests.
121
+
122
+ | Engine/profile | Runtime supported policy | Exact CI-tested versions |
123
+ | --- | --- | --- |
124
+ | SQLite core / optional workflow | Core `>=3.24.0,<4.0.0`; optional workflow `>=3.35.0,<4.0.0` | Runtime-provided SQLite on Python 3.11–3.14 runners; not a pinned server image |
125
+ | PostgreSQL | 17.x and 18.x | 17.10 and 18.4 |
126
+ | MySQL | 8.4.x and 9.7.x | 8.4.11 and 9.7.2 |
127
+ | MariaDB | 11.4.x, 11.8.x, and 12.3.x | 11.4.12, 11.8.8, and 12.3.2 |
128
+ | Dolt | 2.2.x with `>=2.2.2,<2.3.0` | 2.2.2 and 2.2.3 |
129
+
130
+ Microsoft SQL Server (MSSQL) remains roadmap-only and is not a supported
131
+ runtime profile; see the specification's [MSSQL roadmap](https://github.com/OpenStatSpec/specification/blob/main/docs/mssql-dialect-roadmap.md).
132
+
133
+ Use these explicit SQLAlchemy URLs:
87
134
 
88
135
  - SQLite: `sqlite:///dataset.sqlite`
89
136
  - PostgreSQL: `postgresql+psycopg://user:password@host/database`
90
137
  - MySQL/MariaDB: `mysql+pymysql://user:password@host/database`
91
- - Dolt 2.2.2: `mysql+pymysql://user:password@host/database` (detected by server identity)
138
+ - Dolt `>=2.2.2,<2.3.0`: `mysql+pymysql://user:password@host/database` (detected by server identity)
92
139
 
93
140
  The Dolt core profile supports strict wide-table import, validation, and export;
94
141
  the separate Transformation Workflow is unsupported.
@@ -26,6 +26,10 @@ import.
26
26
  capabilities, and loss reports.
27
27
  - `openstatspec.sql`: database connection and wide-table/catalog operations.
28
28
  - `openstatspec.spss`: SAV/ZSAV adapter boundary.
29
+ - `openstatspec.transform`: canonical plans, frontend-neutral schema concepts,
30
+ and plan validation.
31
+ - `openstatspec.frontends.spss`: the SPSS-like syntax frontend and convenience
32
+ execution adapter.
29
33
 
30
34
  ## Intended workflow
31
35
 
@@ -50,12 +54,32 @@ derived datasets through a public catalog API. It uses a separate profile
50
54
  catalog and never presents SQL output as an imported source dataset.
51
55
  Workflow operations support SQLite only in this milestone and fail closed on
52
56
  PostgreSQL/MySQL/MariaDB; core import/export database support is unchanged.
57
+ The core SQLite import/export profile accepts SQLite `>=3.24.0,<4.0.0`; the
58
+ optional transformation workflow deliberately has the narrower
59
+ `>=3.35.0,<4.0.0` runtime preflight. These independent tiers do not change the
60
+ server-profile matrix. Microsoft SQL Server is not supported; its future
61
+ dialect is scoped only in the specification's
62
+ [MSSQL roadmap](https://github.com/OpenStatSpec/specification/blob/main/docs/mssql-dialect-roadmap.md).
53
63
 
54
64
  See [the SQL transformation workflow](docs/sql-transformation-workflow.md) for
55
65
  Python and CLI examples, migration behavior, hashing, atomicity, and the exact
56
66
  implemented capability boundary.
57
67
 
58
- ## 0.2.0 support status
68
+ ## SPSS-like transformation frontend
69
+
70
+ The SPSS-like frontend lowers supported `RECODE`, `VARIABLE LABELS`, and
71
+ `VALUE LABELS` syntax into a language-neutral canonical plan. The in-place
72
+ path applies it to the same logical dataset, physical wide table, and metadata
73
+ catalog without a derived dataset, copied table, snapshot, or separate
74
+ rollback/history layer. Dolt remains the sole versioning layer for Dolt-backed
75
+ edits, and the transformer never calls `DOLT_COMMIT`.
76
+
77
+ See the [dataset transformations manual](docs/transformations.md) for schema
78
+ installation, Python and CLI surfaces, database invariants, audit provenance,
79
+ package layout, and extension guidance. Stata and SAS are unimplemented
80
+ placeholders.
81
+
82
+ ## Current support status
59
83
 
60
84
  The adapter requires `openstatspec-pyspssio==0.5.1.post2` as its sole SPSS
61
85
  engine. Its import module remains `pyspssio`; the exact source commit is recorded
@@ -63,13 +87,36 @@ in operation metadata. There is no fallback reader or writer. It supports unencr
63
87
  SAV/ZSAV export for the semantics exposed by that engine. SQLite is the local
64
88
  reference path.
65
89
  PostgreSQL, MySQL, MariaDB, and Dolt are each covered by separate service-backed CI
66
- conformance checks. Dolt support is an independent core profile pinned to 2.2.2;
67
- other Dolt versions and unknown MySQL-wire products fail closed. Use these explicit SQLAlchemy URLs:
90
+ conformance checks. Dolt support is an independent core profile for the
91
+ canonical stable range `>=2.2.2,<2.3.0`; earlier patches, other families,
92
+ noncanonical versions, and unknown MySQL-wire products fail closed.
93
+
94
+ The supported family claims are broader than the deliberately exact CI
95
+ evidence points: PostgreSQL 17.x/18.x is exercised at 17.10/18.4, MySQL
96
+ 8.4.x/9.7.x at 8.4.11/9.7.2, and MariaDB 11.4.x/11.8.x/12.3.x at
97
+ 11.4.12/11.8.8/12.3.2. Each service job checks the normalized live server
98
+ version against its exact matrix entry before that run can count as evidence.
99
+ Dolt claims the conservative 2.2.x range `>=2.2.2,<2.3.0`; its full service
100
+ suite is exercised independently at exact versions 2.2.2 and 2.2.3 using
101
+ immutable container-image digests.
102
+
103
+ | Engine/profile | Runtime supported policy | Exact CI-tested versions |
104
+ | --- | --- | --- |
105
+ | SQLite core / optional workflow | Core `>=3.24.0,<4.0.0`; optional workflow `>=3.35.0,<4.0.0` | Runtime-provided SQLite on Python 3.11–3.14 runners; not a pinned server image |
106
+ | PostgreSQL | 17.x and 18.x | 17.10 and 18.4 |
107
+ | MySQL | 8.4.x and 9.7.x | 8.4.11 and 9.7.2 |
108
+ | MariaDB | 11.4.x, 11.8.x, and 12.3.x | 11.4.12, 11.8.8, and 12.3.2 |
109
+ | Dolt | 2.2.x with `>=2.2.2,<2.3.0` | 2.2.2 and 2.2.3 |
110
+
111
+ Microsoft SQL Server (MSSQL) remains roadmap-only and is not a supported
112
+ runtime profile; see the specification's [MSSQL roadmap](https://github.com/OpenStatSpec/specification/blob/main/docs/mssql-dialect-roadmap.md).
113
+
114
+ Use these explicit SQLAlchemy URLs:
68
115
 
69
116
  - SQLite: `sqlite:///dataset.sqlite`
70
117
  - PostgreSQL: `postgresql+psycopg://user:password@host/database`
71
118
  - MySQL/MariaDB: `mysql+pymysql://user:password@host/database`
72
- - Dolt 2.2.2: `mysql+pymysql://user:password@host/database` (detected by server identity)
119
+ - Dolt `>=2.2.2,<2.3.0`: `mysql+pymysql://user:password@host/database` (detected by server identity)
73
120
 
74
121
  The Dolt core profile supports strict wide-table import, validation, and export;
75
122
  the separate Transformation Workflow is unsupported.
@@ -0,0 +1,9 @@
1
+ # In-place transformations
2
+
3
+ This page has moved to the comprehensive
4
+ [dataset transformations manual](transformations.md).
5
+
6
+ The manual covers the canonical plan, SPSS-like frontend, executor, audit
7
+ schema, Python and CLI surfaces, database and Dolt invariants, provenance,
8
+ package layout, and future frontend boundary. Stata and SAS remain
9
+ unimplemented placeholders.
@@ -0,0 +1,110 @@
1
+ # 0.4.0 release readiness
2
+
3
+ This page records the expected release contract, not a publication event.
4
+ Creating a version tag remains a separate maintainer action.
5
+
6
+ ## Supported workflow
7
+
8
+ For a supported unencrypted SAV or ZSAV source, the adapter imports one source
9
+ dataset into exactly one dedicated wide data table plus the complete singular,
10
+ UUID-keyed normative catalog from the SPSS profile. It can validate that
11
+ representation and export it back to SAV or ZSAV. Historical `*_catalog`
12
+ tables remain private compatibility storage and do not replace the normative
13
+ database contract.
14
+
15
+ ## SQL profiles
16
+
17
+ | Profile | Connection URL | Claimed versions | Exact CI evidence |
18
+ | --- | --- | --- | --- |
19
+ | SQLite | `sqlite:///dataset.sqlite` | `>=3.24.0,<4.0.0` | Active Python 3.11–3.14 runner version |
20
+ | PostgreSQL | `postgresql+psycopg://…` | 17.x and 18.x | 17.10 and 18.4 |
21
+ | MySQL | `mysql+pymysql://…` | 8.4.x and 9.7.x | 8.4.11 and 9.7.2 |
22
+ | MariaDB | `mysql+pymysql://…` | 11.4.x, 11.8.x, and 12.3.x | 11.4.12, 11.8.8, and 12.3.2 |
23
+ | Dolt | `mysql+pymysql://…` | 2.2.x with `>=2.2.2,<2.3.0` | 2.2.2 and 2.2.3 |
24
+
25
+ Separate service matrices test the shared MySQL/MariaDB profile contract while
26
+ retaining distinct active-server identities and version claims. Dolt is an
27
+ independent core profile that accepts only canonical stable versions in
28
+ `>=2.2.2,<2.3.0`; the optional Transformation Workflow remains SQLite-only. This does not
29
+ claim coverage for every server configuration. The machine-readable capability
30
+ declaration reports both the theoretical profile boundaries and effective
31
+ limits observed from the active connection.
32
+ Every server service job compares the normalized live product version with its
33
+ exact matrix entry, so a moved or mismatched image cannot substantiate the CI
34
+ declaration. SQLite's optional Transformation Workflow retains its narrower
35
+ `>=3.35.0,<4.0.0` live preflight alongside the core `>=3.24.0,<4.0.0` policy.
36
+ Microsoft SQL Server remains unsupported and must not appear in runtime
37
+ capabilities; future implementation requirements are documented in the
38
+ specification's
39
+ [MSSQL dialect roadmap](https://github.com/OpenStatSpec/specification/blob/main/docs/mssql-dialect-roadmap.md).
40
+
41
+
42
+ ## Export-loss policy
43
+
44
+ `capability_matrix()` and `openstatspec capabilities` expose the current
45
+ feature state. Before export, the adapter combines source and writer
46
+ diagnostics. An export with known loss fails unless the caller passes the
47
+ relevant diagnostic codes through `allow_loss` or repeats `--allow-loss CODE`
48
+ in the CLI.
49
+
50
+ The concrete pyspssio boundaries are documented in the [SAV profile](sav-profile.md).
51
+ The pinned OpenStatSpec fork preserves document text, file labels, legacy
52
+ compatible names, separate print/write formats, variable sets,
53
+ multiple-response sets, alignment, scalar attributes, and ordered
54
+ custom-attribute arrays. Non-UTF-8 output requires an explicit matching locale;
55
+ otherwise the strict export policy reports and blocks the loss. A caller
56
+ that supplies consent for an available loss code receives the
57
+ machine-readable loss report with the export result.
58
+
59
+ ## Transformation release gates
60
+
61
+ The canonical transformation core and SPSS syntax frontend are separate public
62
+ surfaces. A release must run the specification-owned canonical-plan and SPSS
63
+ frontend conformance fixtures, then exercise both the generic plan apply API
64
+ and the SPSS compatibility apply path.
65
+
66
+ The gate must prove that:
67
+
68
+ - a TransformationPlan object and its strict JSON mapping produce the same
69
+ plan hash and in-place result;
70
+ - top-level SPSS compiler imports and legacy openstatspec.transform re-exports
71
+ still load from an installed wheel;
72
+ - install-in-place-schema, apply-plan, and apply-spss execute their documented
73
+ CLI workflows;
74
+ - invalid or unsupported plans fail before the first data or metadata mutation;
75
+ - successful applies retain the same dataset ID, physical schema/table
76
+ identity, dataset count, and persistent physical data-table count;
77
+ - audit rows distinguish canonical plans from SPSS syntax, preserve correct
78
+ source/plan hashes and frontend contract, and contain no copied data;
79
+ - no OpenStatSpec rollback, snapshot, staging, copy, derived-dataset, or
80
+ parallel history artifacts are created; and
81
+ - Dolt checks expected branch, HEAD, and a clean working set without committing
82
+ or changing HEAD; other supported SQL connections remain allowed.
83
+
84
+ The built wheel must contain the generic openstatspec.transform modules and the
85
+ implemented openstatspec.frontends.spss package. Stata and SAS remain empty
86
+ source-tree placeholders and must expose no compiler, apply API, CLI choice,
87
+ capability claim, or implied support.
88
+
89
+ ## Maintainer checks before tagging
90
+
91
+ 1. Publish the pinned `openstatspec-pyspssio==0.5.1.post2` engine distribution
92
+ first and confirm that a clean environment can download it from PyPI. The
93
+ main package has no fallback SPSS engine.
94
+ 2. Run `python -m pytest -m "not services"`.
95
+ 3. Confirm the GitHub Actions matrix is green for exact PostgreSQL 17.10/18.4,
96
+ MySQL 8.4.11/9.7.2, MariaDB 11.4.12/11.8.8/12.3.2, and exact Dolt
97
+ 2.2.2/2.2.3 service evidence from the immutable image pins in CI.
98
+ 4. Build with `python -m build` and install the generated wheel in a clean
99
+ environment.
100
+ 5. Confirm `openstatspec capabilities` reflects the intended support boundary.
101
+ 6. Confirm the release tag matches the package version and that CI, release
102
+ fixtures, and capabilities use OpenStatSpec specification release `v0.2.0`
103
+ at exact commit `79339ec3d8f8aa81789b7e85f6b8afa6f1374e50`.
104
+ 7. Review this document, the README, and CHANGELOG for accurate scope.
105
+
106
+ The tag-triggered release workflow repeats the non-service test suite, builds
107
+ the distributions, and installs the wheel with the exact required SPSS engine
108
+ in a clean environment before it can reach the protected `pypi` environment.
109
+ It fails closed if that exact engine version is not already downloadable from
110
+ PyPI, preventing publication of an uninstallable OpenStatSpec release.
@@ -15,12 +15,27 @@ unencrypted .sav and .zsav files, and writes both formats through the same imple
15
15
 
16
16
  SQLite has a local reference fixture. PostgreSQL, MySQL, MariaDB, and Dolt have
17
17
  separate service-backed conformance checks in GitHub Actions that import,
18
- validate, and export the supported fixture. Dolt is an independent core profile
19
- pinned to exact server version 2.2.2 and detected over `mysql+pymysql` by active
20
- server identity; other Dolt versions and unknown MySQL-wire products fail closed.
21
- The core Dolt profile does not claim support for the separate Transformation
22
- Workflow. An import that exceeds a target's strict single-table column or row
23
- envelope fails before it creates a dataset.
18
+ validate, and export the supported fixture. The family claims and exact CI
19
+ points are PostgreSQL 17.x/18.x at 17.10/18.4, MySQL 8.4.x/9.7.x at
20
+ 8.4.11/9.7.2, and MariaDB 11.4.x/11.8.x/12.3.x at
21
+ 11.4.12/11.8.8/12.3.2. CI compares each live normalized server version with
22
+ that exact evidence point.
23
+
24
+ Dolt is an independent core profile for canonical stable versions in
25
+ `>=2.2.2,<2.3.0`, detected over `mysql+pymysql` by active server identity and
26
+ service-tested at exact versions 2.2.2 and 2.2.3. Earlier patches, other
27
+ families, noncanonical versions, and unknown MySQL-wire products fail closed.
28
+ The core Dolt profile does not
29
+ claim support for the separate Transformation Workflow. An import that exceeds
30
+ a target's strict single-table column or row envelope fails before it creates a
31
+ dataset.
32
+
33
+ The core SQLite profile remains `>=3.24.0,<4.0.0`; only the optional
34
+ Transformation Workflow requires `>=3.35.0,<4.0.0`. Microsoft SQL Server is
35
+ not a supported profile. Its driver, SQL-difference, preflight, CI,
36
+ conformance, security, and atomicity work is future scope in the
37
+ specification's
38
+ [MSSQL dialect roadmap](https://github.com/OpenStatSpec/specification/blob/main/docs/mssql-dialect-roadmap.md).
24
39
 
25
40
  ## Fidelity contract
26
41