continuo-python-runtime 0.1.0__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (119) hide show
  1. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/.dockerignore +2 -1
  2. continuo_python_runtime-0.3.0/.github/workflows/ci.yml +61 -0
  3. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/.github/workflows/images.yml +44 -13
  4. continuo_python_runtime-0.3.0/.github/workflows/publish-pypi.yml +74 -0
  5. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/.gitignore +2 -0
  6. continuo_python_runtime-0.3.0/Dockerfile.postgres +22 -0
  7. continuo_python_runtime-0.3.0/Dockerfile.trino +22 -0
  8. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/PKG-INFO +108 -43
  9. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/README.md +103 -40
  10. continuo_python_runtime-0.3.0/adapters/postgres/README.md +36 -0
  11. continuo_python_runtime-0.3.0/adapters/postgres/continuo_python_runtime_postgres/adapter.py +700 -0
  12. {continuo_python_runtime-0.1.0/python-runtime-postgres → continuo_python_runtime-0.3.0/adapters/postgres}/pyproject.toml +13 -5
  13. continuo_python_runtime-0.3.0/adapters/postgres/tests/test_adapter_postgres_validation.py +199 -0
  14. continuo_python_runtime-0.3.0/adapters/postgres/tests/test_adapter_runtime_postgres.py +551 -0
  15. continuo_python_runtime-0.3.0/adapters/postgres/tests/test_integration_postgres_validation.py +450 -0
  16. {continuo_python_runtime-0.1.0/python-runtime-postgres → continuo_python_runtime-0.3.0/adapters/postgres}/tests/test_integration_runtime_postgres.py +34 -6
  17. {continuo_python_runtime-0.1.0/python-runtime-trino → continuo_python_runtime-0.3.0/adapters/trino}/README.md +15 -6
  18. {continuo_python_runtime-0.1.0/python-runtime-trino → continuo_python_runtime-0.3.0/adapters/trino}/continuo_python_runtime_trino/adapter.py +278 -56
  19. {continuo_python_runtime-0.1.0/python-runtime-trino → continuo_python_runtime-0.3.0/adapters/trino}/pyproject.toml +13 -5
  20. continuo_python_runtime-0.3.0/adapters/trino/tests/test_adapter_runtime_trino.py +501 -0
  21. continuo_python_runtime-0.3.0/adapters/trino/tests/test_adapter_trino_validation.py +120 -0
  22. {continuo_python_runtime-0.1.0/python-runtime-trino → continuo_python_runtime-0.3.0/adapters/trino}/tests/test_integration_runtime_trino.py +18 -14
  23. continuo_python_runtime-0.3.0/adapters/trino/tests/test_integration_trino_validation.py +469 -0
  24. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/cli.py +38 -9
  25. continuo_python_runtime-0.3.0/continuo_python_runtime/closure.py +268 -0
  26. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/context.py +2 -2
  27. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/contract/loader.py +155 -129
  28. continuo_python_runtime-0.3.0/continuo_python_runtime/contract/merge.py +157 -0
  29. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/contract/model.py +3 -1
  30. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/harness.py +97 -10
  31. continuo_python_runtime-0.3.0/continuo_python_runtime/hashing.py +89 -0
  32. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/lint.py +40 -26
  33. continuo_python_runtime-0.3.0/continuo_python_runtime/types.py +144 -0
  34. continuo_python_runtime-0.3.0/continuo_python_runtime/validation/__init__.py +1 -0
  35. continuo_python_runtime-0.3.0/continuo_python_runtime/validation/runner.py +245 -0
  36. continuo_python_runtime-0.3.0/continuo_python_runtime/validation/s3.py +45 -0
  37. continuo_python_runtime-0.3.0/contract/README.md +51 -0
  38. continuo_python_runtime-0.3.0/contract/continuo_engine_contract/__init__.py +1 -0
  39. continuo_python_runtime-0.3.0/contract/continuo_engine_contract/config.py +50 -0
  40. continuo_python_runtime-0.3.0/contract/continuo_engine_contract/port.py +189 -0
  41. continuo_python_runtime-0.3.0/contract/continuo_engine_contract/result.py +34 -0
  42. continuo_python_runtime-0.3.0/contract/continuo_engine_contract/sql.py +86 -0
  43. continuo_python_runtime-0.3.0/contract/continuo_engine_contract/types.py +23 -0
  44. continuo_python_runtime-0.3.0/contract/pyproject.toml +19 -0
  45. continuo_python_runtime-0.3.0/contract/tests/test_config.py +55 -0
  46. continuo_python_runtime-0.3.0/contract/tests/test_contract_types.py +25 -0
  47. continuo_python_runtime-0.3.0/contract/tests/test_port.py +349 -0
  48. continuo_python_runtime-0.3.0/contract/tests/test_result.py +40 -0
  49. continuo_python_runtime-0.3.0/contract/tests/test_sql.py +103 -0
  50. continuo_python_runtime-0.3.0/docs/boundary-contract.md +381 -0
  51. continuo_python_runtime-0.3.0/docs/superpowers/plans/2026-08-07-step-3b-config-three-part-hash.md +852 -0
  52. continuo_python_runtime-0.3.0/pyproject.toml +85 -0
  53. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/template/.github/workflows/release.yml +8 -0
  54. continuo_python_runtime-0.3.0/template/Dockerfile +16 -0
  55. continuo_python_runtime-0.3.0/template/contracts/example.yml +38 -0
  56. continuo_python_runtime-0.3.0/template/scripts/example.py +17 -0
  57. continuo_python_runtime-0.3.0/tests/conftest.py +143 -0
  58. continuo_python_runtime-0.3.0/tests/contract/__init__.py +0 -0
  59. continuo_python_runtime-0.3.0/tests/contract/test_loader.py +625 -0
  60. continuo_python_runtime-0.3.0/tests/contract/test_merge.py +430 -0
  61. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/contract/test_model.py +6 -0
  62. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/smoke/postgres-stack/docker-compose.yml +1 -1
  63. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/smoke/trino-stack/docker-compose.yml +4 -4
  64. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/test_cli.py +141 -3
  65. continuo_python_runtime-0.3.0/tests/test_closure.py +448 -0
  66. continuo_python_runtime-0.3.0/tests/test_contract_pin_consistency.py +49 -0
  67. continuo_python_runtime-0.3.0/tests/test_harness.py +614 -0
  68. continuo_python_runtime-0.3.0/tests/test_hashing.py +188 -0
  69. continuo_python_runtime-0.3.0/tests/test_image_smoke_validation.py +69 -0
  70. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/test_lint.py +80 -0
  71. continuo_python_runtime-0.3.0/tests/test_no_legacy_names.py +46 -0
  72. continuo_python_runtime-0.3.0/tests/test_template.py +78 -0
  73. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/test_types.py +54 -0
  74. continuo_python_runtime-0.3.0/tests/test_validation_runner.py +880 -0
  75. continuo_python_runtime-0.3.0/tests/test_validation_s3.py +41 -0
  76. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/uv.lock +85 -20
  77. continuo_python_runtime-0.1.0/.github/workflows/ci.yml +0 -25
  78. continuo_python_runtime-0.1.0/.github/workflows/publish-pypi.yml +0 -129
  79. continuo_python_runtime-0.1.0/Dockerfile.postgres +0 -13
  80. continuo_python_runtime-0.1.0/Dockerfile.trino +0 -13
  81. continuo_python_runtime-0.1.0/continuo_python_runtime/contract/merge.py +0 -80
  82. continuo_python_runtime-0.1.0/continuo_python_runtime/hashing.py +0 -34
  83. continuo_python_runtime-0.1.0/continuo_python_runtime/types.py +0 -208
  84. continuo_python_runtime-0.1.0/docs/boundary-contract.md +0 -152
  85. continuo_python_runtime-0.1.0/pyproject.toml +0 -58
  86. continuo_python_runtime-0.1.0/python-runtime-postgres/README.md +0 -18
  87. continuo_python_runtime-0.1.0/python-runtime-postgres/continuo_python_runtime_postgres/adapter.py +0 -229
  88. continuo_python_runtime-0.1.0/python-runtime-postgres/tests/test_adapter_runtime_postgres.py +0 -115
  89. continuo_python_runtime-0.1.0/python-runtime-trino/tests/test_adapter_runtime_trino.py +0 -210
  90. continuo_python_runtime-0.1.0/template/Dockerfile +0 -8
  91. continuo_python_runtime-0.1.0/template/contracts/example.yml +0 -12
  92. continuo_python_runtime-0.1.0/template/scripts/example.py +0 -3
  93. continuo_python_runtime-0.1.0/tests/conftest.py +0 -81
  94. continuo_python_runtime-0.1.0/tests/contract/test_loader.py +0 -344
  95. continuo_python_runtime-0.1.0/tests/contract/test_merge.py +0 -118
  96. continuo_python_runtime-0.1.0/tests/test_harness.py +0 -261
  97. continuo_python_runtime-0.1.0/tests/test_hashing.py +0 -53
  98. continuo_python_runtime-0.1.0/tests/test_template.py +0 -50
  99. {continuo_python_runtime-0.1.0/python-runtime-postgres → continuo_python_runtime-0.3.0/adapters/postgres}/continuo_python_runtime_postgres/__init__.py +0 -0
  100. {continuo_python_runtime-0.1.0/python-runtime-postgres → continuo_python_runtime-0.3.0/adapters/postgres}/tests/__init__.py +0 -0
  101. {continuo_python_runtime-0.1.0/python-runtime-trino → continuo_python_runtime-0.3.0/adapters/trino}/continuo_python_runtime_trino/__init__.py +0 -0
  102. {continuo_python_runtime-0.1.0/python-runtime-trino → continuo_python_runtime-0.3.0/adapters/trino}/tests/__init__.py +0 -0
  103. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/__init__.py +0 -0
  104. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/conform.py +0 -0
  105. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/contract/__init__.py +0 -0
  106. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/contract/paths.py +0 -0
  107. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/continuo_python_runtime/errors.py +0 -0
  108. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0/contract}/tests/__init__.py +0 -0
  109. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/docs/superpowers/plans/2026-07-31-python-runtime.md +0 -0
  110. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/docs/superpowers/specs/2026-07-31-python-runtime-design.md +0 -0
  111. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/template/README.md +0 -0
  112. {continuo_python_runtime-0.1.0/tests/contract → continuo_python_runtime-0.3.0/tests}/__init__.py +0 -0
  113. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/smoke/node_smoke/contracts/smoke.yml +0 -0
  114. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/smoke/node_smoke/scripts/smoke.py +0 -0
  115. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/smoke/trino-stack/catalog/iceberg.properties +0 -0
  116. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/test_conform.py +0 -0
  117. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/test_context.py +0 -0
  118. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/test_errors.py +0 -0
  119. {continuo_python_runtime-0.1.0 → continuo_python_runtime-0.3.0}/tests/test_package.py +0 -0
@@ -11,4 +11,5 @@ dist/
11
11
  docs/
12
12
  template/
13
13
  tests/
14
- python-runtime-*/tests/
14
+ adapters/*/tests/
15
+ contract/tests/
@@ -0,0 +1,61 @@
1
+ name: ci
2
+ on:
3
+ push: { branches: [main] }
4
+ pull_request:
5
+ jobs:
6
+ test:
7
+ runs-on: ubuntu-latest
8
+ steps:
9
+ - uses: actions/checkout@v4
10
+ - uses: astral-sh/setup-uv@v5
11
+ - name: Sync
12
+ run: uv sync --all-packages --all-groups
13
+ - name: Lint
14
+ run: uv run ruff check .
15
+ - name: Lint (contract)
16
+ run: uv run ruff check contract
17
+ - name: Types (runtime)
18
+ run: uv run mypy continuo_python_runtime
19
+ - name: Types (contract)
20
+ run: uv run mypy contract/continuo_engine_contract
21
+ - name: Types (postgres adapter)
22
+ run: uv run --package continuo-python-runtime-postgres mypy adapters/postgres/continuo_python_runtime_postgres
23
+ - name: Types (trino adapter)
24
+ run: uv run --package continuo-python-runtime-trino mypy adapters/trino/continuo_python_runtime_trino
25
+ # `-m "not image"` deselects tests/test_image_smoke_validation.py, which
26
+ # needs a built engine image and the env naming it. Those tests run in
27
+ # images.yml's smoke jobs, where an image actually exists.
28
+ - name: Tests (runtime)
29
+ run: uv run pytest --cov=continuo_python_runtime -m "not image" -v
30
+ - name: Tests (contract)
31
+ run: uv run pytest contract/tests -v
32
+ - name: Tests (adapter units)
33
+ run: uv run pytest adapters/postgres/tests adapters/trino/tests -m "not integration" -v
34
+
35
+ integration-postgres:
36
+ runs-on: ubuntu-latest
37
+ steps:
38
+ - uses: actions/checkout@v4
39
+ - uses: astral-sh/setup-uv@v5
40
+ - run: uv sync --all-packages --all-groups
41
+ - name: Start postgres stack
42
+ run: docker compose -f tests/smoke/postgres-stack/docker-compose.yml up -d --wait
43
+ - name: Tests (postgres integration)
44
+ run: uv run pytest adapters/postgres/tests -m integration -v
45
+ - name: Tear down postgres stack
46
+ if: always()
47
+ run: docker compose -f tests/smoke/postgres-stack/docker-compose.yml down -v
48
+
49
+ integration-trino:
50
+ runs-on: ubuntu-latest
51
+ steps:
52
+ - uses: actions/checkout@v4
53
+ - uses: astral-sh/setup-uv@v5
54
+ - run: uv sync --all-packages --all-groups
55
+ - name: Start trino stack
56
+ run: docker compose -f tests/smoke/trino-stack/docker-compose.yml up -d --wait
57
+ - name: Tests (trino integration)
58
+ run: uv run pytest adapters/trino/tests -m integration -v
59
+ - name: Tear down trino stack
60
+ if: always()
61
+ run: docker compose -f tests/smoke/trino-stack/docker-compose.yml down -v
@@ -5,8 +5,8 @@ on:
5
5
  - "Dockerfile.*"
6
6
  - "tests/smoke/**"
7
7
  - "continuo_python_runtime/**"
8
- - "python-runtime-postgres/**"
9
- - "python-runtime-trino/**"
8
+ - "contract/**"
9
+ - "adapters/**"
10
10
  - "pyproject.toml"
11
11
  - ".github/workflows/images.yml"
12
12
  push:
@@ -56,6 +56,16 @@ jobs:
56
56
  path: /tmp
57
57
  - name: Load image
58
58
  run: docker load -i /tmp/cpr-smoke-postgres.tar
59
+ - uses: astral-sh/setup-uv@v5
60
+ - name: Sync
61
+ run: uv sync --all-packages --all-groups
62
+ - name: Image smoke tests (postgres)
63
+ env:
64
+ VALIDATION_IMAGE_UNDER_TEST: cpr-smoke-postgres
65
+ VALIDATION_IMAGE_ENGINE: postgres
66
+ # PostgresAdapter.required_env()[0]
67
+ VALIDATION_IMAGE_REQUIRED_ENV: POSTGRES_HOST
68
+ run: uv run pytest tests/test_image_smoke_validation.py -m image -v
59
69
  - name: Install psql client
60
70
  run: |
61
71
  sudo apt-get update
@@ -101,6 +111,16 @@ jobs:
101
111
  path: /tmp
102
112
  - name: Load image
103
113
  run: docker load -i /tmp/cpr-smoke-trino.tar
114
+ - uses: astral-sh/setup-uv@v5
115
+ - name: Sync
116
+ run: uv sync --all-packages --all-groups
117
+ - name: Image smoke tests (trino)
118
+ env:
119
+ VALIDATION_IMAGE_UNDER_TEST: cpr-smoke-trino
120
+ VALIDATION_IMAGE_ENGINE: trino
121
+ # TrinoAdapter.required_env()[0]
122
+ VALIDATION_IMAGE_REQUIRED_ENV: TRINO_HOST
123
+ run: uv run pytest tests/test_image_smoke_validation.py -m image -v
104
124
  - name: Start trino stack
105
125
  run: docker compose -f tests/smoke/trino-stack/docker-compose.yml up -d --wait
106
126
  - name: Seed source table
@@ -133,13 +153,20 @@ jobs:
133
153
  if: always()
134
154
  run: docker compose -f tests/smoke/trino-stack/docker-compose.yml down -v
135
155
 
136
- # On tag push: log in to ghcr.io and push ghcr.io/${{ github.repository }}:<tag>-postgres
137
- # and :<tag>-trino. continuo-validation-contract, continuo-python-runtime-postgres,
138
- # and continuo-python-runtime-trino are all installed from the build context /
139
- # PyPI already (see Dockerfile.postgres / Dockerfile.trino and
140
- # publish-pypi.yml) — nothing pending here.
156
+ # On tag push: build both engine images for linux/amd64 and linux/arm64 and
157
+ # push them to ghcr.io/<owner>/continuo-python-runtime-<engine>:<tag>. The
158
+ # engine belongs in the image NAME, not the tag: Continuo's Helm chart pins an
159
+ # image as `<name>:vX.Y.Z@sha256:<digest>`, so the tag must be the bare
160
+ # version. Both platforms are required: domain repos build their own images
161
+ # FROM these, and Continuo's e2e runs them in a kind cluster on aarch64
162
+ # developer machines as well as on amd64 CI runners, so a single-arch publish
163
+ # is unusable off amd64. The smoke jobs above stay single-arch — they load a
164
+ # `docker save` tarball, which has no multi-platform equivalent.
165
+ # continuo-engine-contract and the two engine adapters are installed from the
166
+ # build context (see Dockerfile.postgres / Dockerfile.trino) — nothing pending
167
+ # here.
141
168
  publish:
142
- if: startsWith(github.ref, 'refs/tags/v')
169
+ if: startsWith(github.ref, 'refs/tags/v') && !contains(github.ref_name, '-test')
143
170
  needs: [smoke-postgres, smoke-trino]
144
171
  strategy:
145
172
  matrix:
@@ -150,6 +177,8 @@ jobs:
150
177
  packages: write
151
178
  steps:
152
179
  - uses: actions/checkout@v4
180
+ - uses: docker/setup-qemu-action@v3
181
+ - uses: docker/setup-buildx-action@v3
153
182
  - name: Log in to ghcr.io
154
183
  uses: docker/login-action@v3
155
184
  with:
@@ -157,8 +186,10 @@ jobs:
157
186
  username: ${{ github.actor }}
158
187
  password: ${{ secrets.GITHUB_TOKEN }}
159
188
  - name: Build and push ${{ matrix.engine }} image
160
- run: |
161
- tag="${GITHUB_REF_NAME}"
162
- image="ghcr.io/${{ github.repository }}:${tag}-${{ matrix.engine }}"
163
- docker build -f Dockerfile.${{ matrix.engine }} -t "$image" .
164
- docker push "$image"
189
+ uses: docker/build-push-action@v6
190
+ with:
191
+ context: .
192
+ file: Dockerfile.${{ matrix.engine }}
193
+ platforms: linux/amd64,linux/arm64
194
+ push: true
195
+ tags: ghcr.io/${{ github.repository_owner }}/continuo-python-runtime-${{ matrix.engine }}:${{ github.ref_name }}
@@ -0,0 +1,74 @@
1
+ name: publish-pypi
2
+
3
+ # Publishes both PyPI distributions this repo owns — continuo-python-runtime
4
+ # (the harness) and continuo-engine-contract (the port, result-block format,
5
+ # and shared guards) — via PyPI Trusted Publishing (OIDC, no stored token).
6
+ #
7
+ # The two engine adapters (continuo-python-runtime-postgres /
8
+ # continuo-python-runtime-trino) are deliberately NOT published here. Nothing
9
+ # installs them from an index: the engine images build them from the build
10
+ # context (see Dockerfile.postgres / Dockerfile.trino and images.yml), and the
11
+ # harness package does not depend on them — adapters are found at run time
12
+ # through the `continuo_engine.adapters` entry-point group, inside the image
13
+ # that installed exactly one. Publishing them would only add releases with no
14
+ # consumer. They remain uv workspace members and are still built, typed, and
15
+ # tested by ci.yml.
16
+ #
17
+ # Tag glob note: `v*` is the repository's single release pattern — the same tag
18
+ # that images.yml builds and pushes both engine images from, so one tag ships
19
+ # the whole release. GitHub Actions tag globs anchor at character 1, so `v*`
20
+ # claims every tag beginning with "v"; no other pattern may be introduced.
21
+ # Tag `v<ver>-test<n>` publishes to TestPyPI; `v<ver>` publishes to real PyPI.
22
+ # The GitHub environment name is what the PyPI "pending publisher" is
23
+ # registered against — both project names need one on each index.
24
+ #
25
+ # Both distributions are built into a single `dist/` and uploaded in one
26
+ # publish call, so there is no ordering constraint between them. The runtime
27
+ # wheel declares continuo-engine-contract as a dependency and resolves it from
28
+ # the index at install time ([tool.uv.sources] is dev-only and is not embedded
29
+ # in the wheel).
30
+ on:
31
+ push:
32
+ tags:
33
+ - "v*"
34
+
35
+ permissions: {}
36
+
37
+ jobs:
38
+ publish-packages:
39
+ if: startsWith(github.ref_name, 'v')
40
+ runs-on: ubuntu-latest
41
+ environment: ${{ contains(github.ref_name, '-test') && 'testpypi' || 'pypi' }}
42
+ permissions:
43
+ id-token: write # OIDC token for Trusted Publishing
44
+ contents: read # actions/checkout needs read access to the repo
45
+ steps:
46
+ - uses: actions/checkout@v4
47
+ - uses: astral-sh/setup-uv@v5
48
+ - name: Test before publishing
49
+ # `-m "not image"` deselects the image smoke tests; images.yml runs
50
+ # those against a built engine image.
51
+ run: |
52
+ uv sync --all-packages --all-groups
53
+ uv run pytest tests contract/tests -m "not image" -q
54
+ - name: Build the contract sdist + wheel
55
+ run: uv build --package continuo-engine-contract -o dist
56
+ - name: Build the runtime sdist + wheel
57
+ run: uv build -o dist
58
+ - name: Publish to TestPyPI
59
+ if: contains(github.ref_name, '-test')
60
+ # Pinned to a commit SHA. release/v1 is a moving branch, and this job
61
+ # holds id-token: write for PyPI Trusted Publishing — a compromised
62
+ # revision could mint a token and publish under this project's name.
63
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
64
+ with:
65
+ repository-url: https://test.pypi.org/legacy/
66
+ packages-dir: dist
67
+ - name: Publish to PyPI
68
+ if: ${{ !contains(github.ref_name, '-test') }}
69
+ # Pinned to a commit SHA. release/v1 is a moving branch, and this job
70
+ # holds id-token: write for PyPI Trusted Publishing — a compromised
71
+ # revision could mint a token and publish under this project's name.
72
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
73
+ with:
74
+ packages-dir: dist
@@ -7,3 +7,5 @@ dist/
7
7
  .pytest_cache/
8
8
  .mypy_cache/
9
9
  .ruff_cache/
10
+ .coverage
11
+ .worktrees/
@@ -0,0 +1,22 @@
1
+ FROM python:3.14-slim
2
+ # One dual-role image per engine: the python-node runtime harness (default
3
+ # command `run`, domain repos build FROM this) and the blue/green validation
4
+ # runner (`validation-op`, continuo's executor sets it as the container
5
+ # command). Everything installs from this repo's own workspace paths in one
6
+ # pip transaction — no PyPI propagation lag; images always build from repo head.
7
+ # docker build -f Dockerfile.postgres .
8
+ COPY pyproject.toml README.md /src/
9
+ COPY contract /src/contract
10
+ COPY continuo_python_runtime /src/continuo_python_runtime
11
+ COPY adapters/postgres /src/adapters/postgres
12
+ RUN pip install --no-cache-dir /src/contract /src/adapters/postgres /src \
13
+ && rm -rf /src
14
+ # PYTHONPATH is belt-and-braces: the harness also inserts APP_ROOT and the
15
+ # node script's own directory at the front of sys.path before executing it.
16
+ ENV CONTRACT_DIR=/app/contracts APP_ROOT=/app PYTHONPATH=/app
17
+ WORKDIR /app
18
+ # uid 65532 matches continuo's executor securityContext expectation.
19
+ RUN useradd --uid 65532 --no-create-home --shell /usr/sbin/nologin nonroot
20
+ USER 65532:65532
21
+ ENTRYPOINT ["continuo-runtime"]
22
+ CMD ["run"]
@@ -0,0 +1,22 @@
1
+ FROM python:3.14-slim
2
+ # One dual-role image per engine: the python-node runtime harness (default
3
+ # command `run`, domain repos build FROM this) and the blue/green validation
4
+ # runner (`validation-op`, continuo's executor sets it as the container
5
+ # command). Everything installs from this repo's own workspace paths in one
6
+ # pip transaction — no PyPI propagation lag; images always build from repo head.
7
+ # docker build -f Dockerfile.trino .
8
+ COPY pyproject.toml README.md /src/
9
+ COPY contract /src/contract
10
+ COPY continuo_python_runtime /src/continuo_python_runtime
11
+ COPY adapters/trino /src/adapters/trino
12
+ RUN pip install --no-cache-dir /src/contract /src/adapters/trino /src \
13
+ && rm -rf /src
14
+ # PYTHONPATH is belt-and-braces: the harness also inserts APP_ROOT and the
15
+ # node script's own directory at the front of sys.path before executing it.
16
+ ENV CONTRACT_DIR=/app/contracts APP_ROOT=/app PYTHONPATH=/app
17
+ WORKDIR /app
18
+ # uid 65532 matches continuo's executor securityContext expectation.
19
+ RUN useradd --uid 65532 --no-create-home --shell /usr/sbin/nologin nonroot
20
+ USER 65532:65532
21
+ ENTRYPOINT ["continuo-runtime"]
22
+ CMD ["run"]
@@ -1,6 +1,6 @@
1
- Metadata-Version: 2.4
1
+ Metadata-Version: 2.5
2
2
  Name: continuo-python-runtime
3
- Version: 0.1.0
3
+ Version: 0.3.0
4
4
  Summary: Runtime harness, contract tooling, and CI lint for Continuo python nodes.
5
5
  Author: Simone Carolini
6
6
  Maintainer: Simone Carolini
@@ -8,9 +8,11 @@ Classifier: Development Status :: 4 - Beta
8
8
  Classifier: Intended Audience :: Developers
9
9
  Classifier: Programming Language :: Python :: 3.14
10
10
  Requires-Python: >=3.14
11
- Requires-Dist: continuo-validation-contract==0.3.0
11
+ Requires-Dist: boto3==1.43.59
12
+ Requires-Dist: continuo-engine-contract==0.7.0
12
13
  Requires-Dist: pyarrow==25.0.0
13
14
  Requires-Dist: pyyaml==6.0.3
15
+ Requires-Dist: sqlglot==30.15.0
14
16
  Description-Content-Type: text/markdown
15
17
 
16
18
  # Continuo Python Runtime
@@ -23,44 +25,69 @@ and register the release with Continuo.
23
25
 
24
26
  ## What this repo is
25
27
 
26
- Three artifacts come out of this repository:
28
+ Four artifacts come out of this repository:
27
29
 
28
30
  - **The `continuo-python-runtime` PyPI package** — the `continuo-runtime` CLI
29
- (`validate` / `merge` / `hash` / `lint` / `run`) and the harness library
30
- (`conform()`, `RunContext`, the error taxonomy) that domain repos install.
31
- - **Per-engine base images**, one per warehouse engine (`...-postgres`,
32
- `...-trino`), that domain repos build `FROM`. Each image bakes in the
33
- runtime, a single `RuntimeAdapter` for that engine, and the
34
- `continuo-runtime run` entrypoint.
31
+ (`validate` / `merge` / `hash` / `lint` / `run` / `validation-op`) and the
32
+ harness library (`conform()`, `RunContext`, the error taxonomy) that domain
33
+ repos install.
34
+ - **The `continuo-engine-contract` PyPI package** — the `WarehouseAdapter`
35
+ port, the contract schema, the shared SQL/type/config guards, and the
36
+ sentinel result-block format. Adapter authors outside this repo pin it.
37
+ - **Per-engine base images**, one per warehouse engine
38
+ (`continuo-python-runtime-postgres`, `continuo-python-runtime-trino`), that
39
+ domain repos build `FROM`. Each image bakes in the runtime and a single
40
+ `WarehouseAdapter` for that engine, and serves both roles that adapter has:
41
+ the node harness (`ENTRYPOINT ["continuo-runtime"]`, `CMD ["run"]`) and the
42
+ validation runner (`continuo-runtime validation-op`).
35
43
  - **`template/`** — a copy-ready domain repo: `Dockerfile`, `contracts/`,
36
44
  `scripts/`, and the `release.yml` CI/CD workflow.
37
45
 
38
- Base images and the PyPI publication of this package land with this repo's
39
- PR 9 (the image pipeline); until then, install the runtime from git as noted
40
- in `template/.github/workflows/release.yml`.
46
+ One `vX.Y.Z` git tag releases all of it: `publish-pypi.yml` builds both
47
+ distributions into a single `dist/` and publishes them together, and
48
+ `images.yml` builds and pushes both engine images multi-arch under the same
49
+ tag.
41
50
 
42
- ### Package map
51
+ ### What this repo owns
43
52
 
44
- Each per-engine base image bakes together three PyPI packages: the harness,
45
- the engine adapter, and the published contract. The harness and both engine
46
- adapters live in *this* repo (`continuo-python-runtime`) as a uv workspace;
47
- `continuo-validation-runners` owns the *validation-side* (offline lint/merge)
48
- counterparts of the same engines, which is a separate concern from the
49
- data-plane adapters here.
53
+ This repository owns the entire python-node surface: the engine contract, both
54
+ engine adapters, the validation runner, and the node harness. The former
55
+ `continuo-validation` repository was merged in — there is no longer a separate
56
+ validation-side port, adapter class, entry-point group, or image. One
57
+ `WarehouseAdapter` per engine serves both the data plane (`fetch` /
58
+ `ensure_table` / `load`) and validation (`ensure_schema` / `drop_schema` /
59
+ `build_empty_from_sql` / `build_empty_from_columns` / `clone_empty_from_prod` /
60
+ `check_binds`), and one image per engine runs both roles.
50
61
 
51
62
  | Package (distribution name) | Module | Lives in | Role |
52
63
  | --- | --- | --- | --- |
53
- | `continuo-python-runtime` | `continuo_python_runtime` | this repo (root) | Harness: CLI, `conform()`, `RunContext`, error taxonomy. |
54
- | `continuo-python-runtime-postgres` | `continuo_python_runtime_postgres` | this repo, `python-runtime-postgres/` | Data-plane `RuntimeAdapter` for Postgres (`fetch`/`ensure_table`/`load`). |
55
- | `continuo-python-runtime-trino` | `continuo_python_runtime_trino` | this repo, `python-runtime-trino/` | Data-plane `RuntimeAdapter` for Trino/Iceberg. |
56
- | `continuo-validation-contract` | `continuo_validation_contract` | `continuo-validation-runners` | The published contract (schema, `RuntimeAdapter` port, result-block format) both sides depend on. |
57
- | `continuo-validation-postgres` / `continuo-validation-trino` | `continuo_validation_postgres` / `continuo_validation_trino` | `continuo-validation-runners` | Validation-side (lint/merge, no live warehouse I/O) adapters — not to be confused with the data-plane adapters above. |
58
-
59
- All three packages built in this repo resolve `continuo-validation-contract`
60
- from PyPI (`==0.3.0`); the two adapter packages are uv workspace members
61
- (`[tool.uv.workspace]` in the root `pyproject.toml`), so `uv sync
62
- --all-packages --all-groups` at the repo root installs everything for local
63
- development.
64
+ | `continuo-python-runtime` | `continuo_python_runtime` | this repo (root) | Harness (CLI, `conform()`, `RunContext`, error taxonomy) **and** the validation runner (`continuo-runtime validation-op`). Published to PyPI. |
65
+ | `continuo-engine-contract` | `continuo_engine_contract` | this repo, `contract/` | The `WarehouseAdapter` port, contract schema, the SQL/type/config guards adapters must run, and the result-block format. Published to PyPI. |
66
+ | `continuo-python-runtime-postgres` | `continuo_python_runtime_postgres` | this repo, `adapters/postgres/` | `PostgresAdapter` — one class, both roles. **Not published to PyPI** — built from source into the image. |
67
+ | `continuo-python-runtime-trino` | `continuo_python_runtime_trino` | this repo, `adapters/trino/` | `TrinoAdapter` — one class, both roles, for Trino/Iceberg. **Not published to PyPI** — built from source into the image. |
68
+
69
+ All four are uv workspace members (`[tool.uv.workspace]` in the root
70
+ `pyproject.toml`), so `uv sync --all-packages --all-groups` at the repo root
71
+ installs everything for local development.
72
+
73
+ **Only `continuo-python-runtime` and `continuo-engine-contract` are published
74
+ to PyPI.** The two engine adapters are built **from source into the engine
75
+ images**: `Dockerfile.postgres` and `Dockerfile.trino` install them out of the
76
+ build context, so each image ships exactly one adapter and the runtime
77
+ discovers it through the `continuo_engine.adapters` entry-point group at run
78
+ time. Nothing installs them from an index — the harness package does not
79
+ depend on them, and domain repos get their adapter by building `FROM` a
80
+ published base image. They are still built, type-checked, and tested by CI on
81
+ every change.
82
+
83
+ ### The result block is a frozen wire contract
84
+
85
+ `continuo_engine_contract.result` writes a sentinel-framed JSON block as the
86
+ last line of stdout, and Continuo's Go side parses it byte-for-byte
87
+ (`pkg/validationresult` in the `continuo` repository). That wire format — the
88
+ sentinel markers, the framing, and the field names inside the block — is
89
+ **frozen**. Reuse it; never change it from this side alone. A change here that
90
+ the Go parser has not been taught is a production outage, not a refactor.
64
91
 
65
92
  ## Quickstart for domain teams
66
93
 
@@ -85,6 +112,28 @@ development.
85
112
  and merges the contracts, builds and pushes the image, uploads the merged
86
113
  contract to S3, and POSTs the release.
87
114
 
115
+ ### Upgrading an existing domain repo
116
+
117
+ `validate` / `merge` / `hash` now hand every declared read to a real SQL
118
+ parser (sqlglot, via `continuo_engine_contract.sql.ensure_single_read`)
119
+ instead of scanning it for a leading `SELECT`/`WITH`. Two things follow for a
120
+ repo written before this, on its next release: SQL a driver would accept but
121
+ a parser will not — most commonly a driver-specific bind placeholder like
122
+ psycopg2's `%(name)s`, which `ctx.read(name)` could never have used anyway —
123
+ now fails validation, and engine-specific syntax (postgres `~`, `@>`, …)
124
+ needs `--dialect <engine>`, which a repo should be passing regardless since
125
+ Continuo bind-checks every read in the install's own warehouse dialect. Run
126
+ the pre-flight check once before your next release; it reports every affected
127
+ read at once:
128
+
129
+ ```bash
130
+ continuo-runtime validate contracts/ --dialect postgres # or trino
131
+ ```
132
+
133
+ The runtime image does not re-run this gate, so a read that passes here is
134
+ not re-judged under a different grammar in production. See
135
+ `docs/boundary-contract.md` §13.1.
136
+
88
137
  ## The script API
89
138
 
90
139
  A node script is a Python file with exactly one required entry point:
@@ -106,9 +155,13 @@ def run(ctx):
106
155
  example a polars DataFrame. Returning anything else raises `ScriptError`.
107
156
  - The dataframe library is your choice. No dataframe library is baked into
108
157
  the base image — the package's own runtime dependencies are `pyarrow`,
109
- `PyYAML`, and `continuo-validation-contract`. Add whatever you script
158
+ `PyYAML`, and `continuo-engine-contract`. Add whatever you script
110
159
  against (pandas, polars, …) as a `RUN pip install` line in your own
111
- `Dockerfile`, on top of the base image.
160
+ `Dockerfile`, on top of the base image. The base image runs as non-root
161
+ (uid 65532), which pip cannot install under, so switch to root for the
162
+ install and back afterward — `USER root`, then the `RUN pip install`
163
+ line, then `USER 65532:65532` — since the executor's pod spec expects
164
+ the image to end at uid 65532. See `template/Dockerfile` for the pattern.
112
165
  - Scripts do not import warehouse drivers, write raw SQL literals, or call
113
166
  data-access methods directly — `continuo-runtime lint` rejects those:
114
167
  - forbidden driver imports (`psycopg2`/`sqlalchemy`/`trino`/etc.),
@@ -125,6 +178,17 @@ def run(ctx):
125
178
  are the driver-import and data-access-call rules, together with the fact
126
179
  that `RunContext` only exposes `ctx.read()` — all warehouse access goes
127
180
  through it.
181
+ - Scripts may import shared in-repo helpers. Before executing a script the
182
+ harness puts the repo root (`APP_ROOT`) and the script's own directory on
183
+ `sys.path`, so both `import helpers` (a sibling of the script) and
184
+ `from lib.shared import ...` (anywhere under the repo root) work, including
185
+ from inside `run()`. Every helper a script reaches transitively is folded
186
+ into `shared_code_hash`, so editing one re-fingerprints the node — but the
187
+ hash does not put the file in the image: **`COPY` every directory your
188
+ scripts import from in your `Dockerfile`**, or the release is valid and the
189
+ node dies with `ModuleNotFoundError` on its first run. Because the repo root
190
+ precedes the standard library on `sys.path`, avoid naming a top-level module
191
+ after a stdlib one (`types.py`, `json.py`, `logging.py`, …).
128
192
  - The harness — not the script — performs the write. It calls `conform()`
129
193
  on whatever `run()` returned and issues the only INSERT; the script never
130
194
  writes directly.
@@ -165,19 +229,20 @@ A domain repo picks its warehouse engine by which base image it builds
165
229
  `FROM`:
166
230
 
167
231
  ```dockerfile
168
- FROM ghcr.io/carolsimone/continuo-python-runtime:v0.1.0-postgres
232
+ FROM ghcr.io/carolsimone/continuo-python-runtime-postgres:v0.3.0
169
233
  # or
170
- FROM ghcr.io/carolsimone/continuo-python-runtime:v0.1.0-trino
234
+ FROM ghcr.io/carolsimone/continuo-python-runtime-trino:v0.3.0
171
235
  ```
172
236
 
173
- Each image bakes in exactly one `RuntimeAdapter` for that engine — installed
174
- from this repo's `python-runtime-postgres/` or `python-runtime-trino/`
175
- package (see the package map above; both adapters live in this repo, not
176
- `continuo-validation-runners`) — registered under the
177
- `continuo_runtime.adapters` entry-point group (entry names `postgres` /
178
- `trino`). The harness discovers it via `discover_runtime_adapter()` at run
179
- time, so a single image serves every node in the service. The executor
180
- injects the
237
+ The engine is part of the image **name**; the tag is the bare version, so
238
+ Continuo's Helm chart can pin an image as `<name>:vX.Y.Z@sha256:<digest>`.
239
+
240
+ Each image bakes in exactly one `WarehouseAdapter` for that engine — installed
241
+ from this repo's `adapters/postgres/` or `adapters/trino/` package (see the
242
+ table above) — registered under the `continuo_engine.adapters` entry-point
243
+ group (entry names `postgres` / `trino`). The runtime discovers it via
244
+ `discover_adapter()` at run time, so a single image serves every node in the
245
+ service and the release-time validation Job for it. The executor injects the
181
246
  warehouse connection as environment variables (engine-native, e.g.
182
247
  `POSTGRES_HOST`/`POSTGRES_DB`/`POSTGRES_USER`) plus the node-selection
183
248
  environment (`NODE_ID`, `TABLE_NAME`, `TARGET_SCHEMA`, and optionally