continuo-python-runtime 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (68) hide show
  1. continuo_python_runtime-0.1.0/.dockerignore +14 -0
  2. continuo_python_runtime-0.1.0/.github/workflows/ci.yml +25 -0
  3. continuo_python_runtime-0.1.0/.github/workflows/images.yml +164 -0
  4. continuo_python_runtime-0.1.0/.github/workflows/publish-pypi.yml +129 -0
  5. continuo_python_runtime-0.1.0/.gitignore +9 -0
  6. continuo_python_runtime-0.1.0/Dockerfile.postgres +13 -0
  7. continuo_python_runtime-0.1.0/Dockerfile.trino +13 -0
  8. continuo_python_runtime-0.1.0/PKG-INFO +193 -0
  9. continuo_python_runtime-0.1.0/README.md +178 -0
  10. continuo_python_runtime-0.1.0/continuo_python_runtime/__init__.py +1 -0
  11. continuo_python_runtime-0.1.0/continuo_python_runtime/cli.py +142 -0
  12. continuo_python_runtime-0.1.0/continuo_python_runtime/conform.py +220 -0
  13. continuo_python_runtime-0.1.0/continuo_python_runtime/context.py +62 -0
  14. continuo_python_runtime-0.1.0/continuo_python_runtime/contract/__init__.py +1 -0
  15. continuo_python_runtime-0.1.0/continuo_python_runtime/contract/loader.py +363 -0
  16. continuo_python_runtime-0.1.0/continuo_python_runtime/contract/merge.py +80 -0
  17. continuo_python_runtime-0.1.0/continuo_python_runtime/contract/model.py +39 -0
  18. continuo_python_runtime-0.1.0/continuo_python_runtime/contract/paths.py +38 -0
  19. continuo_python_runtime-0.1.0/continuo_python_runtime/errors.py +36 -0
  20. continuo_python_runtime-0.1.0/continuo_python_runtime/harness.py +208 -0
  21. continuo_python_runtime-0.1.0/continuo_python_runtime/hashing.py +34 -0
  22. continuo_python_runtime-0.1.0/continuo_python_runtime/lint.py +295 -0
  23. continuo_python_runtime-0.1.0/continuo_python_runtime/types.py +208 -0
  24. continuo_python_runtime-0.1.0/docs/boundary-contract.md +152 -0
  25. continuo_python_runtime-0.1.0/docs/superpowers/plans/2026-07-31-python-runtime.md +1307 -0
  26. continuo_python_runtime-0.1.0/docs/superpowers/specs/2026-07-31-python-runtime-design.md +273 -0
  27. continuo_python_runtime-0.1.0/pyproject.toml +58 -0
  28. continuo_python_runtime-0.1.0/python-runtime-postgres/README.md +18 -0
  29. continuo_python_runtime-0.1.0/python-runtime-postgres/continuo_python_runtime_postgres/__init__.py +0 -0
  30. continuo_python_runtime-0.1.0/python-runtime-postgres/continuo_python_runtime_postgres/adapter.py +229 -0
  31. continuo_python_runtime-0.1.0/python-runtime-postgres/pyproject.toml +45 -0
  32. continuo_python_runtime-0.1.0/python-runtime-postgres/tests/__init__.py +0 -0
  33. continuo_python_runtime-0.1.0/python-runtime-postgres/tests/test_adapter_runtime_postgres.py +115 -0
  34. continuo_python_runtime-0.1.0/python-runtime-postgres/tests/test_integration_runtime_postgres.py +277 -0
  35. continuo_python_runtime-0.1.0/python-runtime-trino/README.md +77 -0
  36. continuo_python_runtime-0.1.0/python-runtime-trino/continuo_python_runtime_trino/__init__.py +0 -0
  37. continuo_python_runtime-0.1.0/python-runtime-trino/continuo_python_runtime_trino/adapter.py +388 -0
  38. continuo_python_runtime-0.1.0/python-runtime-trino/pyproject.toml +45 -0
  39. continuo_python_runtime-0.1.0/python-runtime-trino/tests/__init__.py +0 -0
  40. continuo_python_runtime-0.1.0/python-runtime-trino/tests/test_adapter_runtime_trino.py +210 -0
  41. continuo_python_runtime-0.1.0/python-runtime-trino/tests/test_integration_runtime_trino.py +366 -0
  42. continuo_python_runtime-0.1.0/template/.github/workflows/release.yml +58 -0
  43. continuo_python_runtime-0.1.0/template/Dockerfile +8 -0
  44. continuo_python_runtime-0.1.0/template/README.md +36 -0
  45. continuo_python_runtime-0.1.0/template/contracts/example.yml +12 -0
  46. continuo_python_runtime-0.1.0/template/scripts/example.py +3 -0
  47. continuo_python_runtime-0.1.0/tests/__init__.py +0 -0
  48. continuo_python_runtime-0.1.0/tests/conftest.py +81 -0
  49. continuo_python_runtime-0.1.0/tests/contract/__init__.py +0 -0
  50. continuo_python_runtime-0.1.0/tests/contract/test_loader.py +344 -0
  51. continuo_python_runtime-0.1.0/tests/contract/test_merge.py +118 -0
  52. continuo_python_runtime-0.1.0/tests/contract/test_model.py +25 -0
  53. continuo_python_runtime-0.1.0/tests/smoke/node_smoke/contracts/smoke.yml +13 -0
  54. continuo_python_runtime-0.1.0/tests/smoke/node_smoke/scripts/smoke.py +2 -0
  55. continuo_python_runtime-0.1.0/tests/smoke/postgres-stack/docker-compose.yml +16 -0
  56. continuo_python_runtime-0.1.0/tests/smoke/trino-stack/catalog/iceberg.properties +10 -0
  57. continuo_python_runtime-0.1.0/tests/smoke/trino-stack/docker-compose.yml +60 -0
  58. continuo_python_runtime-0.1.0/tests/test_cli.py +187 -0
  59. continuo_python_runtime-0.1.0/tests/test_conform.py +234 -0
  60. continuo_python_runtime-0.1.0/tests/test_context.py +51 -0
  61. continuo_python_runtime-0.1.0/tests/test_errors.py +27 -0
  62. continuo_python_runtime-0.1.0/tests/test_harness.py +261 -0
  63. continuo_python_runtime-0.1.0/tests/test_hashing.py +53 -0
  64. continuo_python_runtime-0.1.0/tests/test_lint.py +210 -0
  65. continuo_python_runtime-0.1.0/tests/test_package.py +5 -0
  66. continuo_python_runtime-0.1.0/tests/test_template.py +50 -0
  67. continuo_python_runtime-0.1.0/tests/test_types.py +83 -0
  68. continuo_python_runtime-0.1.0/uv.lock +600 -0
@@ -0,0 +1,14 @@
1
+ .venv/
2
+ .git/
3
+ .superpowers/
4
+ .idea/
5
+ __pycache__/
6
+ *.pyc
7
+ .pytest_cache/
8
+ .mypy_cache/
9
+ .ruff_cache/
10
+ dist/
11
+ docs/
12
+ template/
13
+ tests/
14
+ python-runtime-*/tests/
@@ -0,0 +1,25 @@
1
+ name: ci
2
+ on:
3
+ push: { branches: [main] }
4
+ pull_request:
5
+ jobs:
6
+ test:
7
+ runs-on: ubuntu-latest
8
+ steps:
9
+ - uses: actions/checkout@v4
10
+ - uses: astral-sh/setup-uv@v5
11
+ # continuo-validation-contract resolves from PyPI (==0.3.0).
12
+ - name: Sync
13
+ run: uv sync --all-packages --all-groups
14
+ - name: Lint
15
+ run: uv run ruff check .
16
+ - name: Types (runtime)
17
+ run: uv run mypy continuo_python_runtime
18
+ - name: Types (postgres adapter)
19
+ run: uv run --package continuo-python-runtime-postgres mypy python-runtime-postgres/continuo_python_runtime_postgres
20
+ - name: Types (trino adapter)
21
+ run: uv run --package continuo-python-runtime-trino mypy python-runtime-trino/continuo_python_runtime_trino
22
+ - name: Tests (runtime)
23
+ run: uv run pytest --cov=continuo_python_runtime -v
24
+ - name: Tests (adapter units)
25
+ run: uv run pytest python-runtime-postgres/tests python-runtime-trino/tests -m "not integration" -v
@@ -0,0 +1,164 @@
1
+ name: images
2
+ on:
3
+ pull_request:
4
+ paths:
5
+ - "Dockerfile.*"
6
+ - "tests/smoke/**"
7
+ - "continuo_python_runtime/**"
8
+ - "python-runtime-postgres/**"
9
+ - "python-runtime-trino/**"
10
+ - "pyproject.toml"
11
+ - ".github/workflows/images.yml"
12
+ push:
13
+ tags:
14
+ - "v*"
15
+
16
+ jobs:
17
+ build:
18
+ strategy:
19
+ matrix:
20
+ engine: [postgres, trino]
21
+ runs-on: ubuntu-latest
22
+ steps:
23
+ - uses: actions/checkout@v4
24
+ - name: Build ${{ matrix.engine }} image
25
+ run: docker build -f Dockerfile.${{ matrix.engine }} -t cpr-smoke-${{ matrix.engine }} .
26
+ - name: Save image
27
+ run: docker save cpr-smoke-${{ matrix.engine }} -o /tmp/cpr-smoke-${{ matrix.engine }}.tar
28
+ - uses: actions/upload-artifact@v4
29
+ with:
30
+ name: cpr-smoke-${{ matrix.engine }}-image
31
+ path: /tmp/cpr-smoke-${{ matrix.engine }}.tar
32
+ retention-days: 1
33
+
34
+ smoke-postgres:
35
+ needs: build
36
+ runs-on: ubuntu-latest
37
+ services:
38
+ postgres:
39
+ image: postgres:16
40
+ env:
41
+ POSTGRES_USER: continuo
42
+ POSTGRES_PASSWORD: continuo
43
+ POSTGRES_DB: warehouse
44
+ ports:
45
+ - 5432:5432
46
+ options: >-
47
+ --health-cmd "pg_isready -U continuo -d warehouse"
48
+ --health-interval 5s
49
+ --health-timeout 5s
50
+ --health-retries 10
51
+ steps:
52
+ - uses: actions/checkout@v4
53
+ - uses: actions/download-artifact@v4
54
+ with:
55
+ name: cpr-smoke-postgres-image
56
+ path: /tmp
57
+ - name: Load image
58
+ run: docker load -i /tmp/cpr-smoke-postgres.tar
59
+ - name: Install psql client
60
+ run: |
61
+ sudo apt-get update
62
+ sudo apt-get install -y --no-install-recommends postgresql-client
63
+ - name: Seed source table
64
+ env:
65
+ PGPASSWORD: continuo
66
+ run: |
67
+ psql -h localhost -U continuo -d warehouse -c \
68
+ "CREATE SCHEMA analytics; CREATE TABLE analytics.src(id int); INSERT INTO analytics.src VALUES (1),(2),(3);"
69
+ - name: Run smoke node
70
+ run: |
71
+ docker run --rm --network host \
72
+ -v "$PWD/tests/smoke/node_smoke:/app" \
73
+ -e NODE_ID=python-model.smoke.analytics.smoke \
74
+ -e TABLE_NAME=smoke \
75
+ -e TARGET_SCHEMA=analytics \
76
+ -e POSTGRES_HOST=localhost \
77
+ -e POSTGRES_PORT=5432 \
78
+ -e POSTGRES_DB=warehouse \
79
+ -e POSTGRES_USER=continuo \
80
+ -e POSTGRES_PASSWORD=continuo \
81
+ cpr-smoke-postgres | tee /tmp/smoke.out
82
+ test "${PIPESTATUS[0]}" -eq 0
83
+ shell: bash
84
+ - name: Assert success sentinel
85
+ run: grep -q '"status":"success"' /tmp/smoke.out
86
+ - name: Assert row count
87
+ env:
88
+ PGPASSWORD: continuo
89
+ run: |
90
+ count=$(psql -h localhost -U continuo -d warehouse -t -c "select count(*) from analytics.smoke" | tr -d '[:space:]')
91
+ test "$count" = "3"
92
+
93
+ smoke-trino:
94
+ needs: build
95
+ runs-on: ubuntu-latest
96
+ steps:
97
+ - uses: actions/checkout@v4
98
+ - uses: actions/download-artifact@v4
99
+ with:
100
+ name: cpr-smoke-trino-image
101
+ path: /tmp
102
+ - name: Load image
103
+ run: docker load -i /tmp/cpr-smoke-trino.tar
104
+ - name: Start trino stack
105
+ run: docker compose -f tests/smoke/trino-stack/docker-compose.yml up -d --wait
106
+ - name: Seed source table
107
+ run: |
108
+ docker exec "$(docker compose -f tests/smoke/trino-stack/docker-compose.yml ps -q trino)" \
109
+ trino --execute "CREATE SCHEMA IF NOT EXISTS iceberg.analytics; CREATE TABLE iceberg.analytics.src(id int); INSERT INTO iceberg.analytics.src VALUES 1,2,3"
110
+ - name: Run smoke node
111
+ run: |
112
+ docker run --rm \
113
+ -v "$PWD/tests/smoke/node_smoke:/app" \
114
+ -e NODE_ID=python-model.smoke.analytics.smoke \
115
+ -e TABLE_NAME=smoke \
116
+ -e TARGET_SCHEMA=analytics \
117
+ -e TRINO_HOST=host.docker.internal \
118
+ -e TRINO_PORT=18080 \
119
+ -e TRINO_CATALOG=iceberg \
120
+ -e TRINO_USER=continuo \
121
+ --add-host=host.docker.internal:host-gateway \
122
+ cpr-smoke-trino | tee /tmp/smoke-trino.out
123
+ test "${PIPESTATUS[0]}" -eq 0
124
+ shell: bash
125
+ - name: Assert success sentinel
126
+ run: grep -q '"status":"success"' /tmp/smoke-trino.out
127
+ - name: Assert row count
128
+ run: |
129
+ count=$(docker exec "$(docker compose -f tests/smoke/trino-stack/docker-compose.yml ps -q trino)" \
130
+ trino --execute "select count(*) from iceberg.analytics.smoke" | tr -d '[:space:]"')
131
+ test "$count" = "3"
132
+ - name: Tear down trino stack
133
+ if: always()
134
+ run: docker compose -f tests/smoke/trino-stack/docker-compose.yml down -v
135
+
136
+ # On tag push: log in to ghcr.io and push ghcr.io/${{ github.repository }}:<tag>-postgres
137
+ # and :<tag>-trino. continuo-validation-contract, continuo-python-runtime-postgres,
138
+ # and continuo-python-runtime-trino are all installed from the build context /
139
+ # PyPI already (see Dockerfile.postgres / Dockerfile.trino and
140
+ # publish-pypi.yml) — nothing pending here.
141
+ publish:
142
+ if: startsWith(github.ref, 'refs/tags/v')
143
+ needs: [smoke-postgres, smoke-trino]
144
+ strategy:
145
+ matrix:
146
+ engine: [postgres, trino]
147
+ runs-on: ubuntu-latest
148
+ permissions:
149
+ contents: read
150
+ packages: write
151
+ steps:
152
+ - uses: actions/checkout@v4
153
+ - name: Log in to ghcr.io
154
+ uses: docker/login-action@v3
155
+ with:
156
+ registry: ghcr.io
157
+ username: ${{ github.actor }}
158
+ password: ${{ secrets.GITHUB_TOKEN }}
159
+ - name: Build and push ${{ matrix.engine }} image
160
+ run: |
161
+ tag="${GITHUB_REF_NAME}"
162
+ image="ghcr.io/${{ github.repository }}:${tag}-${{ matrix.engine }}"
163
+ docker build -f Dockerfile.${{ matrix.engine }} -t "$image" .
164
+ docker push "$image"
@@ -0,0 +1,129 @@
1
+ name: publish-pypi
2
+
3
+ # Publishes the continuo-python-runtime package (and, in the two adapter jobs
4
+ # below, the continuo-python-runtime-postgres / continuo-python-runtime-trino
5
+ # packages) via PyPI Trusted Publishing (OIDC, no stored token).
6
+ #
7
+ # Tag glob note: "python-runtime-v*" requires a literal "v" immediately after
8
+ # the dash, so it matches only the harness tag (python-runtime-v1.2.3[-testN])
9
+ # and never "python-runtime-postgres-v*" / "python-runtime-trino-v*" (whose
10
+ # char after the shared "python-runtime-" prefix is "p"/"t", not "v") — the
11
+ # three globs are disjoint, so none of the three jobs below shadows another.
12
+ # Tag `<pkg>-v<ver>-test<n>` publishes to TestPyPI; `<pkg>-v<ver>` publishes to
13
+ # real PyPI. The GitHub environment name is what each PyPI "pending publisher"
14
+ # is registered against.
15
+ #
16
+ # Ordering: continuo-validation-contract==0.3.0 must exist on the target index
17
+ # before publishing here — consumers of the wheel resolve it from that index
18
+ # ([tool.uv.sources] is dev-only and not embedded in the wheel).
19
+ on:
20
+ push:
21
+ tags:
22
+ - "python-runtime-v*"
23
+ - "python-runtime-postgres-v*"
24
+ - "python-runtime-trino-v*"
25
+
26
+ permissions: {}
27
+
28
+ jobs:
29
+ publish-runtime:
30
+ if: startsWith(github.ref_name, 'python-runtime-v')
31
+ runs-on: ubuntu-latest
32
+ environment: ${{ contains(github.ref_name, '-test') && 'testpypi' || 'pypi' }}
33
+ permissions:
34
+ id-token: write # OIDC token for Trusted Publishing
35
+ contents: read # actions/checkout needs read access to the repo
36
+ steps:
37
+ - uses: actions/checkout@v4
38
+ - uses: astral-sh/setup-uv@v5
39
+ - name: Test before publishing
40
+ run: |
41
+ uv sync --all-groups
42
+ uv run pytest -q
43
+ - name: Build the sdist + wheel
44
+ run: uv build -o dist
45
+ - name: Publish to TestPyPI
46
+ if: contains(github.ref_name, '-test')
47
+ # Pinned to a commit SHA. release/v1 is a moving branch, and this job
48
+ # holds id-token: write for PyPI Trusted Publishing — a compromised
49
+ # revision could mint a token and publish under this project's name.
50
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
51
+ with:
52
+ repository-url: https://test.pypi.org/legacy/
53
+ packages-dir: dist
54
+ - name: Publish to PyPI
55
+ if: ${{ !contains(github.ref_name, '-test') }}
56
+ # Pinned to a commit SHA. release/v1 is a moving branch, and this job
57
+ # holds id-token: write for PyPI Trusted Publishing — a compromised
58
+ # revision could mint a token and publish under this project's name.
59
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
60
+ with:
61
+ packages-dir: dist
62
+
63
+ publish-runtime-postgres:
64
+ if: startsWith(github.ref_name, 'python-runtime-postgres-v')
65
+ runs-on: ubuntu-latest
66
+ environment: ${{ contains(github.ref_name, '-test') && 'testpypi-postgres' || 'pypi-postgres' }}
67
+ permissions:
68
+ id-token: write # OIDC token for Trusted Publishing
69
+ contents: read # actions/checkout needs read access to the repo
70
+ steps:
71
+ - uses: actions/checkout@v4
72
+ - uses: astral-sh/setup-uv@v5
73
+ - name: Test before publishing
74
+ run: |
75
+ uv sync --all-packages --all-groups
76
+ uv run pytest python-runtime-postgres/tests -m "not integration" -q
77
+ - name: Build the sdist + wheel
78
+ run: uv build --package continuo-python-runtime-postgres python-runtime-postgres -o dist
79
+ - name: Publish to TestPyPI
80
+ if: contains(github.ref_name, '-test')
81
+ # Pinned to a commit SHA. release/v1 is a moving branch, and this job
82
+ # holds id-token: write for PyPI Trusted Publishing — a compromised
83
+ # revision could mint a token and publish under this project's name.
84
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
85
+ with:
86
+ repository-url: https://test.pypi.org/legacy/
87
+ packages-dir: dist
88
+ - name: Publish to PyPI
89
+ if: ${{ !contains(github.ref_name, '-test') }}
90
+ # Pinned to a commit SHA. release/v1 is a moving branch, and this job
91
+ # holds id-token: write for PyPI Trusted Publishing — a compromised
92
+ # revision could mint a token and publish under this project's name.
93
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
94
+ with:
95
+ packages-dir: dist
96
+
97
+ publish-runtime-trino:
98
+ if: startsWith(github.ref_name, 'python-runtime-trino-v')
99
+ runs-on: ubuntu-latest
100
+ environment: ${{ contains(github.ref_name, '-test') && 'testpypi-trino' || 'pypi-trino' }}
101
+ permissions:
102
+ id-token: write # OIDC token for Trusted Publishing
103
+ contents: read # actions/checkout needs read access to the repo
104
+ steps:
105
+ - uses: actions/checkout@v4
106
+ - uses: astral-sh/setup-uv@v5
107
+ - name: Test before publishing
108
+ run: |
109
+ uv sync --all-packages --all-groups
110
+ uv run pytest python-runtime-trino/tests -m "not integration" -q
111
+ - name: Build the sdist + wheel
112
+ run: uv build --package continuo-python-runtime-trino python-runtime-trino -o dist
113
+ - name: Publish to TestPyPI
114
+ if: contains(github.ref_name, '-test')
115
+ # Pinned to a commit SHA. release/v1 is a moving branch, and this job
116
+ # holds id-token: write for PyPI Trusted Publishing — a compromised
117
+ # revision could mint a token and publish under this project's name.
118
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
119
+ with:
120
+ repository-url: https://test.pypi.org/legacy/
121
+ packages-dir: dist
122
+ - name: Publish to PyPI
123
+ if: ${{ !contains(github.ref_name, '-test') }}
124
+ # Pinned to a commit SHA. release/v1 is a moving branch, and this job
125
+ # holds id-token: write for PyPI Trusted Publishing — a compromised
126
+ # revision could mint a token and publish under this project's name.
127
+ uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # release/v1
128
+ with:
129
+ packages-dir: dist
@@ -0,0 +1,9 @@
1
+ .idea/
2
+ .DS_Store
3
+ __pycache__/
4
+ .venv/
5
+ dist/
6
+ .superpowers/
7
+ .pytest_cache/
8
+ .mypy_cache/
9
+ .ruff_cache/
@@ -0,0 +1,13 @@
1
+ FROM python:3.14-slim
2
+ # Until the runtime and adapter packages publish to (Test)PyPI, install both
3
+ # from the build context; the contract from PyPI.
4
+ COPY pyproject.toml README.md /src/
5
+ COPY continuo_python_runtime /src/continuo_python_runtime
6
+ COPY python-runtime-postgres /src/python-runtime-postgres
7
+ RUN pip install --no-cache-dir \
8
+ "continuo-validation-contract==0.3.0" \
9
+ /src/python-runtime-postgres \
10
+ /src
11
+ ENV CONTRACT_DIR=/app/contracts APP_ROOT=/app
12
+ WORKDIR /app
13
+ ENTRYPOINT ["continuo-runtime", "run"]
@@ -0,0 +1,13 @@
1
+ FROM python:3.14-slim
2
+ # Until the runtime and adapter packages publish to (Test)PyPI, install both
3
+ # from the build context; the contract from PyPI.
4
+ COPY pyproject.toml README.md /src/
5
+ COPY continuo_python_runtime /src/continuo_python_runtime
6
+ COPY python-runtime-trino /src/python-runtime-trino
7
+ RUN pip install --no-cache-dir \
8
+ "continuo-validation-contract==0.3.0" \
9
+ /src/python-runtime-trino \
10
+ /src
11
+ ENV CONTRACT_DIR=/app/contracts APP_ROOT=/app
12
+ WORKDIR /app
13
+ ENTRYPOINT ["continuo-runtime", "run"]
@@ -0,0 +1,193 @@
1
+ Metadata-Version: 2.4
2
+ Name: continuo-python-runtime
3
+ Version: 0.1.0
4
+ Summary: Runtime harness, contract tooling, and CI lint for Continuo python nodes.
5
+ Author: Simone Carolini
6
+ Maintainer: Simone Carolini
7
+ Classifier: Development Status :: 4 - Beta
8
+ Classifier: Intended Audience :: Developers
9
+ Classifier: Programming Language :: Python :: 3.14
10
+ Requires-Python: >=3.14
11
+ Requires-Dist: continuo-validation-contract==0.3.0
12
+ Requires-Dist: pyarrow==25.0.0
13
+ Requires-Dist: pyyaml==6.0.3
14
+ Description-Content-Type: text/markdown
15
+
16
+ # Continuo Python Runtime
17
+
18
+ Runtime harness, contract tooling, and CI lint for Continuo python nodes.
19
+ This repo is what domain data teams (marketing, finance, …) template from to
20
+ ship a python node into Continuo: write a contract + a `run(ctx)` script,
21
+ push to `main`, and CI does the rest — lint, validate, merge, build, publish,
22
+ and register the release with Continuo.
23
+
24
+ ## What this repo is
25
+
26
+ Three artifacts come out of this repository:
27
+
28
+ - **The `continuo-python-runtime` PyPI package** — the `continuo-runtime` CLI
29
+ (`validate` / `merge` / `hash` / `lint` / `run`) and the harness library
30
+ (`conform()`, `RunContext`, the error taxonomy) that domain repos install.
31
+ - **Per-engine base images**, one per warehouse engine (`...-postgres`,
32
+ `...-trino`), that domain repos build `FROM`. Each image bakes in the
33
+ runtime, a single `RuntimeAdapter` for that engine, and the
34
+ `continuo-runtime run` entrypoint.
35
+ - **`template/`** — a copy-ready domain repo: `Dockerfile`, `contracts/`,
36
+ `scripts/`, and the `release.yml` CI/CD workflow.
37
+
38
+ Base images and the PyPI publication of this package land with this repo's
39
+ PR 9 (the image pipeline); until then, install the runtime from git as noted
40
+ in `template/.github/workflows/release.yml`.
41
+
42
+ ### Package map
43
+
44
+ Each per-engine base image bakes together three PyPI packages: the harness,
45
+ the engine adapter, and the published contract. The harness and both engine
46
+ adapters live in *this* repo (`continuo-python-runtime`) as a uv workspace;
47
+ `continuo-validation-runners` owns the *validation-side* (offline lint/merge)
48
+ counterparts of the same engines, which is a separate concern from the
49
+ data-plane adapters here.
50
+
51
+ | Package (distribution name) | Module | Lives in | Role |
52
+ | --- | --- | --- | --- |
53
+ | `continuo-python-runtime` | `continuo_python_runtime` | this repo (root) | Harness: CLI, `conform()`, `RunContext`, error taxonomy. |
54
+ | `continuo-python-runtime-postgres` | `continuo_python_runtime_postgres` | this repo, `python-runtime-postgres/` | Data-plane `RuntimeAdapter` for Postgres (`fetch`/`ensure_table`/`load`). |
55
+ | `continuo-python-runtime-trino` | `continuo_python_runtime_trino` | this repo, `python-runtime-trino/` | Data-plane `RuntimeAdapter` for Trino/Iceberg. |
56
+ | `continuo-validation-contract` | `continuo_validation_contract` | `continuo-validation-runners` | The published contract (schema, `RuntimeAdapter` port, result-block format) both sides depend on. |
57
+ | `continuo-validation-postgres` / `continuo-validation-trino` | `continuo_validation_postgres` / `continuo_validation_trino` | `continuo-validation-runners` | Validation-side (lint/merge, no live warehouse I/O) adapters — not to be confused with the data-plane adapters above. |
58
+
59
+ All three packages built in this repo resolve `continuo-validation-contract`
60
+ from PyPI (`==0.3.0`); the two adapter packages are uv workspace members
61
+ (`[tool.uv.workspace]` in the root `pyproject.toml`), so `uv sync
62
+ --all-packages --all-groups` at the repo root installs everything for local
63
+ development.
64
+
65
+ ## Quickstart for domain teams
66
+
67
+ 1. Copy `template/` into a new repository.
68
+ 2. Edit `template/.github/workflows/release.yml` and set `SERVICE` to your
69
+ service name (one service name per domain repo).
70
+ 3. Configure repository variables in GitHub (Settings → Secrets and
71
+ variables → Actions): `REGISTRY` (your Docker registry), `BUCKET` (your
72
+ S3 bucket for contract artifacts), `RELEASE_ENDPOINT` (the release
73
+ webhook endpoint). `RELEASE_ENDPOINT` is the **base URL** of the Continuo
74
+ API (no `/releases` suffix) — the workflow appends `/releases` itself.
75
+ 4. Configure repository secrets: `AWS_ACCESS_KEY_ID` and
76
+ `AWS_SECRET_ACCESS_KEY` for the S3 upload. The template workflow pushes
77
+ the built image to GHCR using the workflow's own `GITHUB_TOKEN` (granted
78
+ `packages: write`) — no registry secret is needed for that. If you point
79
+ `REGISTRY` at a different or private registry, add your own `docker
80
+ login` step to `release.yml`.
81
+ 5. Write a contract file under `contracts/` (see
82
+ `template/contracts/example.yml`) and a script under `scripts/` that
83
+ implements `run(ctx)` (see `template/scripts/example.py`).
84
+ 6. Push to `main`. The `release.yml` workflow lints the scripts, validates
85
+ and merges the contracts, builds and pushes the image, uploads the merged
86
+ contract to S3, and POSTs the release.
87
+
88
+ ## The script API
89
+
90
+ A node script is a Python file with exactly one required entry point:
91
+
92
+ ```python
93
+ def run(ctx):
94
+ ...
95
+ ```
96
+
97
+ - `ctx` is a `RunContext` (`continuo_python_runtime.context.RunContext`).
98
+ Its only method is `ctx.read(name)`, where `name` is one of the read
99
+ names declared under the node's `reads:` map in the contract — reading
100
+ anything else raises `ReadError`. Each declared read is fetched once and
101
+ memoized; `ctx.read(name)` returns a `pyarrow.Table`.
102
+ - `run(ctx)` can return anything Arrow-convertible: a `pyarrow.Table`
103
+ as-is, a pandas `DataFrame` (converted via
104
+ `pa.Table.from_pandas(..., preserve_index=False)`), or any object
105
+ implementing the Arrow C stream protocol (`__arrow_c_stream__`) — for
106
+ example a polars DataFrame. Returning anything else raises `ScriptError`.
107
+ - The dataframe library is your choice. No dataframe library is baked into
108
+ the base image — the package's own runtime dependencies are `pyarrow`,
109
+ `PyYAML`, and `continuo-validation-contract`. Add whatever you script
110
+ against (pandas, polars, …) as a `RUN pip install` line in your own
111
+ `Dockerfile`, on top of the base image.
112
+ - Scripts do not import warehouse drivers, write raw SQL literals, or call
113
+ data-access methods directly — `continuo-runtime lint` rejects those:
114
+ - forbidden driver imports (`psycopg2`/`sqlalchemy`/`trino`/etc.),
115
+ - SQL string literals (in plain strings, f-strings, and `+` concatenation),
116
+ - forbidden data-access calls (`execute`/`read_sql`/etc.), including ones
117
+ reached via a `from ... import` alias (e.g. `from pandas import
118
+ read_sql as rs` then calling `rs(...)`),
119
+ - private/protected attribute access (`obj._x`) — except on `self`/`cls`,
120
+ so a script's own class-private helpers (`self._helper()`) aren't
121
+ flagged.
122
+
123
+ The SQL-literal rule is best-effort: docstrings are exempt, but other
124
+ prose may still occasionally match. The hard guarantees enforced by lint
125
+ are the driver-import and data-access-call rules, together with the fact
126
+ that `RunContext` only exposes `ctx.read()` — all warehouse access goes
127
+ through it.
128
+ - The harness — not the script — performs the write. It calls `conform()`
129
+ on whatever `run()` returned and issues the only INSERT; the script never
130
+ writes directly.
131
+
132
+ ## Conform rules
133
+
134
+ `conform()` (`continuo_python_runtime/conform.py`) enforces the node's
135
+ declared `output_columns` on the table `run()` returned, in this order:
136
+
137
+ | Check | Behavior |
138
+ | --- | --- |
139
+ | Duplicate columns | Any duplicate column name in the returned table always raises `ConformError`. |
140
+ | Extra columns | Governed by the node's `extra_columns` policy: `raise` (default) fails the run; `warn` drops the undeclared column(s) and logs a warning. |
141
+ | Missing columns | Any declared column absent from the returned table always raises `ConformError`. |
142
+ | Column order | The table is reselected into the declared column order. |
143
+ | Strict cast | Each column is cast to its declared Arrow type with `safe=True`. Casts pyarrow's `safe=True` would silently accept but that are not value-lossless are rejected before the cast even runs: floating → decimal (rounds to scale), non-boolean → boolean (coerces truthiness), timestamp → date (drops time-of-day). Any other cast failure also raises `ConformError`. |
144
+ | Not-null | A column declared `nullable: false` that contains any null raises `ConformError`. |
145
+ | VARCHAR/CHAR length | A column declared `VARCHAR(n)`/`CHAR(n)` whose longest value exceeds `n` raises `ConformError`. |
146
+
147
+ ## Error taxonomy
148
+
149
+ Every runtime failure is one of five `HarnessError` subclasses
150
+ (`continuo_python_runtime/errors.py`). The sentinel result block's message
151
+ is prefixed `<ErrorClass>: ` so failures can be triaged without parsing free
152
+ text.
153
+
154
+ | Class | Meaning | Typical fix target |
155
+ | --- | --- | --- |
156
+ | `ContractError` | Contract missing or invalid, node not found for `NODE_ID`, or the declared script is missing/unreachable. | The contract yaml or the `script:` path. |
157
+ | `ReadError` | `ctx.read()` was called with an undeclared name, or a declared read failed at the warehouse. | The `reads:` map, or the upstream query/warehouse access. |
158
+ | `ScriptError` | `run()` raised, has no callable `run`, or returned a value that isn't Arrow-convertible. | The node script. |
159
+ | `ConformError` | Structural mismatch (extra/missing/duplicate columns), a strict-cast failure, a not-null violation, or a VARCHAR/CHAR overflow. | The script's output shape, or the `output_columns` declaration. |
160
+ | `LoadError` | Adapter construction failed, or the DDL/INSERT failed at the warehouse during the write. | Warehouse connectivity/permissions, or the target table. |
161
+
162
+ ## Engine selection
163
+
164
+ A domain repo picks its warehouse engine by which base image it builds
165
+ `FROM`:
166
+
167
+ ```dockerfile
168
+ FROM ghcr.io/carolsimone/continuo-python-runtime:v0.1.0-postgres
169
+ # or
170
+ FROM ghcr.io/carolsimone/continuo-python-runtime:v0.1.0-trino
171
+ ```
172
+
173
+ Each image bakes in exactly one `RuntimeAdapter` for that engine — installed
174
+ from this repo's `python-runtime-postgres/` or `python-runtime-trino/`
175
+ package (see the package map above; both adapters live in this repo, not
176
+ `continuo-validation-runners`) — registered under the
177
+ `continuo_runtime.adapters` entry-point group (entry names `postgres` /
178
+ `trino`). The harness discovers it via `discover_runtime_adapter()` at run
179
+ time, so a single image serves every node in the service. The executor
180
+ injects the
181
+ warehouse connection as environment variables (engine-native, e.g.
182
+ `POSTGRES_HOST`/`POSTGRES_DB`/`POSTGRES_USER`) plus the node-selection
183
+ environment (`NODE_ID`, `TABLE_NAME`, `TARGET_SCHEMA`, and optionally
184
+ `CONTRACT_DIR`/`APP_ROOT`) that `continuo-runtime run` reads to dispatch the
185
+ right node's script.
186
+
187
+ ## Further reading
188
+
189
+ - `docs/superpowers/specs/2026-07-31-python-runtime-design.md` — this
190
+ repo's design.
191
+ - `docs/boundary-contract.md` — the parent design's boundary contract (§13):
192
+ the five surfaces (S3 artifact, `content_hash`, the release call, the
193
+ runtime image, and the domain repo's CI/CD) that this repo implements.