walflux 0.1.0__tar.gz

walflux-0.1.0/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,60 @@
+name: Bug report
+description: Something WalFlux did wrong — wrong aggregates, a crash, a stuck daemon.
+labels: [bug]
+body:
+  - type: input
+    id: walflux-version
+    attributes:
+      label: WalFlux version
+      description: Output of `walflux --version` (plus the commit, if installed from git).
+      placeholder: walflux 0.1.0
+    validations:
+      required: true
+  - type: input
+    id: postgres-version
+    attributes:
+      label: Postgres version
+      description: Output of `SELECT version();`.
+      placeholder: PostgreSQL 16.4
+    validations:
+      required: true
+  - type: dropdown
+    id: hosting
+    attributes:
+      label: Where does Postgres run?
+      options:
+        - Self-hosted
+        - Amazon RDS / Aurora
+        - Google Cloud SQL
+        - Supabase
+        - Neon
+        - Other managed service
+    validations:
+      required: true
+  - type: textarea
+    id: views-config
+    attributes:
+      label: The config's `views:` section
+      description: Redact nothing else — the DSN is not needed.
+      render: yaml
+    validations:
+      required: true
+  - type: textarea
+    id: logs
+    attributes:
+      label: Daemon logs around the failure
+      description: >
+        The daemon logs every flush with its LSN, which usually pinpoints the
+        moment things went wrong — include the lines before and after.
+      render: text
+    validations:
+      required: true
+  - type: textarea
+    id: expected-vs-actual
+    attributes:
+      label: Expected vs actual
+      description: >
+        For wrong aggregates, the ideal evidence is the ground-truth `GROUP BY`
+        over the source table next to the `walflux.<view>` contents.
+    validations:
+      required: true

walflux-0.1.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1 @@
+blank_issues_enabled: true

walflux-0.1.0/.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,27 @@
+name: Feature request
+description: Something WalFlux should do.
+labels: [enhancement]
+body:
+  - type: checkboxes
+    id: roadmap
+    attributes:
+      label: Roadmap check
+      options:
+        - label: >
+            I checked the
+            [roadmap](https://github.com/sedai77/walflux-postgres-incremental-views#roadmap)
+            — joins, min/max, snapshot re-sync, and a Prometheus metrics
+            endpoint are already planned.
+          required: true
+  - type: textarea
+    id: problem
+    attributes:
+      label: The problem
+      description: What you are trying to do — not just the feature you want.
+    validations:
+      required: true
+  - type: textarea
+    id: proposal
+    attributes:
+      label: Proposed behavior
+      description: Config shape, CLI flags, semantics — whatever you have.

walflux-0.1.0/.github/pull_request_template.md

@@ -0,0 +1,10 @@
+## What & why
+
+<!-- Small, focused diffs with a clear "why" land quickly. -->
+
+## Checklist
+
+- [ ] `uv run --extra dev pytest -m "not integration"` passes
+- [ ] `uv run --extra dev ruff check .` is clean
+- [ ] Interface changes update [`docs/SPEC.md`](../docs/SPEC.md) — interface changes are spec changes
+- [ ] Behavior changes have tests; anything touching the apply/checkpoint/ack path keeps the `kill -9` integration test honest

walflux-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,72 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  lint:
+    name: lint (ruff)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: "3.12"
+      - name: ruff check
+        run: uv run --extra dev ruff check .
+
+  unit:
+    name: unit (py ${{ matrix.python }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python: ["3.10", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: ${{ matrix.python }}
+      - name: pytest (unit)
+        run: uv run --extra dev pytest -m "not integration"
+
+  integration:
+    name: integration (postgres 16, wal_level=logical)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: "3.12"
+      # Service containers cannot override the image command, and we need
+      # wal_level=logical — so start Postgres with a plain `docker run`.
+      - name: Start Postgres 16 with logical WAL
+        run: |
+          docker run -d --name pg \
+            -e POSTGRES_PASSWORD=postgres \
+            -p 5432:5432 \
+            postgres:16 \
+            -c wal_level=logical \
+            -c max_replication_slots=8 \
+            -c max_wal_senders=8
+      - name: Wait for Postgres
+        run: |
+          for i in $(seq 1 30); do
+            if docker exec pg pg_isready -U postgres -h localhost; then
+              exit 0
+            fi
+            echo "waiting for postgres ($i)..."
+            sleep 1
+          done
+          echo "postgres never became ready" >&2
+          docker logs pg >&2
+          exit 1
+      - name: pytest (integration)
+        env:
+          WALFLUX_TEST_DSN: postgresql://postgres:postgres@localhost:5432/postgres
+        run: uv run --extra dev pytest -m integration -v
+      - name: Postgres logs (on failure)
+        if: failure()
+        run: docker logs pg

walflux-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,114 @@
+# Release pipeline, triggered by pushing a v* tag (e.g. v0.1.0).
+#
+# No API tokens anywhere: PyPI uses Trusted Publishing (OIDC) and ghcr.io uses
+# the built-in GITHUB_TOKEN. One-time setup by the repo owner:
+#   1. pypi.org -> account -> Publishing -> add a (pending) trusted publisher:
+#      project "walflux", owner "sedai77",
+#      repository "walflux-postgres-incremental-views",
+#      workflow "release.yml", environment "pypi".
+#   2. GitHub repo Settings -> Environments -> create "pypi".
+# Then: bump __version__ in walflux/__init__.py, move the CHANGELOG.md
+# Unreleased entries into a dated section, and push the matching tag.
+name: Release
+
+on:
+  push:
+    tags: ["v*"]
+
+jobs:
+  build:
+    name: build (wheel + sdist)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          python-version: "3.12"
+      - name: Build
+        run: uv build
+      - name: Check metadata
+        run: uvx twine check dist/*
+      - name: Tag must match the package version
+        run: |
+          version="${GITHUB_REF_NAME#v}"
+          if [ ! -f "dist/walflux-${version}.tar.gz" ]; then
+            echo "tag ${GITHUB_REF_NAME} does not match the built version:" >&2
+            ls dist/ >&2
+            echo "bump __version__ in walflux/__init__.py before tagging" >&2
+            exit 1
+          fi
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  pypi:
+    name: publish to PyPI (trusted publishing)
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write # OIDC is the whole credential — no token, no secret
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1
+
+  docker:
+    name: publish image (ghcr.io)
+    needs: build
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: docker/setup-qemu-action@v3
+      - uses: docker/setup-buildx-action@v3
+      - uses: docker/login-action@v3
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - uses: docker/build-push-action@v6
+        with:
+          context: .
+          target: runtime
+          platforms: linux/amd64,linux/arm64
+          push: true
+          tags: |
+            ghcr.io/${{ github.repository }}:${{ github.ref_name }}
+            ghcr.io/${{ github.repository }}:latest
+
+  github-release:
+    name: GitHub release
+    needs: [build, pypi]
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - name: Extract this version's notes from CHANGELOG.md
+        run: |
+          awk -v heading="## [${GITHUB_REF_NAME#v}]" '
+            index($0, heading) == 1 { found = 1; next }
+            found && (/^## / || /^\[.*\]: /) { exit }
+            found { print }
+          ' CHANGELOG.md > notes.md
+          if [ ! -s notes.md ]; then
+            echo "no CHANGELOG.md section for ${GITHUB_REF_NAME#v}" >&2
+            echo "See CHANGELOG.md." > notes.md
+          fi
+      - name: Create the release
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          gh release create "$GITHUB_REF_NAME" dist/* \
+            --title "WalFlux ${GITHUB_REF_NAME#v}" \
+            --notes-file notes.md

walflux-0.1.0/.gitignore

@@ -0,0 +1,25 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/
+.venv/
+venv/
+.pytest_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+
+# Environment
+.env
+*.local.yaml
+
+# Demo artifacts
+demo/.pgdata/
+*.log
+
+# OS / editor
+.DS_Store
+.idea/
+.vscode/

walflux-0.1.0/CHANGELOG.md

@@ -0,0 +1,34 @@
+# Changelog
+
+All notable changes to WalFlux are documented here. The format follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versions follow
+[semver](https://semver.org/) with the usual 0.x caveat: breaking changes may
+land in minor versions (0.2.0), fixes in patch versions (0.1.1). Anything
+behavior-affecting — and anything that requires `walflux setup --force` — is
+called out explicitly in its entry.
+
+## [Unreleased]
+
+## [0.1.0] - 2026-07-26
+
+Initial release.
+
+### Added
+
+- `pgoutput` (protocol v1) decoder for logical replication messages.
+- Incrementally maintained `count` / `sum` / `avg` aggregate tables with
+  `GROUP BY` — including NULL group keys and global (no `group_by`) views.
+- Exactly-once apply: each batch's deltas and its replication checkpoint commit
+  in one target transaction; redelivered transactions are discarded by commit
+  LSN. Proven across `kill -9` by the integration suite and `make kill9`.
+- Consistent snapshot backfill: `walflux setup` builds targets from the slot's
+  exported snapshot, so backfill and stream meet with no gap and no overlap.
+- CLI: `walflux setup` / `run` / `status` / `teardown`, plus
+  `python -m walflux` as an alternative entry point.
+- A PEP 561 `py.typed` marker: the package's type annotations are visible to
+  mypy/pyright in downstream projects.
+- A wrong or unreachable DSN fails as one tidy `cannot connect to database`
+  error line instead of a raw psycopg2 traceback.
+
+[Unreleased]: https://github.com/sedai77/walflux-postgres-incremental-views/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/sedai77/walflux-postgres-incremental-views/releases/tag/v0.1.0

walflux-0.1.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,134 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual
+identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the overall
+  community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or advances
+  of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email address,
+  without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement by contacting the
+repository maintainers through GitHub — a direct message, or
+[GitHub's report abuse form](https://github.com/contact/report-abuse) for
+serious cases. All complaints will be reviewed and investigated promptly and
+fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series of
+actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or permanent
+ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior, harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within the
+community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
+[https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations

walflux-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,89 @@
+# Contributing to WalFlux
+
+Thanks for helping out. This is a small, sharply-scoped codebase — the fastest way to
+contribute well is to read [`docs/SPEC.md`](docs/SPEC.md) first. It is the authoritative
+contract for every module's interface, and pull requests are reviewed against it.
+The reasoning behind the design lives in [`DESIGN.md`](DESIGN.md).
+
+## Development setup
+
+WalFlux uses [uv](https://docs.astral.sh/uv/) for everything. There is nothing to
+"install" beyond cloning:
+
+```bash
+git clone https://github.com/sedai77/walflux-postgres-incremental-views.git
+cd walflux-postgres-incremental-views
+uv run --python 3.12 --extra dev pytest -m "not integration"
+```
+
+`uv run` resolves and caches the environment (runtime deps: `psycopg2-binary`,
+`PyYAML`; dev deps: `pytest`, `ruff`) on first use.
+
+## Running the tests
+
+Unit tests need no database:
+
+```bash
+uv run --python 3.12 --extra dev pytest -m "not integration"
+```
+
+The integration suite needs a Postgres 15+ with `wal_level=logical` and a
+superuser-ish DSN in `WALFLUX_TEST_DSN` (it creates replication slots, publications,
+and scratch schemas). The easiest way is docker:
+
+```bash
+docker run -d --name walflux-test-pg \
+  -e POSTGRES_PASSWORD=postgres -p 5432:5432 postgres:16 \
+  -c wal_level=logical -c max_replication_slots=8 -c max_wal_senders=8
+
+WALFLUX_TEST_DSN=postgresql://postgres:postgres@localhost:5432/postgres \
+  uv run --python 3.12 --extra dev pytest -m integration -v
+
+docker rm -f walflux-test-pg   # when done
+```
+
+Without `WALFLUX_TEST_DSN` the integration tests skip with an explanatory reason —
+that is expected locally.
+
+CI runs three jobs on every push and pull request: `ruff check`, the unit suite on
+Python 3.10 and 3.13 (the oldest and newest supported), and the integration suite
+against Postgres 16. All three must pass.
+
+## Style
+
+- `ruff check .` must be clean (`uv run --python 3.12 --extra dev ruff check .`);
+  configuration lives in `pyproject.toml`.
+- Full type hints, `from __future__ import annotations` at the top of every module.
+- Docstrings on public functions. Comments are reserved for invariants the code
+  cannot express (the skip rule, the snapshot handshake, feedback ordering).
+- SQL identifiers go through `walflux.common.quote_ident()`; values are always
+  parameterized (`%s`), never interpolated.
+- No new runtime dependencies. `psycopg2-binary` + `PyYAML` is the complete list,
+  and keeping it that way is a feature. psycopg2 imports stay inside the modules
+  that talk to the database (`replication.py`, `bootstrap.py`, `checkpoint.py`,
+  `daemon.py`) so everything else is importable without a database.
+
+## Making changes
+
+1. Check whether your change conflicts with [`docs/SPEC.md`](docs/SPEC.md). If it
+   does, the PR should update the spec too — interface changes are spec changes.
+2. Add or update tests. Behavior changes to the protocol decoder, delta arithmetic,
+   or config validation need unit tests; anything touching the apply/checkpoint/ack
+   path needs to keep the kill -9 integration test honest.
+3. Keep PRs focused. Small, reviewable diffs with a clear "why" in the description
+   land quickly.
+
+Bug reports are most useful with: Postgres version, WalFlux version, the config's
+`views:` section, and daemon logs around the failure (the daemon logs every flush
+with its LSN, which usually pinpoints the moment things went wrong).
+
+## Repository metadata (for maintainers)
+
+Suggested GitHub "About" description:
+
+> Millisecond-fresh materialized views for Postgres via logical replication —
+> no extensions, no triggers, no second database.
+
+Suggested topics: `postgres`, `postgresql`, `materialized-views`,
+`incremental-view-maintenance`, `change-data-capture`, `logical-replication`,
+`pgoutput`, `cdc`, `streaming`, `python`.