dash-observe 0.1.2__tar.gz

dash_observe-0.1.2/.editorconfig

@@ -0,0 +1,15 @@
+root = true
+
+[*]
+charset = utf-8
+end_of_line = lf
+insert_final_newline = true
+trim_trailing_whitespace = true
+indent_style = space
+indent_size = 4
+
+[*.{yml,yaml,json}]
+indent_size = 2
+
+[*.md]
+trim_trailing_whitespace = false

dash_observe-0.1.2/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,27 @@
+---
+name: Bug report
+about: Something doesn't work the way it should
+title: ''
+labels: bug
+assignees: ''
+---
+
+**Describe the bug**
+A clear description of what's wrong.
+
+**To Reproduce**
+```python
+from dashobserve import MonitorConfig, run_monitors
+# minimal config that reproduces the issue
+```
+
+**Expected behavior**
+What you expected to happen instead.
+
+**Environment**
+- `dash-observe` version: `pip show dash-observe`
+- Python version:
+- Running in: Databricks notebook / Jupyter / other
+
+**Additional context**
+Anything else relevant (screenshots, stack trace, etc.).

dash_observe-0.1.2/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,19 @@
+---
+name: Feature request
+about: Suggest a new monitor type or capability
+title: ''
+labels: enhancement
+assignees: ''
+---
+
+**What problem does this solve?**
+What are you trying to monitor that you currently can't (or can only do awkwardly)?
+
+**Proposed API**
+```python
+from dashobserve import MonitorConfig
+# what you'd want to be able to configure
+```
+
+**Alternatives considered**
+Any workarounds you're using today.

dash_observe-0.1.2/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,11 @@
+## Summary
+
+What does this PR change, and why?
+
+## Checklist
+
+- [ ] Tests added/updated for any behavior change
+- [ ] `pytest tests/ -v` passes locally
+- [ ] `ruff check dashobserve/` passes locally
+- [ ] `CLAUDE.md` updated if the module structure or a design rule changed
+- [ ] No version bump included (the release workflow handles that)

dash_observe-0.1.2/.github/workflows/ci.yml

@@ -0,0 +1,53 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: pip install ruff
+      - run: ruff check dashobserve/
+
+  test:
+    runs-on: ubuntu-latest
+    needs: lint
+    strategy:
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install
+        run: pip install -e ".[dev]" pytest pytest-cov
+      - name: Test
+        run: pytest tests/ -v --cov=dashobserve --cov-report=xml
+      - name: Upload coverage
+        uses: codecov/codecov-action@v4
+        with:
+          files: coverage.xml
+
+  build:
+    runs-on: ubuntu-latest
+    needs: test
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - run: pip install hatch
+      - run: hatch build
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/

dash_observe-0.1.2/.github/workflows/daily.yml

@@ -0,0 +1,33 @@
+name: Daily Tests
+
+on:
+  schedule:
+    - cron: "0 6 * * *"   # Every day 06:00 UTC — tests only, no commit
+  workflow_dispatch:
+
+jobs:
+  test:
+    name: Test (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install
+        run: pip install -e ".[dev]" pytest pytest-cov
+
+      - name: Run tests
+        run: pytest tests/ -v --cov=dashobserve --cov-report=xml --cov-report=term-missing
+
+      - name: Upload coverage
+        uses: codecov/codecov-action@v4
+        with:
+          files: coverage.xml
+          fail_ci_if_error: false

dash_observe-0.1.2/.github/workflows/release.yml

@@ -0,0 +1,265 @@
+name: Weekly Release
+
+on:
+  schedule:
+    - cron: "0 9 * * 1"   # Every Monday 09:00 UTC
+  workflow_dispatch:
+    inputs:
+      release_note:
+        description: "Optional release note (shown in GitHub release body)"
+        required: false
+        default: ""
+
+jobs:
+  # ── Gate: tests must pass ────────────────────────────────────────────────
+  test:
+    name: Test (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: true
+      matrix:
+        python-version: ["3.9", "3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install
+        run: pip install -e ".[dev]" pytest pytest-cov
+
+      - name: Run tests
+        run: pytest tests/ -v --cov=dashobserve --cov-report=xml --cov-report=term-missing
+
+      - name: Upload coverage
+        uses: codecov/codecov-action@v4
+        with:
+          files: coverage.xml
+          fail_ci_if_error: false
+
+  # ── Generate docs ────────────────────────────────────────────────────────
+  docs:
+    name: Generate API docs
+    runs-on: ubuntu-latest
+    needs: test
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install
+        run: pip install -e ".[dev]" pdoc
+
+      - name: Generate docs
+        run: |
+          pdoc dashobserve --output-dir docs/api --docformat google
+          echo "Docs generated at $(date -u)" > docs/api/.generated
+
+      - name: Upload docs artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: api-docs
+          path: docs/api/
+
+  # ── Release: tag, GitHub release, commit docs ────────────────────────────
+  release:
+    name: Bump version & release
+    runs-on: ubuntu-latest
+    needs: [test, docs]
+    permissions:
+      contents: write
+      pull-requests: write
+    outputs:
+      version: ${{ steps.bump.outputs.version }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install build tools
+        run: pip install hatch pdoc
+
+      - name: Bump patch version
+        id: bump
+        run: |
+          current=$(hatch version)
+          hatch version patch
+          new=$(hatch version)
+          echo "version=$new" >> $GITHUB_OUTPUT
+          echo "prev_version=$current" >> $GITHUB_OUTPUT
+          echo "Bumped $current → $new"
+
+      - name: Regenerate docs into repo
+        run: |
+          pip install -e ".[dev]"
+          pdoc dashobserve --output-dir docs/api --docformat google
+
+      - name: Build wheel + sdist
+        run: hatch build
+
+      - name: Write release notes
+        env:
+          VERSION: ${{ steps.bump.outputs.version }}
+          PREV_VERSION: ${{ steps.bump.outputs.prev_version }}
+          RELEASE_NOTE: ${{ github.event.inputs.release_note }}
+        run: |
+          cat > RELEASE_NOTES.md << EOF
+          ## DashObserve v${VERSION}
+
+          **Released:** $(date -u '+%Y-%m-%d')
+          **Previous:** v${PREV_VERSION}
+
+          $( [ -n "${RELEASE_NOTE}" ] && echo "### Notes" && echo "${RELEASE_NOTE}" || true )
+
+          ### What's included
+          - All tests passing across Python 3.9, 3.10, 3.11, 3.12
+          - API documentation regenerated (see \`docs/api/\`)
+          - Published to PyPI and Databricks Marketplace
+
+          ### Install
+          \`\`\`bash
+          pip install dash-observe==${VERSION}
+          \`\`\`
+
+          ### Quick Start (Databricks notebook)
+          \`\`\`python
+          %pip install dash-observe==${VERSION}
+          import dashobserve
+          dashobserve.launch()
+          \`\`\`
+          EOF
+
+      - name: Commit version bump + docs to a release branch
+        env:
+          VERSION: ${{ steps.bump.outputs.version }}
+        run: |
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+          # Clean up any orphaned tag/branch from a previous partial run
+          git push origin --delete "refs/tags/v${VERSION}" 2>/dev/null || true
+          git push origin --delete "release/v${VERSION}" 2>/dev/null || true
+          git tag -d "v${VERSION}" 2>/dev/null || true
+          git checkout -b "release/v${VERSION}"
+          git add .
+          git commit -m "release: v${VERSION} — tests passed, docs updated"
+          git tag "v${VERSION}"
+          git push origin "release/v${VERSION}"
+          git push origin "v${VERSION}"
+
+      - name: Create GitHub Release
+        uses: softprops/action-gh-release@v2
+        with:
+          tag_name: "v${{ steps.bump.outputs.version }}"
+          body_path: RELEASE_NOTES.md
+          files: dist/*
+
+      - name: Upload dist artifact for PyPI job
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Open and auto-merge release PR
+        continue-on-error: true
+        env:
+          GH_TOKEN: ${{ secrets.RELEASE_TOKEN || github.token }}
+          VERSION: ${{ steps.bump.outputs.version }}
+        run: |
+          gh pr create --base main --head "release/v${VERSION}" \
+            --title "release: v${VERSION}" \
+            --body "Automated release PR — tests passed, docs regenerated, version bumped to v${VERSION}." \
+            2>/dev/null || true
+          gh pr merge "release/v${VERSION}" --merge --admin --delete-branch
+
+  # ── Publish to PyPI (Trusted Publisher / OIDC — no token needed) ─────────
+  publish-pypi:
+    name: Publish to PyPI
+    runs-on: ubuntu-latest
+    needs: release
+    permissions:
+      id-token: write   # required for OIDC trusted publisher
+    environment:
+      name: pypi
+      url: https://pypi.org/project/dash-observe
+    steps:
+      - name: Download dist
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+
+  # ── Package for Databricks Marketplace ───────────────────────────────────
+  publish-databricks:
+    name: Package for Databricks Marketplace
+    runs-on: ubuntu-latest
+    needs: release
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: "v${{ needs.release.outputs.version }}"
+
+      - name: Download dist
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Build Marketplace bundle
+        env:
+          VERSION: ${{ needs.release.outputs.version }}
+        run: |
+          mkdir -p marketplace-bundle/files
+
+          cp dist/*.whl marketplace-bundle/files/
+
+          cat > marketplace-bundle/files/DashObserve_Quickstart.py << NBEOF
+          # Databricks notebook source
+          # MAGIC %md
+          # MAGIC # DashObserve v${VERSION} — Data Observability
+          # MAGIC Install and launch the interactive UI.
+
+          # COMMAND ----------
+          # MAGIC %pip install dash-observe==${VERSION}
+
+          # COMMAND ----------
+          dbutils.library.restartPython()
+
+          # COMMAND ----------
+          import dashobserve
+          dashobserve.launch()
+          NBEOF
+
+          cat > marketplace-bundle/listing.json << LEOF
+          {
+            "listing_name": "DashObserve — Data Observability",
+            "version": "${VERSION}",
+            "short_description": "Freshness, volume, and schema-change monitoring for Databricks tables",
+            "long_description": "DashObserve provides an ipywidgets UI inside Databricks notebooks to monitor table freshness, row-count volume, and schema changes, persisting results to a Delta history table. No external service to deploy.",
+            "categories": ["Data Engineering", "Developer Tools"],
+            "tags": ["data-observability", "databricks", "unity-catalog", "pyspark"],
+            "provider": "dash-libs",
+            "documentation_url": "https://github.com/dash-libs/dash-observe",
+            "source_url": "https://github.com/dash-libs/dash-observe",
+            "pypi_package": "dash-observe==${VERSION}"
+          }
+          LEOF
+
+          cd marketplace-bundle && zip -r ../dashobserve-marketplace-${VERSION}.zip .
+          echo "Bundle created: dashobserve-marketplace-${VERSION}.zip"
+
+      - name: Upload Marketplace bundle artifact
+        uses: actions/upload-artifact@v4
+        with:
+          name: marketplace-bundle
+          path: dashobserve-marketplace-*.zip
+          retention-days: 90

dash_observe-0.1.2/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.pyc
+*.egg-info/
+dist/
+build/
+.coverage
+coverage.xml
+.pytest_cache/
+.ruff_cache/

dash_observe-0.1.2/CHANGELOG.md

@@ -0,0 +1,13 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project uses patch-only [semantic versioning](https://semver.org/)
+bumps via the automated release workflow.
+
+## [Unreleased]
+
+### Added
+- Initial release: freshness, volume, and schema-change monitoring
+  (`MonitorConfig`, `run_monitors`, `MonitorReport`)
+- Notebook UI built on `dashui` (`dashobserve.launch()`)

dash_observe-0.1.2/CLAUDE.md

@@ -0,0 +1,46 @@
+# CLAUDE.md — dash-observe
+
+Part of the **Dashlibs** suite. Full suite-wide context (release process
+gotchas, PR/review norms, repo list, rationale for past decisions):
+see `~/dashlibs/CLAUDE.md`.
+
+## Purpose
+Data observability for Databricks — Monte Carlo-style monitoring without leaving the
+notebook. v1 covers freshness, volume, and schema-change monitoring. monitors.py=pure-logic
+check functions (no Spark, fully unit-testable), runner.py=MonitorConfig/run_monitors()/Delta
+history persistence (the Spark-touching glue).
+
+## Structure
+- `/ui.py`        — ipywidgets UI (built on `dashui`), `launch()` entrypoint
+- `/monitors.py`  — `check_freshness`, `check_volume`, `check_schema`/`diff_schema`, `MonitorResult`
+- `/runner.py`    — `MonitorConfig`, `MonitorReport`, `run_monitors()`
+- `tests/`        — pytest, no Spark dependency for unit tests
+
+## Key Design Rules
+- Never import Spark at module level — always inside functions
+- Keep check logic in `monitors.py` pure (plain Python in, plain Python out) so it's testable
+  without Spark; `runner.py` is the only place that touches `SparkSession`/Delta
+- Volume baselines and schema snapshots are read from the *same* history Delta table that
+  `run_monitors()` writes to (`mode("append")`) — there's no separate baseline store
+- UI widgets come from the shared `dashui` package (PyPI: `dash-uis`) — don't reimplement
+  headers/source pickers/output panels locally
+- `launch()` is always the public entrypoint for business users
+
+## v1 scope vs. planned direction
+v1 deliberately covers only freshness, volume, and schema-change — the most common pipeline
+failure modes, fast to ship correctly with pure-Python, fully-tested check logic.
+
+**Explicitly requested next direction (not yet built):** go deep on Unity Catalog / Databricks
+SDK integration rather than staying with basic `spark.table()` calls — system tables,
+`information_schema`, table properties/tags, audit logs, and especially **lineage** (UC's
+native lineage APIs, possibly cross-referencing dash-relate's ontology). Be comprehensive here,
+not conservative — this was explicit user direction, not a "nice to have."
+
+## CI
+- `ci.yml`    — PR gate: lint → test → build
+- `daily.yml` — 06:00 UTC: tests + .health/log.txt commit
+- `release.yml`— Monday 09:00 UTC: patch bump on a release branch → PR → GitHub release → PyPI
+
+This repo has no branch protection configured yet, but the same norm applies regardless:
+**every change goes through a PR with real human review before merging — no exceptions.**
+Don't self-merge, don't push directly to `main`. Prefer small, targeted commits/PRs.

dash_observe-0.1.2/CODE_OF_CONDUCT.md

@@ -0,0 +1,51 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic
+status, nationality, personal appearance, race, religion, or sexual identity
+and orientation.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment:
+
+- Demonstrating empathy and kindness toward other people
+- Being respectful of differing opinions, viewpoints, and experiences
+- Giving and gracefully accepting constructive feedback
+- Accepting responsibility and apologizing to those affected by our mistakes
+
+Examples of unacceptable behavior:
+
+- The use of sexualized language or imagery, and sexual attention of any kind
+- Trolling, insulting or derogatory comments, and personal or political attacks
+- Public or private harassment
+- Publishing others' private information without explicit permission
+
+## Enforcement Responsibilities
+
+Maintainers are responsible for clarifying and enforcing standards of
+acceptable behavior and will take appropriate, fair corrective action in
+response to any behavior they deem inappropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies
+when an individual is officially representing the community in public spaces.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the maintainers at the contact listed in `pyproject.toml`. All
+complaints will be reviewed and investigated promptly and fairly.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.1, available at
+https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
+
+[homepage]: https://www.contributor-covenant.org

dash_observe-0.1.2/CONTRIBUTING.md

@@ -0,0 +1,48 @@
+# Contributing to dash-observe
+
+Thanks for considering a contribution. This is a small, focused library — keep
+changes scoped and avoid adding dependencies unless they're essential.
+
+## Development setup
+
+```bash
+git clone https://github.com/dash-libs/dash-observe.git
+cd dash-observe
+python3 -m venv .venv && source .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+## Running tests
+
+```bash
+pytest tests/ -v
+```
+
+Tests must run without a Spark session or a live Databricks workspace — keep
+any Spark/Databricks-specific code inside functions (never at module level),
+and keep monitor check logic in `monitors.py` pure (plain Python in, plain
+Python out) so it's fully testable without Spark. `runner.py` is the only
+place that should touch `SparkSession`/Delta.
+
+## Linting
+
+```bash
+ruff check dashobserve/
+```
+
+CI runs lint → test (Python 3.9–3.12) → build on every PR; all three must pass.
+
+## Making a change
+
+1. Open an issue first for anything beyond a small fix, so we can agree on
+   the approach before you write code.
+2. Add or update tests for any behavior change.
+3. Keep `CLAUDE.md` in sync if you change the module structure or a design
+   rule documented there.
+4. Open a PR against `main`. The release workflow handles versioning —
+   don't bump the version yourself.
+
+## Reporting bugs / requesting features
+
+Use the issue templates in `.github/ISSUE_TEMPLATE/`. For security issues,
+see [SECURITY.md](SECURITY.md) instead of opening a public issue.