view-scheduler 0.4.0__tar.gz

view_scheduler-0.4.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Kiran B
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

view_scheduler-0.4.0/PKG-INFO

@@ -0,0 +1,230 @@
+Metadata-Version: 2.4
+Name: view-scheduler
+Version: 0.4.0
+Summary: Recommend when a Unity Catalog view can be materialized with every dependency already refreshed
+Author: Kiran B
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/kiranb-1508/view-scheduler-docs
+Keywords: databricks,unity-catalog,delta,scheduling
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: databricks-sdk>=0.30
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: build>=1.0; extra == "dev"
+Dynamic: license-file
+
+# View Materialization Scheduler
+
+Two questions, two tools.
+
+- **When** should this view run? → `advise`. It reads history and suggests a time.
+- **Can** it run right now? → `can_refresh`, the gate. It checks today's state.
+
+```
+$ view-scheduler sales_gold.customer_360
+
+VIEW   sales_gold.customer_360
+       8 dependencies · last 30 days · times Asia/Kolkata
+
+       SCHEDULE AT  06:25    safe — every job had finished, on all 30 days
+       EARLIER      05:45    one job had not finished yet, on 1 of 30 days
+```
+
+## Read this before you rely on it
+
+**The recommendation is advisory.** It says when a view *usually* can run; it
+does not check today. If a dependency runs later than it ever has, the job
+starts anyway and publishes stale data, silently. Pair it with the gate —
+`can_refresh` — which checks every dependency's actual last write before the
+refresh runs. Set the schedule from history; guard the run with the gate.
+
+**The arithmetic is still being validated.** A 30-day fixture rig
+(`rig/`) is running now to check the day-spread logic against real multi-day
+commit history. Until that completes, treat output as indicative. Hence `0.4.0`.
+
+## Install
+
+### Databricks (recommended)
+
+Ship the wheel through a Unity Catalog Volume, so access is governed by UC
+grants rather than by GitHub accounts.
+
+Once, to create the shared location:
+
+```sql
+CREATE VOLUME IF NOT EXISTS <catalog>.<schema>.libs;
+GRANT READ VOLUME ON VOLUME <catalog>.<schema>.libs TO `<group>`;
+```
+
+Build and upload a release:
+
+```bash
+python -m build --wheel
+databricks fs cp dist/view_scheduler-0.4.0-py3-none-any.whl \
+  dbfs:/Volumes/<catalog>/<schema>/libs/
+```
+
+Keep old versions in the volume so running jobs can stay pinned.
+
+Then, in a notebook:
+
+```python
+%pip install /Volumes/<catalog>/<schema>/libs/view_scheduler-0.4.0-py3-none-any.whl
+dbutils.library.restartPython()
+```
+
+Or as a job library, including from a DAB:
+
+```yaml
+tasks:
+  - task_key: recommend
+    libraries:
+      - whl: /Volumes/<catalog>/<schema>/libs/view_scheduler-0.4.0-py3-none-any.whl
+```
+
+### From source
+
+For working on the package itself:
+
+```bash
+pip install -e ".[dev]"                                     # a checkout
+pip install git+https://github.com/kiranb-1508/view-scheduler-docs.git
+```
+
+## Use
+
+As a command:
+
+```bash
+export WAREHOUSE_ID=<sql warehouse id>
+view-scheduler catalog.schema.my_view
+view-scheduler catalog.schema.my_view --days 60 --buffer 30 --quiet
+```
+
+As a library:
+
+```python
+from view_scheduler import advise
+
+advice = advise("sales.gold.customer_360")   # WAREHOUSE_ID from the env
+advice.safe_time      # datetime.time(6, 25)  or None
+advice.earlier_time   # datetime.time(5, 45)  or None
+advice.next_day       # True when the (+1 day) marker applies
+advice.dependencies   # one row per upstream job
+advice.to_dict()      # JSON, for a job log
+print(advice)         # the report
+```
+
+`advise` prints the report and returns an `Advice`. Nothing is dropped
+silently — anything unreadable comes back in `advice.unreadable` with a reason.
+
+`analyze` is the low-level call. It returns a tuple
+`(text, histories, skipped)` for code that needs the raw per-table history.
+
+Four settings: `days_history` (30), `buffer_min` (15), `timezone`
+(`Asia/Kolkata`), and `fresh_hours` (24) — the gate's window. There is a
+standing rule to ask before adding a fifth.
+
+### Gate: is my view ready to refresh?
+
+Before you materialize a view, ask whether its inputs are actually fresh. The
+gate walks every dependency and checks that its last real write is inside a
+rolling window (24 hours by default).
+
+```python
+from view_scheduler import can_refresh
+
+result = can_refresh("sales.gold.v_orders")          # WAREHOUSE_ID from $env
+if result.ready:
+    ...run the refresh...
+else:
+    print(result)                                    # says exactly what to fix
+```
+
+A list works the same way. You get one verdict per view, and one overall:
+
+```python
+result = can_refresh(["sales.gold.v_orders", "sales.gold.v_revenue"])
+print(result.ready)                 # True only when every view is ready
+print(result.blockers())            # ('sales.silver.daily_sales',)
+print(result.to_dict())             # JSON, for a job log
+```
+
+```
+GATE  BLOCKED   2 views, window 24h, times Asia/Kolkata
+
+  READY   sales.gold.v_orders
+      fresh  sales.silver.daily_orders  (MANAGED)  last refresh 2026-10-07 04:00 IST (11h 48m ago)
+
+  BLOCKED sales.gold.v_revenue
+      fresh  sales.silver.orders        (MANAGED)  last refresh 2026-10-07 04:00 IST (11h 48m ago)
+      stale  sales.silver.daily_sales   (MANAGED)  last refresh 2026-10-05 04:00 IST (2d 11h ago) — older than the 24h window
+
+To unblock, refresh:
+  sales.silver.daily_sales   (needed by sales.gold.v_revenue)
+```
+
+The gate decides; it never refreshes. It is read-only, like `analyze`. The rules
+are the whole table:
+
+| Case | Verdict |
+|---|---|
+| Wrote inside the window | fresh |
+| Wrote before the window | stale — **blocks** |
+| No qualifying write in its history | **blocks** |
+| Cannot be read (missing, no permission) | **blocks** — the gate fails closed |
+| Streaming table | checked from its history, like any table — one on a schedule is stale between runs |
+| Materialized view | read from its pipeline event log |
+| Function dependency (a UDF) | a note, not a blocker |
+
+`Workspace(warehouse_id=...)` and `Config.fresh_hours` are optional. Without a
+`Workspace`, the gate reads `WAREHOUSE_ID` from the environment, so the
+one-liner above is enough inside a job.
+
+**Permissions.** The caller needs `SELECT` on each dependency (for
+`DESCRIBE HISTORY`) and `USE` on the catalogs and schemas involved. Materialized
+views are read through their pipeline event log, which by default only the
+pipeline's run-as user can read. Without that access the gate **blocks** the MV:
+it reports `history unreadable (permission)`. The call itself does not raise.
+
+## Design docs
+
+| File | What it is |
+|---|---|
+| [docs/01-plan.html](docs/01-plan.html) | The plan. Start here — output shape, what's built, what was deliberately cut |
+| [docs/02-scheduler-design.html](docs/02-scheduler-design.html) | Scheduler design: architecture, code flow, four steps, worked example |
+| [docs/03-impact-design.html](docs/03-impact-design.html) | Impact analysis: given a table, what depends on it. Separate tool, separate data source |
+| [docs/04-library-flow.html](docs/04-library-flow.html) | Visual end-to-end flow: input, functions, branches, recommendation, and report |
+| [docs/05-interactive-guide.html](docs/05-interactive-guide.html) | Interactive guide: clickable flow, a gate simulator, the design decisions, and the open findings |
+| [docs/06-code-walkthrough.html](docs/06-code-walkthrough.html) | Code walkthrough: the module map, call traces, a class tour, and a searchable index of all 66 functions |
+| [rig/README.md](rig/README.md) | The 30-day fixture rig and its checkpoints |
+| [GENIE-PROMPT.md](GENIE-PROMPT.md) | Build prompt for Databricks Genie Code, phased with verification gates |
+
+Section 08 of the scheduler design lists what the docs describe but nobody
+built. The runtime gate is no longer on that list — it ships as `can_refresh`.
+
+## The two things most likely to bite
+
+1. **Operation class comes from `operationParameters.mode`, not the operation
+   name.** The commonest Delta operation name is plain `WRITE`; only its mode
+   says whether it replaced or appended. Classify by name and most tables read
+   as dormant — a confident wrong answer, not an error.
+2. **`DESCRIBE HISTORY` carries no partition values.** Verified against the docs.
+   Any design that attributes commits to business dates via partitions is built
+   on an input that does not exist.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+176 tests, all offline — no Spark session, no network, no credentials.
+`tools/` holds diagnostic scripts that run against a live workspace;
+`rig/` builds and checks the 30-day fixture. Neither ships in the wheel.

view_scheduler-0.4.0/README.md

@@ -0,0 +1,210 @@
+# View Materialization Scheduler
+
+Two questions, two tools.
+
+- **When** should this view run? → `advise`. It reads history and suggests a time.
+- **Can** it run right now? → `can_refresh`, the gate. It checks today's state.
+
+```
+$ view-scheduler sales_gold.customer_360
+
+VIEW   sales_gold.customer_360
+       8 dependencies · last 30 days · times Asia/Kolkata
+
+       SCHEDULE AT  06:25    safe — every job had finished, on all 30 days
+       EARLIER      05:45    one job had not finished yet, on 1 of 30 days
+```
+
+## Read this before you rely on it
+
+**The recommendation is advisory.** It says when a view *usually* can run; it
+does not check today. If a dependency runs later than it ever has, the job
+starts anyway and publishes stale data, silently. Pair it with the gate —
+`can_refresh` — which checks every dependency's actual last write before the
+refresh runs. Set the schedule from history; guard the run with the gate.
+
+**The arithmetic is still being validated.** A 30-day fixture rig
+(`rig/`) is running now to check the day-spread logic against real multi-day
+commit history. Until that completes, treat output as indicative. Hence `0.4.0`.
+
+## Install
+
+### Databricks (recommended)
+
+Ship the wheel through a Unity Catalog Volume, so access is governed by UC
+grants rather than by GitHub accounts.
+
+Once, to create the shared location:
+
+```sql
+CREATE VOLUME IF NOT EXISTS <catalog>.<schema>.libs;
+GRANT READ VOLUME ON VOLUME <catalog>.<schema>.libs TO `<group>`;
+```
+
+Build and upload a release:
+
+```bash
+python -m build --wheel
+databricks fs cp dist/view_scheduler-0.4.0-py3-none-any.whl \
+  dbfs:/Volumes/<catalog>/<schema>/libs/
+```
+
+Keep old versions in the volume so running jobs can stay pinned.
+
+Then, in a notebook:
+
+```python
+%pip install /Volumes/<catalog>/<schema>/libs/view_scheduler-0.4.0-py3-none-any.whl
+dbutils.library.restartPython()
+```
+
+Or as a job library, including from a DAB:
+
+```yaml
+tasks:
+  - task_key: recommend
+    libraries:
+      - whl: /Volumes/<catalog>/<schema>/libs/view_scheduler-0.4.0-py3-none-any.whl
+```
+
+### From source
+
+For working on the package itself:
+
+```bash
+pip install -e ".[dev]"                                     # a checkout
+pip install git+https://github.com/kiranb-1508/view-scheduler-docs.git
+```
+
+## Use
+
+As a command:
+
+```bash
+export WAREHOUSE_ID=<sql warehouse id>
+view-scheduler catalog.schema.my_view
+view-scheduler catalog.schema.my_view --days 60 --buffer 30 --quiet
+```
+
+As a library:
+
+```python
+from view_scheduler import advise
+
+advice = advise("sales.gold.customer_360")   # WAREHOUSE_ID from the env
+advice.safe_time      # datetime.time(6, 25)  or None
+advice.earlier_time   # datetime.time(5, 45)  or None
+advice.next_day       # True when the (+1 day) marker applies
+advice.dependencies   # one row per upstream job
+advice.to_dict()      # JSON, for a job log
+print(advice)         # the report
+```
+
+`advise` prints the report and returns an `Advice`. Nothing is dropped
+silently — anything unreadable comes back in `advice.unreadable` with a reason.
+
+`analyze` is the low-level call. It returns a tuple
+`(text, histories, skipped)` for code that needs the raw per-table history.
+
+Four settings: `days_history` (30), `buffer_min` (15), `timezone`
+(`Asia/Kolkata`), and `fresh_hours` (24) — the gate's window. There is a
+standing rule to ask before adding a fifth.
+
+### Gate: is my view ready to refresh?
+
+Before you materialize a view, ask whether its inputs are actually fresh. The
+gate walks every dependency and checks that its last real write is inside a
+rolling window (24 hours by default).
+
+```python
+from view_scheduler import can_refresh
+
+result = can_refresh("sales.gold.v_orders")          # WAREHOUSE_ID from $env
+if result.ready:
+    ...run the refresh...
+else:
+    print(result)                                    # says exactly what to fix
+```
+
+A list works the same way. You get one verdict per view, and one overall:
+
+```python
+result = can_refresh(["sales.gold.v_orders", "sales.gold.v_revenue"])
+print(result.ready)                 # True only when every view is ready
+print(result.blockers())            # ('sales.silver.daily_sales',)
+print(result.to_dict())             # JSON, for a job log
+```
+
+```
+GATE  BLOCKED   2 views, window 24h, times Asia/Kolkata
+
+  READY   sales.gold.v_orders
+      fresh  sales.silver.daily_orders  (MANAGED)  last refresh 2026-10-07 04:00 IST (11h 48m ago)
+
+  BLOCKED sales.gold.v_revenue
+      fresh  sales.silver.orders        (MANAGED)  last refresh 2026-10-07 04:00 IST (11h 48m ago)
+      stale  sales.silver.daily_sales   (MANAGED)  last refresh 2026-10-05 04:00 IST (2d 11h ago) — older than the 24h window
+
+To unblock, refresh:
+  sales.silver.daily_sales   (needed by sales.gold.v_revenue)
+```
+
+The gate decides; it never refreshes. It is read-only, like `analyze`. The rules
+are the whole table:
+
+| Case | Verdict |
+|---|---|
+| Wrote inside the window | fresh |
+| Wrote before the window | stale — **blocks** |
+| No qualifying write in its history | **blocks** |
+| Cannot be read (missing, no permission) | **blocks** — the gate fails closed |
+| Streaming table | checked from its history, like any table — one on a schedule is stale between runs |
+| Materialized view | read from its pipeline event log |
+| Function dependency (a UDF) | a note, not a blocker |
+
+`Workspace(warehouse_id=...)` and `Config.fresh_hours` are optional. Without a
+`Workspace`, the gate reads `WAREHOUSE_ID` from the environment, so the
+one-liner above is enough inside a job.
+
+**Permissions.** The caller needs `SELECT` on each dependency (for
+`DESCRIBE HISTORY`) and `USE` on the catalogs and schemas involved. Materialized
+views are read through their pipeline event log, which by default only the
+pipeline's run-as user can read. Without that access the gate **blocks** the MV:
+it reports `history unreadable (permission)`. The call itself does not raise.
+
+## Design docs
+
+| File | What it is |
+|---|---|
+| [docs/01-plan.html](docs/01-plan.html) | The plan. Start here — output shape, what's built, what was deliberately cut |
+| [docs/02-scheduler-design.html](docs/02-scheduler-design.html) | Scheduler design: architecture, code flow, four steps, worked example |
+| [docs/03-impact-design.html](docs/03-impact-design.html) | Impact analysis: given a table, what depends on it. Separate tool, separate data source |
+| [docs/04-library-flow.html](docs/04-library-flow.html) | Visual end-to-end flow: input, functions, branches, recommendation, and report |
+| [docs/05-interactive-guide.html](docs/05-interactive-guide.html) | Interactive guide: clickable flow, a gate simulator, the design decisions, and the open findings |
+| [docs/06-code-walkthrough.html](docs/06-code-walkthrough.html) | Code walkthrough: the module map, call traces, a class tour, and a searchable index of all 66 functions |
+| [rig/README.md](rig/README.md) | The 30-day fixture rig and its checkpoints |
+| [GENIE-PROMPT.md](GENIE-PROMPT.md) | Build prompt for Databricks Genie Code, phased with verification gates |
+
+Section 08 of the scheduler design lists what the docs describe but nobody
+built. The runtime gate is no longer on that list — it ships as `can_refresh`.
+
+## The two things most likely to bite
+
+1. **Operation class comes from `operationParameters.mode`, not the operation
+   name.** The commonest Delta operation name is plain `WRITE`; only its mode
+   says whether it replaced or appended. Classify by name and most tables read
+   as dormant — a confident wrong answer, not an error.
+2. **`DESCRIBE HISTORY` carries no partition values.** Verified against the docs.
+   Any design that attributes commits to business dates via partitions is built
+   on an input that does not exist.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+pytest
+```
+
+176 tests, all offline — no Spark session, no network, no credentials.
+`tools/` holds diagnostic scripts that run against a live workspace;
+`rig/` builds and checks the 30-day fixture. Neither ships in the wheel.

view_scheduler-0.4.0/pyproject.toml

@@ -0,0 +1,41 @@
+[build-system]
+requires = ["setuptools>=77", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "view-scheduler"
+version = "0.4.0"
+description = "Recommend when a Unity Catalog view can be materialized with every dependency already refreshed"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+authors = [{ name = "Kiran B" }]
+keywords = ["databricks", "unity-catalog", "delta", "scheduling"]
+
+classifiers = [
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+]
+
+dependencies = [
+    "databricks-sdk>=0.30",
+]
+
+[project.urls]
+Homepage = "https://github.com/kiranb-1508/view-scheduler-docs"
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.0",
+    "build>=1.0",
+]
+
+[project.scripts]
+view-scheduler = "view_scheduler.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+# src layout: only view_scheduler ships. rig/, tests/ and tools/ are repo-only
+# and stay out of the wheel — `unzip -l dist/*.whl` is the check.

view_scheduler-0.4.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

view_scheduler-0.4.0/src/view_scheduler/__init__.py

@@ -0,0 +1,34 @@
+"""View materialization scheduler.
+
+Two tools for a Unity Catalog view:
+
+  * `advise` — suggest the time each day the view can run with every upstream
+    job already finished. Computed from history. ADVISORY.
+  * `can_refresh` — a runtime GATE. Right now, has every dependency of this
+    view refreshed recently enough? READY or BLOCKED.
+
+READ ONLY. This package never writes, alters, optimizes, vacuums or drops a
+user table. Metadata and history reads only.
+
+`advise` predicts. `can_refresh` checks. Use the first to set the schedule,
+the second to guard the run. See README.md.
+"""
+
+from .advice import Advice, DependencyRun, advise
+from .client import Workspace
+from .config import Config
+from .gate import Dependency, GateResult, ViewResult, can_refresh
+from .run import analyze
+
+__all__ = [
+    "advise",
+    "Advice",
+    "analyze",
+    "can_refresh",
+    "Config",
+    "Dependency",
+    "DependencyRun",
+    "GateResult",
+    "ViewResult",
+    "Workspace",
+]