runrail 0.3.1__tar.gz

runrail-0.3.1/.gitignore

@@ -0,0 +1,16 @@
+.runrail/
+.env
+.venv/
+__pycache__/
+*.py[cod]
+.pytest_cache/
+.ruff_cache/
+frontend/node_modules/
+dist/
+*.egg-info/
+
+# macOS
+.DS_Store
+
+# Local Claude Code settings (machine-specific, may contain local paths)
+.claude/

runrail-0.3.1/CHANGELOG.md

@@ -0,0 +1,71 @@
+# Changelog
+
+Notable changes to RunRail. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow [Semantic Versioning](https://semver.org/).
+
+## [0.3.1] — 2026-07-15
+
+### Packaging
+- Available on PyPI — `pipx install runrail`. The wheel bundles the prebuilt web UI, so no Node is needed at runtime, and `pyproject.toml` now carries full metadata (description, Rutr Labs authors, keywords, trove classifiers, and project URLs).
+- Releases publish to PyPI via Trusted Publishing (OIDC) from CI when a release is published; a manual dispatch can dry-run to TestPyPI.
+- Data now lives in a stable per-user application-data directory by default — `~/Library/Application Support/RunRail` (macOS), `~/.local/share/RunRail` (Linux, honouring `$XDG_DATA_HOME`), `%LOCALAPPDATA%\RunRail` (Windows) — instead of `./.runrail` in the current directory. Override with `RUNRAIL_HOME` (e.g. `RUNRAIL_HOME=./.runrail`). `runrail serve` creates the directory and database on first run, so a fresh install needs only `pipx install runrail && runrail serve`.
+
+### Added
+- Aggregated stats endpoints: `GET /api/stats/summary` computes the dashboard and run-history metrics (live counts, 24-hour totals, 7-day success rate, average duration) in SQL, so the numbers stay accurate no matter how much history the browser has loaded.
+- Activity heatmap range selector: 4, 8, or 16 weeks or 6 months on the dashboard and workflow pages, remembered across sessions. The grid runs Monday through Sunday with every weekday labelled.
+- Task timeouts can be entered in seconds, minutes, hours, or days; values are stored as seconds and shown in the friendliest unit.
+- First-run import: when `runrail serve` targets a brand-new home from an interactive terminal, it offers to bring over an existing setup — either a previous RunRail data directory (database, logs, artifacts, and environments are copied after an integrity check, then upgraded by the normal startup migrations) or a workflows YAML from `runrail export`. Also available any time as `runrail import <path>`; it never overwrites an existing database.
+
+### Changed
+- Running progress bars redesigned as a comet: the fill grows with progress behind a bright twinkling head and leaves a dimmer, still-lit dot trail. Past the median duration the trail gradually shifts toward amber — fully amber at 135% — while the head holds its color until it catches up. One reusable treatment covers the wallboard, the run timeline, and indeterminate loaders.
+- The run timeline renders while a run is executing; running bars grow against the wall clock instead of the timeline appearing only after the run finishes.
+- Wallboard tiles are larger and clickable — each tile links to its workflow page.
+- Scheduled runs no longer record a `ds` parameter they never used; `{{ ds }}` still renders (defaulting to the run date) and backfills still set it explicitly.
+- Legibility pass: status chips use full-strength status colors, project cards match the workflow-card layout, and dependency-graph nodes got larger type.
+
+### Docs
+- Clarified the concurrency model to remove a wording contradiction: different workflows and independent tasks run concurrently on one machine, runs of the same workflow serialize by `max_concurrent_runs`, and distributed/remote workers are not supported yet.
+
+## [0.3.0] — 2026-07-09
+
+### Added
+- Live dependency graph: workflow pages render the task DAG, and run pages overlay live statuses — nodes highlight while running, edges trace as branches execute.
+- Gantt timeline v2: one lane per task with retry attempts as separate bars, a time axis, and detailed hover tooltips; parallel execution is visible at a glance, and running bars grow live against the wall clock.
+- ANSI color rendering in the log viewer (16-color, 256-color, and truecolor), plus in-log search with match navigation and tail-follow.
+- Daily activity heatmap on the dashboard and per workflow, backed by a new `GET /api/stats/daily` aggregation endpoint.
+- Wallboard: a full-screen, auto-refreshing status view at `/wallboard` for a team TV — health verdict, live-run cards with ETA, urgency-sorted tiles, failure streaks, overdue-schedule detection, and a failures strip.
+- Sparkle-fill progress indicator: running progress bars render as a dot-matrix that fills left-to-right with a scattered sparkle as a run advances toward its median duration; reused as a shared `LoadingBar` for environment builds and queued runs.
+- Graceful shutdown: `runrail serve` reports which runs are still executing (with a median-based ETA) on the first Ctrl+C and force-quits on the second; interrupted runs are recovered as failed on the next start so a killed worker never leaves a phantom run holding a concurrency slot.
+
+### Changed
+- Dashboard reorganized: the activity heatmap moved into the hero beside the headline, removing a large half-empty panel.
+- Run and dashboard views refresh reliably while live — a polling regression (the interval was torn down every render) is fixed, so the Gantt and statuses update without a manual reload.
+
+### Fixed
+- Removed a heavy visual-effects pass (film grain, cursor-tracked specular wash, and a modal depth-of-field blur) that desaturated content and hurt readability; surfaces are flat and legible again in both themes.
+- WebSocket log streams close promptly instead of hanging graceful shutdown, and no longer emit `CancelledError` tracebacks on exit.
+
+## [0.2.0] — 2026-07-08
+
+### Added
+- Failure notifications: per-workflow or global webhook (`RUNRAIL_NOTIFY_WEBHOOK_URL`), posting on the first failure after a success and again on recovery. Slack and Teams incoming webhooks work without configuration.
+- Auto-pause: workflows can disable themselves after N consecutive failures, with a webhook notice.
+- Run retry: `POST /api/runs/{id}/retry` and a Retry button on the run page re-queue any finished run with identical parameters.
+- Workflows as code: `runrail export` and `runrail apply` round-trip workflow and task definitions through YAML, referencing projects and environments by name for portability across machines.
+- GitHub Actions CI: lint and tests on Python 3.11/3.12 plus a frontend typecheck and build.
+- Live log tailing follows output as it streams, with a terminal-style caret while a task is running.
+
+### Fixed
+- Windows: managed environment builds no longer fail with `WinError 5` when OneDrive or antivirus briefly locks the build directory; directory promotion now retries and falls back to copy.
+- Hourly cron schedules (`30 * * * *`) were displayed as daily; schedule previews and the Upcoming list now interpret them correctly.
+- Schedule labels now render in the viewer's local timezone, matching the timestamps shown beside them.
+- Dark mode: native dropdown options and the date-picker icon are now legible on all platforms.
+- Reduced-motion environments (including Windows with animation effects off) keep functional indicators: running spinners continue, entrances fall back to fades instead of disappearing.
+
+## [0.1.0] — 2026-07-06
+
+Initial public release: projects, managed Python environments, workflows with dependency-ordered tasks (shell, Python, notebook, SQL), cron scheduling with coalescing, parallel task execution, backfills, retries, live logs, artifacts with retention cleanup, and the bundled web UI.
+
+[0.3.1]: https://github.com/rutr-labs/runrail/compare/v0.3.0...v0.3.1
+[0.3.0]: https://github.com/rutr-labs/runrail/compare/v0.2.0...v0.3.0
+[0.2.0]: https://github.com/rutr-labs/runrail/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/rutr-labs/runrail/releases/tag/v0.1.0

runrail-0.3.1/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2026 rutr-labs
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

runrail-0.3.1/PKG-INFO

@@ -0,0 +1,194 @@
+Metadata-Version: 2.4
+Name: runrail
+Version: 0.3.1
+Summary: A self-hosted workflow orchestrator for the Python scripts, notebooks, SQL, and shell commands you already have.
+Project-URL: Homepage, https://github.com/rutr-labs/runrail
+Project-URL: Repository, https://github.com/rutr-labs/runrail
+Project-URL: Documentation, https://github.com/rutr-labs/runrail#readme
+Project-URL: Issues, https://github.com/rutr-labs/runrail/issues
+Project-URL: Changelog, https://github.com/rutr-labs/runrail/blob/main/CHANGELOG.md
+Author-email: Rutr Labs <therutr@outlook.com>
+Maintainer-email: Rutr Labs <therutr@outlook.com>
+License: Apache-2.0
+License-File: LICENSE
+Keywords: airflow-alternative,cron,dag,data-engineering,elt,etl,jobs,notebooks,orchestration,orchestrator,papermill,pipelines,scheduler,self-hosted,workflow
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Web Environment
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Information Technology
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Office/Business :: Scheduling
+Classifier: Topic :: Software Development :: Build Tools
+Classifier: Topic :: System :: Monitoring
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: alembic>=1.13
+Requires-Dist: apscheduler<4,>=3.10
+Requires-Dist: fastapi>=0.115
+Requires-Dist: jinja2>=3.1
+Requires-Dist: platformdirs>=4
+Requires-Dist: psycopg[binary]>=3.2
+Requires-Dist: pydantic-settings>=2.5
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: sqlalchemy>=2.0
+Requires-Dist: typer>=0.12
+Requires-Dist: uvicorn[standard]>=0.30
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: httpx>=0.27; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Provides-Extra: notebook
+Requires-Dist: nbconvert>=7.16; extra == 'notebook'
+Requires-Dist: papermill>=2.6; extra == 'notebook'
+Description-Content-Type: text/markdown
+
+# RunRail
+
+**A self-hosted workflow orchestrator for the scripts you already have.** Schedule Python scripts, Jupyter notebooks, SQL files, and shell commands with dependencies, retries, logs, backfills, and a fast web UI — without rewriting anything.
+
+[![CI](https://github.com/rutr-labs/runrail/actions/workflows/ci.yml/badge.svg)](https://github.com/rutr-labs/runrail/actions/workflows/ci.yml)
+[![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
+![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)
+
+Most teams outgrow cron long before they need a distributed orchestrator. RunRail covers that middle ground: one process, one SQLite file, and your existing code running on a real scheduler with full observability. No DAG files to author, no decorators to add, no platform to operate.
+
+## Features
+
+- **Bring your own code** — point RunRail at a folder; scripts, notebooks, SQL, and shell commands run as-is in isolated subprocesses
+- **Dependency graphs with parallel execution** — independent tasks run concurrently; each task starts the moment its dependencies succeed
+- **Cron scheduling built for high frequency** — minute-level schedules with per-workflow concurrency limits, coalescing, and missed-run tolerance
+- **Managed Python environments** — declare pip requirements in the UI; RunRail builds and atomically swaps isolated virtualenvs, keeping the last working build on failure
+- **Failure webhooks with sane semantics** — one alert on the first failure, one on recovery; Slack and Teams incoming webhooks work as-is
+- **Auto-pause** — optionally disable a workflow after N consecutive failures instead of failing hundreds of times overnight
+- **Backfills and retries** — queue one run per date over a range; re-run any finished run with identical parameters in one click
+- **Workflows as code** — `runrail export` / `runrail apply` round-trip definitions through YAML for version control and code review
+- **Live dependency graph** — every run renders its task DAG with statuses updating in place as branches execute
+- **Live logs and artifacts** — streamed stdout/stderr with ANSI colors, search, tail-follow, timestamped notebook outputs, automatic retention cleanup
+- **A UI you'll actually use** — command palette (⌘K), Gantt timelines, activity heatmaps, a full-screen wallboard for the team TV, dark and light themes
+
+## Install
+
+RunRail ships as a single Python package with the web UI bundled in — **no Node required to run it**. Python 3.11+ is the only prerequisite.
+
+```bash
+pipx install runrail    # isolated, recommended — or: pip install runrail
+runrail serve
+```
+
+Then open **http://127.0.0.1:8080**. That's the whole setup.
+
+On first launch RunRail creates its database, logs, and artifact store automatically and starts the API, web UI, scheduler, and worker in a single process — no separate `init` step, no migrations to run by hand. Everything lives in a per-user application-data directory:
+
+| OS | Default location |
+|---|---|
+| macOS | `~/Library/Application Support/RunRail` |
+| Linux | `~/.local/share/RunRail` (honours `$XDG_DATA_HOME`) |
+| Windows | `%LOCALAPPDATA%\RunRail` |
+
+Set `RUNRAIL_HOME` to store everything somewhere else (e.g. `RUNRAIL_HOME=./.runrail` to keep data beside a project). For notebook tasks, install the extra: `pipx install "runrail[notebook]"`.
+
+From the UI, connect a project folder, add an environment, and build your first workflow.
+
+## Core concepts
+
+**Projects** point at directories where your code lives. **Environments** define how Python runs — a managed virtualenv built from declared pip requirements, an existing interpreter, or a Conda environment. **Workflows** group **tasks** into a dependency graph with an optional cron schedule. Environments resolve per task, falling back to the workflow default, then the project default; Python and notebook tasks never silently run on RunRail's own interpreter.
+
+The recommended environment setup is *Environments → New environment → Managed Python*: declare requirements such as `pandas==2.3.0`, and RunRail builds an isolated virtualenv, records the build log and Python version, and rebuilds atomically when the requirements change. A failed rebuild preserves the previous working runtime.
+
+## CLI
+
+```bash
+runrail run daily-refresh --param region=ca      # queue a manual run
+runrail backfill daily-refresh --from 2026-06-01 --to 2026-06-30
+runrail export -o workflows.yml                  # workflows as version-controllable YAML
+runrail apply workflows.yml                      # declarative upsert by name
+runrail import ~/old/.runrail                    # bring a previous setup into this home
+runrail cleanup --older-than-days 30 --dry-run   # prune old runs, logs, artifacts
+runrail status
+
+# run components as separate processes
+runrail api
+runrail scheduler
+runrail worker --concurrency 8
+```
+
+## Templating
+
+Task commands and paths are Jinja templates. Every run receives:
+
+| Variable | Meaning |
+|---|---|
+| `ds` | Logical date (`2026-07-08`); injected per-date during backfills |
+| `ts` / `ts_nodash` | Run timestamp, ISO and compact (`20260708T141005`) |
+| `run_id`, `task_run_id` | Identifiers for the current execution |
+| `project_root`, `artifacts_dir` | Resolved paths for the run |
+| *your parameters* | Run parameters and per-task parameters, merged |
+
+```bash
+python scripts/daily.py --date {{ ds }} --region {{ region }}
+```
+
+## Notifications
+
+Set `RUNRAIL_NOTIFY_WEBHOOK_URL` globally or a webhook per workflow. RunRail posts on the first failure after a success and again on recovery — never once per red run, so a broken two-minute schedule produces one alert rather than three hundred. The payload's `text` field renders directly in Slack and Microsoft Teams; structured fields are included for custom receivers. Pair with per-workflow auto-pause to stop repeat failures entirely.
+
+## Docker
+
+```bash
+docker compose up --build
+```
+
+Compose starts RunRail with PostgreSQL and persistent volumes on port 8080. Docker is optional for local use.
+
+## Architecture
+
+A FastAPI process serves the API and the prebuilt React UI. APScheduler acts purely as a clock, writing due runs to the database; while a run executes, the next scheduled iteration coalesces to a single queued run instead of piling up or being dropped. A bounded worker pool (`RUNRAIL_WORKER_CONCURRENCY`) claims queued runs atomically, so **different workflows execute concurrently** and a long job never blocks a frequent one. Runs of the *same* workflow are serialized by its `max_concurrent_runs` (default 1) so a slow run can't overlap its own next iteration. Within a run, the task graph executes with real parallelism (`RUNRAIL_TASK_PARALLELISM`): **independent tasks run concurrently**, each starting the moment its dependencies succeed, and every task is a subprocess with its own logs, timeout, and process-group cleanup.
+
+All of that concurrency happens on one machine (threads and subprocesses); RunRail does not yet distribute work across remote worker nodes.
+
+SQLite in WAL mode is the default store; set `RUNRAIL_DB_URL` for PostgreSQL. Logs live under `$RUNRAIL_HOME/logs/run_<id>/`, artifacts under `$RUNRAIL_HOME/artifacts/<id>/` with timestamped filenames so frequent runs and retries never collide.
+
+## Configuration
+
+| Variable | Default |
+|---|---|
+| `RUNRAIL_HOME` | Per-user data dir (see [Install](#install)) |
+| `RUNRAIL_DB_URL` | SQLite at `$RUNRAIL_HOME/runrail.db` |
+| `RUNRAIL_HOST` / `RUNRAIL_PORT` | `127.0.0.1` / `8080` |
+| `RUNRAIL_WORKER_CONCURRENCY` | `4` concurrent runs |
+| `RUNRAIL_TASK_PARALLELISM` | `4` concurrent tasks per run |
+| `RUNRAIL_RETENTION_DAYS` | unset; when set, finished runs older than this are deleted with their logs and artifacts |
+| `RUNRAIL_NOTIFY_WEBHOOK_URL` | unset; default failure/recovery webhook |
+| `RUNRAIL_BROWSE_ROOT` | user home; confines the UI file picker |
+
+Schedules evaluate in UTC; the UI displays them in your local timezone. For deployments reachable beyond localhost, set `RUNRAIL_BROWSE_ROOT` to a narrow directory and place the server behind authentication — RunRail does not yet ship its own.
+
+## Current limitations
+
+- Runs on a single machine — workflows and independent tasks run concurrently there, but work is not distributed across remote worker nodes yet
+- No built-in authentication or RBAC
+- SQL tasks execute against SQLite only
+- Secrets are plain environment variables
+
+## Roadmap
+
+Warehouse and database adapters, artifact previews, secrets management, API tokens and authentication, and richer run analytics. See open issues for details.
+
+## Development
+
+```bash
+pip install -e '.[dev]'
+pytest
+ruff check .
+cd frontend && npm install && npm run build   # writes the bundle into src/runrail/web/static
+```
+
+Licensed under [Apache-2.0](LICENSE).

runrail-0.3.1/README.md

@@ -0,0 +1,142 @@
+# RunRail
+
+**A self-hosted workflow orchestrator for the scripts you already have.** Schedule Python scripts, Jupyter notebooks, SQL files, and shell commands with dependencies, retries, logs, backfills, and a fast web UI — without rewriting anything.
+
+[![CI](https://github.com/rutr-labs/runrail/actions/workflows/ci.yml/badge.svg)](https://github.com/rutr-labs/runrail/actions/workflows/ci.yml)
+[![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)
+![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)
+
+Most teams outgrow cron long before they need a distributed orchestrator. RunRail covers that middle ground: one process, one SQLite file, and your existing code running on a real scheduler with full observability. No DAG files to author, no decorators to add, no platform to operate.
+
+## Features
+
+- **Bring your own code** — point RunRail at a folder; scripts, notebooks, SQL, and shell commands run as-is in isolated subprocesses
+- **Dependency graphs with parallel execution** — independent tasks run concurrently; each task starts the moment its dependencies succeed
+- **Cron scheduling built for high frequency** — minute-level schedules with per-workflow concurrency limits, coalescing, and missed-run tolerance
+- **Managed Python environments** — declare pip requirements in the UI; RunRail builds and atomically swaps isolated virtualenvs, keeping the last working build on failure
+- **Failure webhooks with sane semantics** — one alert on the first failure, one on recovery; Slack and Teams incoming webhooks work as-is
+- **Auto-pause** — optionally disable a workflow after N consecutive failures instead of failing hundreds of times overnight
+- **Backfills and retries** — queue one run per date over a range; re-run any finished run with identical parameters in one click
+- **Workflows as code** — `runrail export` / `runrail apply` round-trip definitions through YAML for version control and code review
+- **Live dependency graph** — every run renders its task DAG with statuses updating in place as branches execute
+- **Live logs and artifacts** — streamed stdout/stderr with ANSI colors, search, tail-follow, timestamped notebook outputs, automatic retention cleanup
+- **A UI you'll actually use** — command palette (⌘K), Gantt timelines, activity heatmaps, a full-screen wallboard for the team TV, dark and light themes
+
+## Install
+
+RunRail ships as a single Python package with the web UI bundled in — **no Node required to run it**. Python 3.11+ is the only prerequisite.
+
+```bash
+pipx install runrail    # isolated, recommended — or: pip install runrail
+runrail serve
+```
+
+Then open **http://127.0.0.1:8080**. That's the whole setup.
+
+On first launch RunRail creates its database, logs, and artifact store automatically and starts the API, web UI, scheduler, and worker in a single process — no separate `init` step, no migrations to run by hand. Everything lives in a per-user application-data directory:
+
+| OS | Default location |
+|---|---|
+| macOS | `~/Library/Application Support/RunRail` |
+| Linux | `~/.local/share/RunRail` (honours `$XDG_DATA_HOME`) |
+| Windows | `%LOCALAPPDATA%\RunRail` |
+
+Set `RUNRAIL_HOME` to store everything somewhere else (e.g. `RUNRAIL_HOME=./.runrail` to keep data beside a project). For notebook tasks, install the extra: `pipx install "runrail[notebook]"`.
+
+From the UI, connect a project folder, add an environment, and build your first workflow.
+
+## Core concepts
+
+**Projects** point at directories where your code lives. **Environments** define how Python runs — a managed virtualenv built from declared pip requirements, an existing interpreter, or a Conda environment. **Workflows** group **tasks** into a dependency graph with an optional cron schedule. Environments resolve per task, falling back to the workflow default, then the project default; Python and notebook tasks never silently run on RunRail's own interpreter.
+
+The recommended environment setup is *Environments → New environment → Managed Python*: declare requirements such as `pandas==2.3.0`, and RunRail builds an isolated virtualenv, records the build log and Python version, and rebuilds atomically when the requirements change. A failed rebuild preserves the previous working runtime.
+
+## CLI
+
+```bash
+runrail run daily-refresh --param region=ca      # queue a manual run
+runrail backfill daily-refresh --from 2026-06-01 --to 2026-06-30
+runrail export -o workflows.yml                  # workflows as version-controllable YAML
+runrail apply workflows.yml                      # declarative upsert by name
+runrail import ~/old/.runrail                    # bring a previous setup into this home
+runrail cleanup --older-than-days 30 --dry-run   # prune old runs, logs, artifacts
+runrail status
+
+# run components as separate processes
+runrail api
+runrail scheduler
+runrail worker --concurrency 8
+```
+
+## Templating
+
+Task commands and paths are Jinja templates. Every run receives:
+
+| Variable | Meaning |
+|---|---|
+| `ds` | Logical date (`2026-07-08`); injected per-date during backfills |
+| `ts` / `ts_nodash` | Run timestamp, ISO and compact (`20260708T141005`) |
+| `run_id`, `task_run_id` | Identifiers for the current execution |
+| `project_root`, `artifacts_dir` | Resolved paths for the run |
+| *your parameters* | Run parameters and per-task parameters, merged |
+
+```bash
+python scripts/daily.py --date {{ ds }} --region {{ region }}
+```
+
+## Notifications
+
+Set `RUNRAIL_NOTIFY_WEBHOOK_URL` globally or a webhook per workflow. RunRail posts on the first failure after a success and again on recovery — never once per red run, so a broken two-minute schedule produces one alert rather than three hundred. The payload's `text` field renders directly in Slack and Microsoft Teams; structured fields are included for custom receivers. Pair with per-workflow auto-pause to stop repeat failures entirely.
+
+## Docker
+
+```bash
+docker compose up --build
+```
+
+Compose starts RunRail with PostgreSQL and persistent volumes on port 8080. Docker is optional for local use.
+
+## Architecture
+
+A FastAPI process serves the API and the prebuilt React UI. APScheduler acts purely as a clock, writing due runs to the database; while a run executes, the next scheduled iteration coalesces to a single queued run instead of piling up or being dropped. A bounded worker pool (`RUNRAIL_WORKER_CONCURRENCY`) claims queued runs atomically, so **different workflows execute concurrently** and a long job never blocks a frequent one. Runs of the *same* workflow are serialized by its `max_concurrent_runs` (default 1) so a slow run can't overlap its own next iteration. Within a run, the task graph executes with real parallelism (`RUNRAIL_TASK_PARALLELISM`): **independent tasks run concurrently**, each starting the moment its dependencies succeed, and every task is a subprocess with its own logs, timeout, and process-group cleanup.
+
+All of that concurrency happens on one machine (threads and subprocesses); RunRail does not yet distribute work across remote worker nodes.
+
+SQLite in WAL mode is the default store; set `RUNRAIL_DB_URL` for PostgreSQL. Logs live under `$RUNRAIL_HOME/logs/run_<id>/`, artifacts under `$RUNRAIL_HOME/artifacts/<id>/` with timestamped filenames so frequent runs and retries never collide.
+
+## Configuration
+
+| Variable | Default |
+|---|---|
+| `RUNRAIL_HOME` | Per-user data dir (see [Install](#install)) |
+| `RUNRAIL_DB_URL` | SQLite at `$RUNRAIL_HOME/runrail.db` |
+| `RUNRAIL_HOST` / `RUNRAIL_PORT` | `127.0.0.1` / `8080` |
+| `RUNRAIL_WORKER_CONCURRENCY` | `4` concurrent runs |
+| `RUNRAIL_TASK_PARALLELISM` | `4` concurrent tasks per run |
+| `RUNRAIL_RETENTION_DAYS` | unset; when set, finished runs older than this are deleted with their logs and artifacts |
+| `RUNRAIL_NOTIFY_WEBHOOK_URL` | unset; default failure/recovery webhook |
+| `RUNRAIL_BROWSE_ROOT` | user home; confines the UI file picker |
+
+Schedules evaluate in UTC; the UI displays them in your local timezone. For deployments reachable beyond localhost, set `RUNRAIL_BROWSE_ROOT` to a narrow directory and place the server behind authentication — RunRail does not yet ship its own.
+
+## Current limitations
+
+- Runs on a single machine — workflows and independent tasks run concurrently there, but work is not distributed across remote worker nodes yet
+- No built-in authentication or RBAC
+- SQL tasks execute against SQLite only
+- Secrets are plain environment variables
+
+## Roadmap
+
+Warehouse and database adapters, artifact previews, secrets management, API tokens and authentication, and richer run analytics. See open issues for details.
+
+## Development
+
+```bash
+pip install -e '.[dev]'
+pytest
+ruff check .
+cd frontend && npm install && npm run build   # writes the bundle into src/runrail/web/static
+```
+
+Licensed under [Apache-2.0](LICENSE).