hpc-runner 0.2.1__tar.gz → 0.3.0__tar.gz

hpc_runner-0.3.0/.github/workflows/docs.yml

@@ -0,0 +1,56 @@
+name: Docs (GitHub Pages)
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: pages
+  cancel-in-progress: false
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+
+      - name: Install dependencies
+        run: pip install -e ".[dev]"
+
+      - name: Build docs
+        run: make -C docs html SPHINXOPTS="-W --keep-going"
+
+      - name: Upload Pages artifact
+        if: github.event_name != 'pull_request'
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: docs/build/html
+
+  deploy:
+    if: github.event_name != 'pull_request'
+    needs: build
+    runs-on: ubuntu-latest
+    permissions:
+      pages: write
+      id-token: write
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v4
+

{hpc_runner-0.2.1 → hpc_runner-0.3.0}/CLAUDE.md

@@ -36,7 +36,7 @@ hpc --scheduler sge run --cpu 4 --mem 8G "python script.py"
 - **JobResult/ArrayJobResult** - Returned from submission, provides status polling and output access
 - **JobStatus** - Unified enum: PENDING, RUNNING, COMPLETED, FAILED, CANCELLED, TIMEOUT, UNKNOWN
 - **ResourceSet** - Collection of named resources (gpu, licenses, etc.)
-- **HPCConfig** - TOML-based config with hierarchy: ./hpc-tools.toml > pyproject.toml > git root > ~/.config > package defaults
+- **HPCConfig** - TOML-based config with hierarchy: ./hpc-runner.toml > pyproject.toml > git root > ~/.config > package defaults
 
 ### Scheduler System (`src/hpc_runner/schedulers/`)
 

{hpc_runner-0.2.1 → hpc_runner-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hpc-runner
-Version: 0.2.1
+Version: 0.3.0
 Summary: Unified HPC job submission across multiple schedulers
 Project-URL: Homepage, https://github.com/sjalloq/hpc-runner
 Project-URL: Repository, https://github.com/sjalloq/hpc-runner
@@ -26,19 +26,25 @@ Requires-Dist: textual>=6.11
 Requires-Dist: tomli>=2.0; python_version < '3.11'
 Provides-Extra: all
 Requires-Dist: build; extra == 'all'
+Requires-Dist: furo>=2024.0.0; extra == 'all'
 Requires-Dist: hatch-vcs; extra == 'all'
 Requires-Dist: mypy; extra == 'all'
+Requires-Dist: pytest-asyncio; extra == 'all'
 Requires-Dist: pytest-cov; extra == 'all'
 Requires-Dist: pytest>=7.0; extra == 'all'
 Requires-Dist: ruff; extra == 'all'
+Requires-Dist: sphinx>=7.0; extra == 'all'
 Requires-Dist: twine; extra == 'all'
 Provides-Extra: dev
 Requires-Dist: build; extra == 'dev'
+Requires-Dist: furo>=2024.0.0; extra == 'dev'
 Requires-Dist: hatch-vcs; extra == 'dev'
 Requires-Dist: mypy; extra == 'dev'
+Requires-Dist: pytest-asyncio; extra == 'dev'
 Requires-Dist: pytest-cov; extra == 'dev'
 Requires-Dist: pytest>=7.0; extra == 'dev'
 Requires-Dist: ruff; extra == 'dev'
+Requires-Dist: sphinx>=7.0; extra == 'dev'
 Requires-Dist: twine; extra == 'dev'
 Description-Content-Type: text/markdown
 
@@ -173,10 +179,10 @@ p.wait()
 hpc-runner uses TOML configuration files. Location priority:
 
 1. `--config /path/to/config.toml`
-2. `./hpc-tools.toml`
-3. `./pyproject.toml` under `[tool.hpc-tools]`
-4. Git repository root `hpc-tools.toml`
-5. `~/.config/hpc-tools/config.toml`
+2. `./hpc-runner.toml`
+3. `./pyproject.toml` under `[tool.hpc-runner]`
+4. Git repository root `hpc-runner.toml`
+5. `~/.config/hpc-runner/config.toml`
 6. Package defaults
 
 ### Example Configuration
@@ -275,11 +281,6 @@ ruff check src/hpc_runner
 ruff format src/hpc_runner
 ```
 
-## Documentation
-
-- [Programmatic API Reference](docs/programmatic_api.md)
-- [TUI Styling Guide](docs/TEXTUAL_STYLING_COOKBOOK.md)
-
 ## License
 
 MIT License - see LICENSE file for details.

{hpc_runner-0.2.1 → hpc_runner-0.3.0}/README.md

@@ -129,10 +129,10 @@ p.wait()
 hpc-runner uses TOML configuration files. Location priority:
 
 1. `--config /path/to/config.toml`
-2. `./hpc-tools.toml`
-3. `./pyproject.toml` under `[tool.hpc-tools]`
-4. Git repository root `hpc-tools.toml`
-5. `~/.config/hpc-tools/config.toml`
+2. `./hpc-runner.toml`
+3. `./pyproject.toml` under `[tool.hpc-runner]`
+4. Git repository root `hpc-runner.toml`
+5. `~/.config/hpc-runner/config.toml`
 6. Package defaults
 
 ### Example Configuration
@@ -231,11 +231,6 @@ ruff check src/hpc_runner
 ruff format src/hpc_runner
 ```
 
-## Documentation
-
-- [Programmatic API Reference](docs/programmatic_api.md)
-- [TUI Styling Guide](docs/TEXTUAL_STYLING_COOKBOOK.md)
-
 ## License
 
 MIT License - see LICENSE file for details.

{hpc_runner-0.2.1 → hpc_runner-0.3.0}/defaults/config.toml

@@ -1,4 +1,4 @@
-# hpc-tools default configuration
+# hpc-runner default configuration
 
 [defaults]
 scheduler = "auto"

hpc_runner-0.3.0/docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

hpc_runner-0.3.0/docs/source/_static/.gitkeep

@@ -0,0 +1 @@
+

hpc_runner-0.3.0/docs/source/_templates/.gitkeep

@@ -0,0 +1 @@
+

hpc_runner-0.3.0/docs/source/cli.rst

@@ -0,0 +1,79 @@
+CLI
+===
+
+The primary entry point is the ``hpc`` command.
+
+.. note::
+
+   This page focuses on the stable “user facing” flags. Any unknown options that
+   start with ``-`` are passed through to the underlying scheduler (e.g. SGE
+   ``qsub`` flags like ``-q`` and ``-l``).
+
+
+Global options
+--------------
+
+These options come before the subcommand:
+
+- ``--config PATH``: use an explicit config file (bypasses discovery)
+- ``--scheduler NAME``: force a scheduler (``sge``, ``slurm``, ``pbs``, ``local``)
+- ``--verbose``: extra debug output
+
+
+``hpc run`` (submit)
+--------------------
+
+Submit a job to the scheduler.
+
+Common options:
+
+- ``--job-name TEXT``
+- ``--cpu N``
+- ``--mem TEXT`` (e.g. ``16G``)
+- ``--time TEXT`` (e.g. ``4:00:00``)
+- ``--queue TEXT`` (SGE queue)
+- ``--directory PATH`` (working dir)
+- ``--module TEXT`` (repeatable)
+- ``--job-type TEXT`` (named profile from config)
+- ``--array TEXT`` (e.g. ``1-100``)
+- ``--depend TEXT``
+- ``--interactive`` (SGE: qrsh)
+- ``--dry-run`` (render, don’t submit)
+- ``--wait`` (wait for completion)
+- ``--keep-script`` (debug: keep generated script)
+
+Examples:
+
+.. code-block:: bash
+
+   hpc run --cpu 4 --mem 16G --time 2:00:00 "python train.py"
+   hpc run --job-type gpu "python train.py"
+   hpc run -q gpu.q -l gpu=1 "python train.py"   # scheduler passthrough
+
+
+``submit`` alias
+----------------
+
+There is also a convenience console script called ``submit`` which behaves like:
+
+.. code-block:: bash
+
+   submit ...  ==  hpc run ...
+
+It still supports global flags:
+
+.. code-block:: bash
+
+   submit --config ./hpc-runner.toml --scheduler sge --dry-run "python train.py"
+
+
+Other commands
+--------------
+
+- ``hpc status JOB_ID``: show job status
+- ``hpc cancel JOB_ID``: cancel a job
+- ``hpc monitor``: launch the TUI job monitor
+- ``hpc config show``: print the active config file contents
+- ``hpc config path``: print the active config file path
+- ``hpc config init [--global]``: create a starter config
+

hpc_runner-0.3.0/docs/source/conf.py

@@ -0,0 +1,41 @@
+"""Sphinx documentation configuration for hpc-runner."""
+
+from __future__ import annotations
+
+import os
+import sys
+from pathlib import Path
+
+# -- Project information -----------------------------------------------------
+
+project = "HPC Runner"
+copyright = "2026, Shareef Jalloq"
+author = "Shareef Jalloq"
+
+# -- General configuration ---------------------------------------------------
+
+# Make package importable for autodoc (src layout)
+REPO_ROOT = Path(__file__).resolve().parents[2]
+sys.path.insert(0, str(REPO_ROOT / "src"))
+
+extensions = [
+    "sphinx.ext.autodoc",
+    "sphinx.ext.napoleon",
+    "sphinx.ext.autosectionlabel",
+    "sphinx.ext.viewcode",
+]
+
+templates_path = ["_templates"]
+exclude_patterns: list[str] = []
+
+# Avoid autosectionlabel collisions across pages
+autosectionlabel_prefix_document = True
+
+# Honor SOURCE_DATE_EPOCH when set (reproducible builds)
+if os.environ.get("SOURCE_DATE_EPOCH"):
+    today_fmt = "%Y-%m-%d"
+
+# -- Options for HTML output -------------------------------------------------
+
+html_theme = "furo"
+html_static_path = ["_static"]

hpc_runner-0.3.0/docs/source/configuration.rst

@@ -0,0 +1,144 @@
+:orphan:
+
+Configuration
+=============
+
+This page describes the TOML configuration format used by **hpc-runner** and how
+configuration values map to scheduler flags (with a focus on SGE).
+
+
+Where configuration is loaded from
+----------------------------------------
+
+Configuration can be provided via a standalone TOML file or embedded inside
+``pyproject.toml``.
+
+Discovery/precedence (highest to lowest):
+
+1. ``./hpc-runner.toml``
+2. ``./pyproject.toml`` under ``[tool.hpc-runner]``
+3. ``<git root>/hpc-runner.toml``
+4. ``~/.config/hpc-runner/config.toml``
+5. Package defaults
+
+You can always bypass discovery by providing ``--config /path/to/file.toml``.
+
+
+Top-level sections
+------------------
+
+Configuration supports four top-level namespaces:
+
+- ``[defaults]``: baseline job settings applied to all jobs
+- ``[tools.<name>]``: overrides keyed by tool name (e.g., ``python``, ``make``)
+- ``[types.<name>]``: named job profiles (e.g., ``gpu``, ``interactive``)
+- ``[schedulers.<name>]``: scheduler-specific behavior (e.g., SGE resource names)
+
+
+How jobs pick up config
+-----------------------
+
+When creating a job from config, hpc-runner merges values in this order:
+
+1. start from ``[defaults]``
+2. merge either ``[types.<name>]`` **or** ``[tools.<name>]`` (types take precedence)
+
+Then CLI options override whatever came from config.
+
+.. note::
+
+   List values use a "merge by union" strategy.
+   If you need to *replace* a list instead of merging, use a leading ``"-"`` entry
+   to reset it, e.g. ``modules = ["-", "python/3.11"]``.
+
+
+SGE mapping notes
+-----------------
+
+- ``queue`` maps to SGE ``qsub -q <queue>``.
+- ``cpu`` maps to SGE parallel environment slots: ``qsub -pe <parallel_environment> <cpu>``.
+- ``mem`` and ``time`` map to SGE hard resources via ``-l <resource>=<value>``, where the
+  resource names are configured under ``[schedulers.sge]``.
+
+
+Example: fully populated config (standalone file)
+-------------------------------------------------
+
+Save as ``hpc-runner.toml`` (or ``~/.config/hpc-runner/config.toml``):
+
+.. code-block:: toml
+
+   [defaults]
+   scheduler = "auto"          # auto|sge|slurm|pbs|local
+   name = "job"
+   cpu = 1
+   mem = "4G"
+   time = "1:00:00"
+   queue = "batch.q"           # SGE default queue
+   priority = 0
+   workdir = "."
+   shell = "/bin/bash"
+   use_cwd = true
+   inherit_env = true
+
+   stdout = "hpc.%N.%J.out"
+   stderr = ""                 # empty means "unset"
+
+   modules = ["gcc/12.2", "python/3.11"]
+   modules_path = []
+
+   raw_args = []
+   sge_args = []
+
+   resources = [
+     { name = "scratch", value = "20G" }
+   ]
+
+   [schedulers.sge]
+   parallel_environment = "smp"
+   memory_resource = "mem_free"
+   time_resource = "h_rt"
+
+   merge_output = true
+   purge_modules = false
+   silent_modules = false
+   module_init_script = ""
+   expand_makeflags = true
+   unset_vars = ["https_proxy", "http_proxy"]
+
+   [tools.python]
+   cpu = 4
+   mem = "16G"
+   time = "4:00:00"
+   queue = "short.q"
+   modules = ["-", "python/3.11"]  # replace list rather than union-merge
+   resources = [
+     { name = "tmpfs", value = "8G" }
+   ]
+
+   [types.interactive]
+   queue = "interactive.q"
+   time = "8:00:00"
+   cpu = 2
+   mem = "8G"
+
+   [types.gpu]
+   queue = "gpu.q"
+   cpu = 8
+   mem = "64G"
+   time = "12:00:00"
+   resources = [
+     { name = "gpu", value = 1 }
+   ]
+
+
+Embedding in pyproject.toml
+---------------------------
+
+To embed the same config in ``pyproject.toml``, nest the same keys under:
+
+.. code-block:: toml
+
+   [tool.hpc-runner]
+   # ... same content as the standalone file ...
+

hpc_runner-0.3.0/docs/source/getting_started.rst

@@ -0,0 +1,78 @@
+Getting Started
+===============
+
+This guide gets you from “installed” to a first successful submission.
+
+
+Install
+-------
+
+In a virtual environment:
+
+.. code-block:: bash
+
+   pip install hpc-runner
+
+
+Quick start: submit a job
+-------------------------
+
+Submit a batch job:
+
+.. code-block:: bash
+
+   hpc run "python -c 'print(\"hello\")'"
+
+Show what would be submitted without actually submitting:
+
+.. code-block:: bash
+
+   hpc run --dry-run "python train.py"
+
+Run interactively (SGE: ``qrsh``):
+
+.. code-block:: bash
+
+   hpc run --interactive "bash"
+
+
+Pick a scheduler
+----------------
+
+By default, hpc-runner tries to auto-detect the scheduler. You can force it:
+
+.. code-block:: bash
+
+   hpc --scheduler sge run "python train.py"
+
+Or use the environment variable:
+
+.. code-block:: bash
+
+   export HPC_SCHEDULER=sge
+
+
+First config
+------------
+
+Create a project-local config (safe to commit):
+
+.. code-block:: bash
+
+   hpc config init
+
+Create a user config:
+
+.. code-block:: bash
+
+   hpc config init --global
+
+Then open the file and set your site defaults, e.g. a default SGE queue:
+
+.. code-block:: toml
+
+   [defaults]
+   queue = "batch.q"
+
+See :doc:`configuration` for the full configuration format and examples.
+

hpc_runner-0.3.0/docs/source/index.rst

@@ -0,0 +1,19 @@
+HPC Runner
+==========
+
+.. note::
+
+   **Overview (write your why/what here):**
+   Replace this note with your project perspective (motivation, principles,
+   and what “good” looks like). The rest of the docs are organized below.
+
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   getting_started
+   configuration
+   cli
+   sge
+   programmatic_api