slurming 0.2.3a1__tar.gz

slurming-0.2.3a1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Yuxiang Luo
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

slurming-0.2.3a1/PKG-INFO

@@ -0,0 +1,257 @@
+Metadata-Version: 2.4
+Name: slurming
+Version: 0.2.3a1
+Summary: 
+License: MIT
+License-File: LICENSE
+Author: Yuxiang Luo
+Author-email: yuxiang.lll@outlook.com
+Requires-Python: >=3.8,<4.0
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: pandas (>=2.2.2,<3.0.0) ; python_version >= "3.9" and python_version < "4.0"
+Description-Content-Type: text/markdown
+
+# SlurmUtils
+
+A Python utility library for generating and managing [Slurm](https://slurm.schedmd.com/) HPC job scripts.
+
+**Version:** 0.2.3
+**Author:** Yuxiang Luo ([luo.929@osu.edu](mailto:luo.929@osu.edu))
+**License:** GPL-2.0
+**Affiliation:** Center for Weldability, The Ohio State University
+
+---
+
+## Installation
+
+```bash
+pip install slurmutils
+```
+
+Or with Poetry:
+
+```bash
+poetry add slurmutils
+```
+
+**Requirements:** Python ^3.8, pandas ^2.2.2 (Python ^3.9+)
+
+---
+
+## Package Structure
+
+```
+slurmutils/
+├── Slurm/
+│   ├── slurm.py        # Query Slurm job queue
+│   └── shellUtils.py   # Shell script generation helpers
+├── SlurmJob/
+│   └── job.py          # SlurmJob class (OOP job builder)
+└── DecentJob/
+    └── metaJob.py      # DataFrame-based job metadata utilities
+```
+
+---
+
+## Modules
+
+### `slurmutils.Slurm`
+
+#### `get_job_dict(cluster_name)`
+
+Query the Slurm queue and return job status dicts.
+
+```python
+from slurmutils.Slurm.slurm import get_job_dict
+
+jobs, running_jobs, pending_jobs = get_job_dict("myCluster")
+```
+
+Returns:
+- `jobs` — `{cluster: {job_name: {state, job_id}}}` for all visible jobs
+- `running_jobs` — `{job_name: {state, job_id}}` filtered to `RUNNING`
+- `pending_jobs` — `{job_name: {state, job_id}}` filtered to `PENDING` on the specified cluster
+
+Tries `squeue --clusters=all` first; falls back to local cluster on error.
+
+---
+
+#### `make_shell_script(...)`
+
+Generate an sbatch shell script file.
+
+```python
+from slurmutils.Slurm.shellUtils import make_shell_script
+from pathlib import Path
+
+make_shell_script(
+    account="PAS2138",
+    script_path=Path("run_job.sh"),
+    content=["python train.py"],
+    hours=4,
+    cores=28,
+    modules=["python/3.10"],
+    memory=64,          # GB
+    gpus=1,
+    jobname="my_train",
+    sbatch_log=Path("logs/job.log"),
+)
+```
+
+Key parameters:
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `account` | `str` | — | Slurm account to charge |
+| `script_path` | `Path` | — | Output `.sh` file path |
+| `content` | `List[str]` | — | Shell commands to run |
+| `hours` / `minutes` / `seconds` | `int` | `2` / `0` / `0` | Wall time |
+| `cores` | `int` | `28` | CPUs per task |
+| `gpus` | `int` | `0` | Number of GPUs |
+| `memory` | `int` | `0` | Memory in GB (0 = unset) |
+| `modules` | `List[str]` | `[]` | Modules to load |
+| `module_profie` | `str\|None` | `None` | Custom module profile path |
+| `python_env` | `str` | `""` | Path to Python virtualenv activate script |
+| `env_vars` | `Dict[str,str]` | `{}` | Environment variables to export |
+| `aliases` | `Dict[str,str]` | `{}` | Shell aliases to define |
+| `paths` | `List[str]` | `[]` | Extra paths to append to `$PATH` |
+| `notifies` | `List[str]` | `["FAIL"]` | Slurm mail-type events |
+| `sbatch_args` | `Dict` | `{}` | Extra `#SBATCH` key-value directives |
+| `license` | `Dict[str,int]` | `{}` | Software licenses (OSC format) |
+| `set_flag` | `str\|None` | `"x"` | Bash `set -<flag>` option |
+| `interactive` | `bool` | `False` | Use `#!/bin/bash -i` |
+| `bashinit` | `List[str]` | `[]` | Lines inserted before env setup |
+
+---
+
+#### `make_command(head, params, params1_dict, params2_dict, stdout_redirect, connection)`
+
+Build a shell command string from parts.
+
+```python
+from slurmutils.Slurm.shellUtils import make_command
+
+cmd = make_command(
+    "mpirun",
+    params=["./solver"],
+    params1_dict={"n": 28},
+    params2_dict={"bind-to": "core"},
+)
+# => "mpirun -n=28 --bind-to=core ./solver"
+```
+
+---
+
+#### `make_if_statement(if_st, elif_sts, else_st)`
+
+Build a bash `if/elif/else/fi` block as a list of strings.
+
+```python
+from slurmutils.Slurm.shellUtils import make_if_statement
+
+lines = make_if_statement(
+    if_st=(["$ret -eq 0"], ["echo success"]),
+    else_st=["echo failure"],
+)
+```
+
+---
+
+### `slurmutils.SlurmJob`
+
+#### `SlurmJob`
+
+OOP interface for constructing and writing sbatch scripts.
+
+```python
+from slurmutils.SlurmJob.job import SlurmJob
+from pathlib import Path
+
+job = SlurmJob(
+    account="PAS2138",
+    content=["python simulate.py"],
+    licenses={},
+    modules=["python/3.10"],
+    env_vars={},
+    notify_email=["FAIL"],
+    aliases={},
+    paths=[],
+    sbatch_args={},
+    output_storage=[],
+    hours=8,
+    cpus_per_task=28,
+    memory=128,
+    job_name="simulation",
+    partition="gpu",
+    gpus=1,
+    python_home=Path("/home/user/.venv"),
+    working_dir=Path("/scratch/user/project"),
+)
+
+job.append(["echo done"])
+job.write(job_name="simulation", script_path=Path("sim.sh"))
+```
+
+Key methods:
+
+| Method | Description |
+|--------|-------------|
+| `append(content)` | Append lines to the job script |
+| `prepend(content)` | Prepend lines to the job script |
+| `write(job_name, script_path, log_file, delete_log_on_completion, delete_script_on_completion)` | Write the sbatch script to disk |
+
+---
+
+### `slurmutils.DecentJob`
+
+Utilities for managing collections of jobs tracked in a pandas DataFrame.
+
+#### `find_job_by_params(condition_dict, df, exception, is_exact)`
+
+Query a DataFrame for jobs matching a parameter dict.
+
+```python
+from slurmutils.DecentJob.metaJob import find_job_by_params
+
+result = find_job_by_params(
+    condition_dict={"solver": "myFoam", "cores": 28},
+    df=job_df,
+    is_exact=True,
+)
+```
+
+#### `query_dataframe(condition_dict, df)`
+
+Low-level DataFrame query builder.
+
+```python
+from slurmutils.DecentJob.metaJob import query_dataframe
+
+result, query_str = query_dataframe({"cores": 28, "solver": "myFoam"}, df)
+```
+
+#### `migrate_static(new_job_df, static_dir)`
+
+Migrate a directory of static job folders to new parameter-based indices by matching `case.json` metadata against a new DataFrame layout.
+
+```python
+from slurmutils.DecentJob.metaJob import migrate_static
+from pathlib import Path
+
+migrate_static(new_job_df=job_df, static_dir=Path("/scratch/cases"))
+```
+
+---
+
+## License
+
+GPL-2.0. See [LICENSE](LICENSE) for details.
+

slurming-0.2.3a1/README.md

@@ -0,0 +1,235 @@
+# SlurmUtils
+
+A Python utility library for generating and managing [Slurm](https://slurm.schedmd.com/) HPC job scripts.
+
+**Version:** 0.2.3
+**Author:** Yuxiang Luo ([luo.929@osu.edu](mailto:luo.929@osu.edu))
+**License:** GPL-2.0
+**Affiliation:** Center for Weldability, The Ohio State University
+
+---
+
+## Installation
+
+```bash
+pip install slurmutils
+```
+
+Or with Poetry:
+
+```bash
+poetry add slurmutils
+```
+
+**Requirements:** Python ^3.8, pandas ^2.2.2 (Python ^3.9+)
+
+---
+
+## Package Structure
+
+```
+slurmutils/
+├── Slurm/
+│   ├── slurm.py        # Query Slurm job queue
+│   └── shellUtils.py   # Shell script generation helpers
+├── SlurmJob/
+│   └── job.py          # SlurmJob class (OOP job builder)
+└── DecentJob/
+    └── metaJob.py      # DataFrame-based job metadata utilities
+```
+
+---
+
+## Modules
+
+### `slurmutils.Slurm`
+
+#### `get_job_dict(cluster_name)`
+
+Query the Slurm queue and return job status dicts.
+
+```python
+from slurmutils.Slurm.slurm import get_job_dict
+
+jobs, running_jobs, pending_jobs = get_job_dict("myCluster")
+```
+
+Returns:
+- `jobs` — `{cluster: {job_name: {state, job_id}}}` for all visible jobs
+- `running_jobs` — `{job_name: {state, job_id}}` filtered to `RUNNING`
+- `pending_jobs` — `{job_name: {state, job_id}}` filtered to `PENDING` on the specified cluster
+
+Tries `squeue --clusters=all` first; falls back to local cluster on error.
+
+---
+
+#### `make_shell_script(...)`
+
+Generate an sbatch shell script file.
+
+```python
+from slurmutils.Slurm.shellUtils import make_shell_script
+from pathlib import Path
+
+make_shell_script(
+    account="PAS2138",
+    script_path=Path("run_job.sh"),
+    content=["python train.py"],
+    hours=4,
+    cores=28,
+    modules=["python/3.10"],
+    memory=64,          # GB
+    gpus=1,
+    jobname="my_train",
+    sbatch_log=Path("logs/job.log"),
+)
+```
+
+Key parameters:
+
+| Parameter | Type | Default | Description |
+|-----------|------|---------|-------------|
+| `account` | `str` | — | Slurm account to charge |
+| `script_path` | `Path` | — | Output `.sh` file path |
+| `content` | `List[str]` | — | Shell commands to run |
+| `hours` / `minutes` / `seconds` | `int` | `2` / `0` / `0` | Wall time |
+| `cores` | `int` | `28` | CPUs per task |
+| `gpus` | `int` | `0` | Number of GPUs |
+| `memory` | `int` | `0` | Memory in GB (0 = unset) |
+| `modules` | `List[str]` | `[]` | Modules to load |
+| `module_profie` | `str\|None` | `None` | Custom module profile path |
+| `python_env` | `str` | `""` | Path to Python virtualenv activate script |
+| `env_vars` | `Dict[str,str]` | `{}` | Environment variables to export |
+| `aliases` | `Dict[str,str]` | `{}` | Shell aliases to define |
+| `paths` | `List[str]` | `[]` | Extra paths to append to `$PATH` |
+| `notifies` | `List[str]` | `["FAIL"]` | Slurm mail-type events |
+| `sbatch_args` | `Dict` | `{}` | Extra `#SBATCH` key-value directives |
+| `license` | `Dict[str,int]` | `{}` | Software licenses (OSC format) |
+| `set_flag` | `str\|None` | `"x"` | Bash `set -<flag>` option |
+| `interactive` | `bool` | `False` | Use `#!/bin/bash -i` |
+| `bashinit` | `List[str]` | `[]` | Lines inserted before env setup |
+
+---
+
+#### `make_command(head, params, params1_dict, params2_dict, stdout_redirect, connection)`
+
+Build a shell command string from parts.
+
+```python
+from slurmutils.Slurm.shellUtils import make_command
+
+cmd = make_command(
+    "mpirun",
+    params=["./solver"],
+    params1_dict={"n": 28},
+    params2_dict={"bind-to": "core"},
+)
+# => "mpirun -n=28 --bind-to=core ./solver"
+```
+
+---
+
+#### `make_if_statement(if_st, elif_sts, else_st)`
+
+Build a bash `if/elif/else/fi` block as a list of strings.
+
+```python
+from slurmutils.Slurm.shellUtils import make_if_statement
+
+lines = make_if_statement(
+    if_st=(["$ret -eq 0"], ["echo success"]),
+    else_st=["echo failure"],
+)
+```
+
+---
+
+### `slurmutils.SlurmJob`
+
+#### `SlurmJob`
+
+OOP interface for constructing and writing sbatch scripts.
+
+```python
+from slurmutils.SlurmJob.job import SlurmJob
+from pathlib import Path
+
+job = SlurmJob(
+    account="PAS2138",
+    content=["python simulate.py"],
+    licenses={},
+    modules=["python/3.10"],
+    env_vars={},
+    notify_email=["FAIL"],
+    aliases={},
+    paths=[],
+    sbatch_args={},
+    output_storage=[],
+    hours=8,
+    cpus_per_task=28,
+    memory=128,
+    job_name="simulation",
+    partition="gpu",
+    gpus=1,
+    python_home=Path("/home/user/.venv"),
+    working_dir=Path("/scratch/user/project"),
+)
+
+job.append(["echo done"])
+job.write(job_name="simulation", script_path=Path("sim.sh"))
+```
+
+Key methods:
+
+| Method | Description |
+|--------|-------------|
+| `append(content)` | Append lines to the job script |
+| `prepend(content)` | Prepend lines to the job script |
+| `write(job_name, script_path, log_file, delete_log_on_completion, delete_script_on_completion)` | Write the sbatch script to disk |
+
+---
+
+### `slurmutils.DecentJob`
+
+Utilities for managing collections of jobs tracked in a pandas DataFrame.
+
+#### `find_job_by_params(condition_dict, df, exception, is_exact)`
+
+Query a DataFrame for jobs matching a parameter dict.
+
+```python
+from slurmutils.DecentJob.metaJob import find_job_by_params
+
+result = find_job_by_params(
+    condition_dict={"solver": "myFoam", "cores": 28},
+    df=job_df,
+    is_exact=True,
+)
+```
+
+#### `query_dataframe(condition_dict, df)`
+
+Low-level DataFrame query builder.
+
+```python
+from slurmutils.DecentJob.metaJob import query_dataframe
+
+result, query_str = query_dataframe({"cores": 28, "solver": "myFoam"}, df)
+```
+
+#### `migrate_static(new_job_df, static_dir)`
+
+Migrate a directory of static job folders to new parameter-based indices by matching `case.json` metadata against a new DataFrame layout.
+
+```python
+from slurmutils.DecentJob.metaJob import migrate_static
+from pathlib import Path
+
+migrate_static(new_job_df=job_df, static_dir=Path("/scratch/cases"))
+```
+
+---
+
+## License
+
+GPL-2.0. See [LICENSE](LICENSE) for details.

slurming-0.2.3a1/pyproject.toml

@@ -0,0 +1,23 @@
+[tool.poetry]
+name = "slurming"
+version = "0.2.3a1"
+description = ""
+authors = ["Yuxiang Luo <yuxiang.lll@outlook.com>"]
+license = "MIT"
+readme = "README.md"
+
+[tool.poetry.dependencies]
+python = "^3.8"
+pandas =[
+    {version="^2.2.2",python="^3.9"}
+]
+
+
+[tool.poetry.group.dev.dependencies]
+build = "^1.2.1"
+yapf = "^0.40.2"
+
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"

slurming-0.2.3a1/slurming/DecentJob/metaJob.py

@@ -0,0 +1,101 @@
+
+
+import json
+from typing import Dict, List, Union
+import pandas as pd
+from pathlib import Path
+
+def find_job_by_params(
+    condition_dict: Dict,
+    df: pd.DataFrame,
+    exception: List[str] = [],
+    is_exact: bool = False,
+):
+
+    result, quest = query_dataframe(
+        {
+            key: val
+            for key, val in condition_dict.items() if key not in exception
+        }, df)
+    if (len(result) == 0):
+        return {}
+
+    if (is_exact):
+        assert len(result) == 1, f"Query return {result} on quest {quest}"
+
+        result = result.to_dict(orient="index")
+        for key, val in result.items():
+            result[key]["Index"] = key
+
+        result_dict: Dict = result[list(result.keys())[0]]
+    else:
+        result_dict = result.to_dict(orient="index")
+
+    return result_dict
+
+
+def migrate_static(new_job_df: pd.DataFrame, static_dir: Path):
+
+    for dir_path in static_dir.iterdir():
+        if (dir_path.is_file()):
+            continue
+        with open(dir_path / "case.json", 'r') as fin:
+            dir_dict = json.load(fin)
+
+        new_keys = [key for key in new_job_df.columns if key not in dir_dict]
+
+        multi_val_keys = [
+            key for key in new_keys if len(new_job_df[key].unique()) != 1
+        ]
+        assert len(
+            multi_val_keys
+        ) == 0, f"{[[key , new_job_df[key].unique().tolist()] for key in multi_val_keys]}"
+
+        if (len(new_keys) > 0):
+            print(f"{dir_path/ 'case.json'} Found new keys {new_keys}")
+            for key in new_keys:
+                dir_dict[key] = new_job_df[key].unique().tolist()[0]
+            print(dir_dict)
+            input()
+            with open(dir_path / "case.json", 'w') as fout:
+                json.dump(dir_dict, fout)
+
+        new_job_dict = find_job_by_params(
+            dir_dict,
+            new_job_df,
+            is_exact=True,
+        )
+        if (len(new_job_dict) == 0):
+            dir_path.rename(
+                dir_path.parent /
+                f"{dir_path.name}_empty".replace("_empty_empty", "_empty"))
+        else:
+            dir_path.rename(dir_path.parent / f"{new_job_dict['Index']}_ren")
+            print(
+                f"Migrating {dir_path} -> {dir_path.parent/new_job_dict['Index']}"
+            )
+
+    for dir_path in static_dir.glob("*_ren"):
+
+        dir_path.rename(dir_path.parent / dir_path.name.replace("_ren", ""))
+
+
+def query_dataframe(condition_dict: Dict, df: pd.DataFrame):
+    conditions = []
+
+    for key, val in condition_dict.items():
+        # print(key, val, type(val))
+        if (type(val) in [int, float, bool]):
+            conditions.append(f"{key}=={val}")
+        else:
+            conditions.append(f"{key}==\"{val}\"")
+
+    # print(conditions)
+    # input()
+    quest = " and ".join(conditions)
+    # print(quest)
+    # input()
+
+    result = df.query(quest)
+
+    return result, quest

slurming-0.2.3a1/slurming/Slurm/shellUtils.py

@@ -0,0 +1,187 @@
+
+
+import json
+from pathlib import Path
+from typing import Dict, List, Union, Tuple
+import os
+import stat
+
+LICENSE_USAGE = {
+    28: 70,
+    14: 50,
+    8: 40,
+    7: 48,
+    1: 12,
+}
+CONCURRENCY_CORE_USAGE = {
+    28: 28,
+    14: 28,
+    8: 8,
+    7: 28,
+    1: 8,
+}
+TIME_CORE_USAGE = {28: 8, 14: 13, 8: 13, 7: 13, 1: 84}
+
+
+def make_shell_script(
+    account: str,
+    script_path: Path,
+    content: List[str],
+    hours=2,
+    minutes=0,
+    seconds=0,
+    license: Dict[str, int] = {},
+    cores: int = 28,
+    module_profie: Union[str, None] = None,
+    modules: List[str] = [],
+    python_env: str = "",
+    gpus: int = 0,
+    env_vars: Dict = {},
+    set_flag: Union[str, None] = "x",
+    sbatch_log: Union[Path, None] = None,
+    pre_set_content: List[str] = [],
+    aliases: Dict[str, str] = {},
+    paths: List[str] = [],
+    chmod: bool = True,
+    notifies: List[str] = ["FAIL"],
+    memory: int = 0,
+    sbatch_args: Dict = {},
+    jobname: str = "",
+    interactive: bool = False,
+    bashinit: List[str] = [],
+):
+    """_summary_
+
+    Args:
+        script_path (Path): The path of new shell script to be generated
+        content (List[str]): The content of shell script, in a list of strings.
+        hours (int, optional): Allown hours to run for this job. Defaults to 2.
+        minutes (int, optional): Allown hours to run for this job. Defaults to 0.
+        license (Dict[str,int], optional): The software licences to be used for this job, keys and values are the software name and number of tokens repectively. Defaults to {}.
+        cores (int, optional): number of cores to be used for this job. Defaults to 28 (owens).
+        account (str, optional): account to charge from for this job. Defaults to 'PAS2138'.
+        module_profie (str,optional): Local module profile files to use for modules.
+        modules (List[str], optional): modules to be activated for this job. Defaults to [].
+        python_env (str, optional): python environment to be activated for this job. Defaults to "". Example for python -m venv environments: "source .pythonenv/bin/activate"
+        gpus (int, optional): number of GPUs for this job. Defaults to 0.
+        env_vars (Dict, optional): Diction of environment variable, name and content. Defaults to {}.
+        set_flag (Union[str, None], optional): flat for "set" command. Defaults to "x".
+        sbatch_log (Union[Path, None], optional): the log file for this job. Defaults to None.
+    """
+
+    # cores = max(cores, 8)
+    env_var_decls = []
+    for vname, vval in env_vars.items():
+        env_var_decls.extend([f"{vname}={vval}", f"export {vname}"])
+    alias_var_decls = []
+    for vname, vval in aliases.items():
+        alias_var_decls.extend([f"alias {vname}={vval}"])
+
+    path_export = [f"PATH=$PATH:{':'.join(paths)}", f"export PATH"
+                   ] if len(paths) > 0 else []
+
+    sbatch_log_file = "output/%j.log" if (sbatch_log is None) else sbatch_log
+
+    shell_script_head = [
+        "#!/bin/bash" if not interactive else "#!/bin/bash -i",
+        f"#SBATCH --job-name={jobname}" if len(jobname) > 0 else "",
+        f"#SBATCH --time={hours:02d}:{minutes:02d}:{seconds:02d}",
+        f"#SBATCH --account={account}",
+        f"#SBATCH --output={sbatch_log_file}",
+        f"#SBATCH --mail-type={','.join(notifies)}" if
+        (len(notifies) > 0) else "",
+        f"#SBATCH --cpus-per-task={cores}",
+        f"#SBATCH --mem={memory}G" if memory > 0 else "",
+        *[f"#SBATCH -L {key}@osc:{val}" for key, val in license.items()],
+        f"#SBATCH --gres=gpu:{gpus}" if gpus > 0 else "",
+        *[
+            f"#SBATCH --{key}={val}"
+            if not isinstance(val, bool) else f"#SBATCH --{key}"
+            for key, val in sbatch_args.items() if len(str(val)) > 0
+        ],
+        *bashinit,
+        # *["whoami", f"echo $SHELL", "w", "tty", "ps"],
+        # *init_bashrc,
+        *env_var_decls,
+        *alias_var_decls,
+        *path_export,
+        "" if module_profie is None else f"module use {module_profie}",
+        *[f"module load {key}" for key in modules],
+        *pre_set_content,
+        f"source \"{python_env}\"" if python_env != "" else "",
+        "" if set_flag is None else f"set -{set_flag}",
+    ]
+    shell_script_tail = [
+        "mv \"$SLURM_SUBMIT_DIR/output/$SLURM_JOB_ID.log\" \"$OUTPUT_DIR/sbatch.log\""
+        if sbatch_log is None else "",
+    ]
+
+    # print(content)
+
+    with open(script_path, 'w') as fout:
+        fout.write("\n".join(shell_script_head + content + shell_script_tail))
+    if (chmod):
+        # os.system(f"chmod {chmod} {script_path}")
+        os.chmod(script_path, stat.S_IRWXU)
+
+
+def make_command(
+    head,
+    params: List[str] = [],
+    params1_dict: Dict = {},
+    params2_dict: Dict = {},
+    stdout_redirect: str = "",
+    connection="=",
+):
+    for key, val in list(params1_dict.items()):
+        if (isinstance(val, bool)):
+            if (val):
+                params1_dict[key] = ""
+            else:
+                del params1_dict[key]
+        elif (isinstance(val, str)):
+            if (len(val) == 0):
+                del params1_dict[key]
+            else:
+                params1_dict[key] = f'"{val}"'
+
+    for key, val in list(params2_dict.items()):
+        if (isinstance(val, bool)):
+            if (val):
+                params2_dict[key] = ""
+            else:
+                del params2_dict[key]
+        elif (isinstance(val, str)):
+            if (len(val) == 0):
+                del params2_dict[key]
+            else:
+                params2_dict[key] = f'"{val}"'
+    command = [
+        head,
+        *[f"-{key}{connection}{val}" for key, val in params1_dict.items()],
+        *[f"--{key}{connection}{val}" for key, val in params2_dict.items()],
+        *params,
+        f">> {stdout_redirect}" if len(stdout_redirect) > 0 else "",
+    ]
+    return " ".join(command).replace("  ", " ").strip()
+
+
+def make_if_statement(
+    if_st: Tuple[(List[str], List[str])],
+    elif_sts: List[Tuple[(str, List[str])]] = [],
+    else_st: List[str] = [],
+):
+
+    if_cond, if_act = if_st
+    if_cond_str = " && ".join([f"[ {c} ]" for c in if_cond])
+    terms = [f"if {if_cond_str}; then", *if_act]
+
+    for cond, act in elif_sts:
+        terms.extend([f"elif [ {cond} ]; then", *act])
+
+    if (len(else_st) > 0):
+        terms.extend(["else", *else_st])
+
+    terms.append("fi")
+
+    return terms

slurming-0.2.3a1/slurming/Slurm/slurm.py

@@ -0,0 +1,61 @@
+import subprocess
+from collections import defaultdict
+from typing import Dict
+
+
+def get_job_dict(cluster_name):
+    """
+    Return {job_name: (state, cluster, job_id)} for *all* visible jobs.
+    Falls back gracefully if federation/cluster info is absent.
+    """
+    # Ask every cluster we can see (requires Slurm ≥20.02 with Federation or dbd)
+
+    fields = ["Name", "StateCompact", "Cluster"]
+    cmd = ["squeue", "--noheader", f"-O{','.join(fields)}"]
+
+    fmt = "%.8j|%i|%T|%q"  # id | state | cluster (M == cluster in squeue ≥22.05)
+    user_name = subprocess.getoutput("whoami")
+    try:
+        cmd = [
+            "squeue",
+            f"--user={user_name}",
+            "--noheader",
+            "--clusters=all",
+            "-o",
+            fmt,
+        ]
+        out = subprocess.check_output(cmd, text=True).strip().splitlines()
+    except subprocess.CalledProcessError:
+        # Fallback: ask only the local cluster
+        cmd = ["squeue", f"--user={user_name}", "--noheader", "-o", fmt]
+        out = subprocess.check_output(cmd, text=True).strip().splitlines()
+
+    jobs: Dict[str, Dict[str, Dict]] = defaultdict(dict)
+    for line in out:
+        # If the user is on a single cluster, the third field will be empty;
+        # use $SLURM_CLUSTER_NAME or 'local' instead.
+        fields = line.split("|")
+        job_name, job_id, state, cluster = fields
+        cluster, *_ = cluster.split("-")
+        if (cluster not in jobs):
+            jobs[cluster]: Dict[str, Dict] = defaultdict(dict)
+        cluster_dict = jobs[cluster]
+        cluster_dict[job_name]["state"] = state
+        cluster_dict[job_name]["job_id"] = job_id
+
+    running_jobs = {
+        k: v
+        for cluster, job_dict in jobs.items()
+        for k, v in job_dict.items() if "RUNNING" == v['state']
+    }
+    pending_jobs = {
+        k: v
+        for k, v in jobs[cluster_name].items() if "PENDING" == v['state']
+    }
+
+    print(f"Detected {len(running_jobs)} running jobs and {len(pending_jobs)} pending jobs on cluster '{cluster_name}'.")
+    return jobs, running_jobs, pending_jobs
+
+
+if __name__ == "__main__":
+    print(get_job_dict())

slurming-0.2.3a1/slurming/SlurmJob/job.py

@@ -0,0 +1,163 @@
+from typing import Dict, List, Union, Tuple
+from pathlib import Path
+from slurming.Slurm.shellUtils import make_shell_script
+
+
+class SlurmJob:
+
+    account: str
+    content: List[str]
+
+    licenses: Dict[str, int]
+
+    modules: List[str]
+    module_profile: Union[str, None]
+
+    env_vars: Dict[str, str]
+    notify_email: List[str]
+    alias: Dict[str, str]
+    paths: List[str]
+
+    days: int
+    hours: int
+    minutes: int
+    seconds: int
+
+    cpus_per_task: int
+    gpus: int
+    memory: int
+
+    python_home: Union[Path, None]
+    set_flag: str
+
+    pre_content: List[str]
+
+    partition: Union[str, None]
+    job_name: Union[str, None]
+
+    interactive: bool
+    bashinit: List[str]
+
+    working_dir: Union[Path, None]
+
+    sbatch_args: Dict[str, Union[str, int, bool]]
+
+    output_storage: List[str]
+
+    def __init__(
+        self,
+        account: str,
+        content: List[str],
+        licenses: Dict[str, int],
+        modules: List[str],
+        env_vars: Dict[str, str],
+        notify_email: List[str],
+        aliases: Dict[str, str],
+        paths: List[str],
+        sbatch_args: Dict[str, Union[str, int, bool]],
+        output_storage: List[str],
+        days: int = 0,
+        hours: int = 0,
+        minutes: int = 0,
+        seconds: int = 0,
+        cpus_per_task: int = 1,
+        gpus: int = 0,
+        memory: int = 0,
+        module_profile: Union[str, None] = None,
+        python_home: Union[Path, None] = None,
+        set_flag: str = "x",
+        job_name: str = "",
+        partition: str = "",
+        interactive: bool = False,
+        working_dir: Union[Path, None] = None,
+        bashinit: List[str] = [],
+    ):
+        self.job_name = job_name
+        self.partition = partition
+        self.gpus = gpus
+        self.cpus_per_task = cpus_per_task
+        self.days = days
+        self.hours = hours
+        self.minutes = minutes
+        self.seconds = seconds
+
+        self.memory = memory
+        self.notify_email = notify_email
+        self.set_flag = set_flag
+        self.interactive = interactive
+        self.aliases = aliases
+        self.paths = paths
+        self.account = account
+        self.python_home = python_home
+        self.modules = modules
+        self.content = content
+        self.licenses = licenses
+        self.working_dir = working_dir
+
+        self.module_profile = module_profile
+        self.env_vars = env_vars
+        self.sbatch_args = sbatch_args
+        self.output_storage = output_storage
+
+        self.bashinit = bashinit
+
+    def prepend(self, content: List[str]):
+        """Append content to the job script."""
+        self.content = [*content, *self.content]
+
+    def append(self, content: List[str]):
+        """Append content to the job script."""
+        self.content = [*self.content, *content]
+
+    def write(
+        self,
+        job_name: str,
+        script_path: Path,
+        log_file: Union[Path, None] = None,
+        delete_log_on_completion: bool = False,
+        delete_script_on_completion: bool = False,
+    ):
+        job_name = "" if (self.job_name is None) else self.job_name
+        python_env = str(self.python_home / "bin" /
+                         "activate") if self.python_home else ""
+        log_file = log_file if log_file is not None else script_path.with_suffix(
+            '.log')
+        set_flags = set(list(self.set_flag))
+
+        if (delete_log_on_completion):
+            self.append([f"rm -f {log_file}"])
+            set_flags.add('e')
+        if (delete_script_on_completion):
+            self.append([f"rm -f {script_path}"])
+            set_flags.add('e')
+
+        preset_content = []
+        if (self.working_dir is not None):
+            preset_content.append(f"cd {self.working_dir}")
+
+        make_shell_script(
+            account=self.account,
+            script_path=script_path,
+            jobname=job_name,
+            gpus=self.gpus,
+            cores=self.cpus_per_task,
+            hours=self.hours + self.days * 24,
+            minutes=self.minutes,
+            seconds=self.seconds,
+            memory=self.memory,
+            notifies=self.notify_email,
+            sbatch_log=log_file,
+            set_flag=''.join(set_flags),
+            interactive=self.interactive,
+            aliases=self.aliases,
+            paths=self.paths,
+            python_env=python_env,
+            modules=self.modules,
+            content=self.content,
+            license=self.licenses,
+            pre_set_content=preset_content,
+            module_profie=self.module_profile,
+            env_vars=self.env_vars,
+            sbatch_args=self.sbatch_args,
+            bashinit=self.bashinit,
+        )