wrfrun 0.2.0__tar.gz → 0.3.0__tar.gz

{wrfrun-0.2.0 → wrfrun-0.3.0}/.gitignore

@@ -22,3 +22,6 @@ test.*
 config.yaml
 main.py
 namelist.*
+uv.lock
+
+!**/namelist.py

wrfrun-0.3.0/.python-version

@@ -0,0 +1 @@
+3.10

wrfrun-0.3.0/PKG-INFO

@@ -0,0 +1,240 @@
+Metadata-Version: 2.1
+Name: wrfrun
+Version: 0.3.0
+Summary: wrfrun is a comprehensive toolkit for managing and using WRF
+Keywords: WRF
+Author-Email: Syize <syizeliu@gmail.com>
+Maintainer-Email: Syize <syizeliu@gmail.com>
+License: GPL-3.0-or-later
+Project-URL: homepage, https://github.com/Syize/wrfrun
+Project-URL: repository, https://github.com/Syize/wrfrun
+Project-URL: documentation, https://wrfrun.syize.cn
+Project-URL: Bug Tracker, https://github.com/Syize/wrfrun/issues
+Requires-Python: >=3.10
+Requires-Dist: numpy
+Requires-Dist: xarray
+Requires-Dist: netCDF4
+Requires-Dist: rich
+Requires-Dist: pandas
+Requires-Dist: f90nml
+Requires-Dist: cdsapi
+Requires-Dist: matplotlib
+Requires-Dist: Cartopy
+Requires-Dist: tomli
+Requires-Dist: tomli-w
+Requires-Dist: haversine
+Requires-Dist: wrf-python>=1.3.1
+Requires-Dist: cfgrib>=0.9.15.1
+Description-Content-Type: text/markdown
+
+# 🌀 wrfrun
+
+> A modern, unified framework for running and managing numerical models.
+>
+> Designed for researchers who want to focus on science — not on the details of model execution.
+
+## 📖 Introduction
+
+`wrfrun` is a Python package that provides a general-purpose, reproducible, and extensible framework for running numerical models.
+
+It automates the tedious parts of model execution — preparing input data, handling `namelist` configurations, organizing logs, and submitting jobs — so that you can spend your time on research, not on managing model runs.
+
+## ⚡ Quick Installation
+
+```bash
+pip install wrfrun
+```
+
+## 📖 Documentation
+
+Check [https://wrfrun.syize.cn](https://wrfrun.syize.cn/).
+
+## 🌟 Core Features
+
+### 🧩 Unified Interface Architecture
+
+`wrfrun` enforces a unified interface specification for all numerical models. Specifically, each model interface must inherit from a provided base class, ensuring a consistent structure and behavior across different models.
+
+This design makes model execution intuitive — any supported model can be launched simply by calling a Python function or class method, while `wrfrun` automatically handles all background tasks such as data preparation and configuration file management.
+
+```python
+from wrfrun import WRFRun
+from wrfrun.model.wrf import geogrid, metgrid, real, ungrib, wrf
+
+# wrfrun will prepare input data, generate namelist file, 
+# save outputs and logs automatically.
+with WRFRun("./config.toml", init_workspace=True) as wrf_run:
+    geogrid()
+    ungrib()
+    metgrid()
+    real()
+    wrf()
+```
+
+### 🪶 Record & Replay
+
+Every simulation can be fully recorded and later reproduced from a single `.replay` file — ensuring total reproducibility.
+
+```python
+from wrfrun import WRFRun
+from wrfrun.model.wrf import geogrid, ungrib, metgrid, real, wrf
+
+
+# 1. Record simulation with method `record_simulation`.
+with WRFRun("./config.toml", init_workspace=True) as wrf_run:
+    wrf_run.record_simulation(output_path="./outputs/example.replay")
+
+    geogrid()
+    ungrib()
+    metgrid()
+    real()
+    wrf()
+
+# 2. Replay the simulation in a different directory or on a different machine.
+with WRFRun("./config.toml", init_workspace=True) as wrf_run:
+    wrf_run.replay_simulation("./example.replay")
+```
+
+### ⚙️ Simplified Configuration
+
+Manage all simulation settings in TOML files: A main config file, and model config files.
+
+For more information about the configuration file, check [config](wrfrun/res/config).
+
+```toml
+# main config file: config.toml
+work_dir = "./.wrfrun"
+
+input_data_path = ""
+output_path = "./outputs"
+log_path = "./logs"
+
+server_host = "localhost"
+server_port = 54321
+
+core_num = 36
+
+[job_scheduler]
+job_scheduler = "pbs"
+
+queue_name = ""
+node_num = 1
+env_settings = {}
+python_interpreter = "/usr/bin/python3"     # or just "python3"
+
+[model]
+[model.wrf]
+use = false
+include = "./configs/wrf.toml"
+```
+
+`wrfrun` remains compatible with original `namelist` inputs, just set namelist file path in the model config.
+
+```toml
+# WRF model config file: wrf.toml
+wps_path = '/path/to/your/WPS/folder'
+wrf_path = '/path/to/your/WRF/folder'
+wrfda_path = ''		# WRFDA is optional.
+geog_data_path = '/path/to/your/geog/data'
+user_wps_namelist = ''		# set your own namelist file here
+user_real_namelist = ''		# set your own namelist file here
+user_wrf_namelist = ''		# set your own namelist file here
+user_wrfda_namelist = ''	# set your own namelist file here
+restart_mode = false
+debug_level = 100
+
+[time]
+start_date = 2021-03-24T12:00:00Z   # or [2021-03-24T12:00:00Z, 2021-03-24T12:00:00Z]
+end_date = 2021-03-26T00:00:00Z     # or [2021-03-26T00:00:00Z, 2021-03-24T12:00:00Z]
+input_data_interval = 10800
+output_data_interval = 180
+time_step = 120
+parent_time_step_ratio = [1, 3, 4]
+restart_interval = -1
+
+
+[domain]
+domain_num = 3
+parent_grid_ratio = [1, 3, 9]
+i_parent_start = [1, 17, 72]
+j_parent_start = [1, 17, 36]
+e_we = [120, 250, 1198]
+e_sn = [120, 220, 1297]
+dx = 9000
+dy = 9000
+map_proj = 'lambert'
+truelat1 = 34.0
+truelat2 = 40.0
+ref_lat = 37.0
+ref_lon = 120.5
+stand_lon = 120.5
+
+
+[scheme]
+long_wave_scheme = { name = "rrtm", option = {} }
+short_wave_scheme = { name = "rrtmg", option = {} }
+cumulus_scheme = { name = "kf", option = {} }
+pbl_scheme = { name = "ysu", option = { ysu_topdown_pblmix = 1} }
+land_surface_scheme = { name = "noah", option = {} }
+surface_layer_scheme = { name = "mm5", option = {} }
+```
+
+### 💻 Job Scheduling Integration
+
+Automatically submit jobs to supported schedulers:
+
+- PBS
+- Slurm
+- LSF
+
+`wrfrun` takes care of resource requests and queue management automatically.
+
+```python
+from wrfrun import WRFRun
+from wrfrun.model.wrf import geogrid, metgrid, real, ungrib, wrf
+
+# just set submit_job=True
+with WRFRun("./config.toml", init_workspace=True, submit_job=True) as wrf_run:
+    geogrid()
+    ungrib()
+    metgrid()
+    real()
+    wrf()
+```
+
+### 📡 Real-time Monitoring
+
+`wrfrun` can parse model log files and start a lightweight socket server to report simulation progress.
+
+```python
+from wrfrun import WRFRun
+from wrfrun.model.wrf import geogrid, metgrid, real, ungrib, wrf
+
+# just set start_server=True
+with WRFRun("./config.toml", init_workspace=True, start_server=True) as wrf_run:
+    geogrid()
+    ungrib()
+    metgrid()
+    real()
+    wrf()
+```
+
+## 🌍 Current Capabilities
+
+- Automated ERA5 data download (requires `cdsapi` authentication)
+- Real-time progress reporting via socket interface
+- Partial WRF support:
+  - Full support for WPS
+  - Wrapped execution for `real` and `wrf`
+- Job submission on PBS, Slurm, and LSF
+- `record` / `replay` reproducibility for all compliant interfaces
+
+## 🧭 TODO
+
+- [ ] Full WRF model integration.
+- [ ] Broaden model support.
+- [ ] Enhanced progress visualization dashboard.
+
+## 🤝 Contributing
+
+This project is currently for personal and research use. If you have ideas or feature requests, feel free to open an issue.

wrfrun-0.3.0/README.md

@@ -0,0 +1,211 @@
+# 🌀 wrfrun
+
+> A modern, unified framework for running and managing numerical models.
+>
+> Designed for researchers who want to focus on science — not on the details of model execution.
+
+## 📖 Introduction
+
+`wrfrun` is a Python package that provides a general-purpose, reproducible, and extensible framework for running numerical models.
+
+It automates the tedious parts of model execution — preparing input data, handling `namelist` configurations, organizing logs, and submitting jobs — so that you can spend your time on research, not on managing model runs.
+
+## ⚡ Quick Installation
+
+```bash
+pip install wrfrun
+```
+
+## 📖 Documentation
+
+Check [https://wrfrun.syize.cn](https://wrfrun.syize.cn/).
+
+## 🌟 Core Features
+
+### 🧩 Unified Interface Architecture
+
+`wrfrun` enforces a unified interface specification for all numerical models. Specifically, each model interface must inherit from a provided base class, ensuring a consistent structure and behavior across different models.
+
+This design makes model execution intuitive — any supported model can be launched simply by calling a Python function or class method, while `wrfrun` automatically handles all background tasks such as data preparation and configuration file management.
+
+```python
+from wrfrun import WRFRun
+from wrfrun.model.wrf import geogrid, metgrid, real, ungrib, wrf
+
+# wrfrun will prepare input data, generate namelist file, 
+# save outputs and logs automatically.
+with WRFRun("./config.toml", init_workspace=True) as wrf_run:
+    geogrid()
+    ungrib()
+    metgrid()
+    real()
+    wrf()
+```
+
+### 🪶 Record & Replay
+
+Every simulation can be fully recorded and later reproduced from a single `.replay` file — ensuring total reproducibility.
+
+```python
+from wrfrun import WRFRun
+from wrfrun.model.wrf import geogrid, ungrib, metgrid, real, wrf
+
+
+# 1. Record simulation with method `record_simulation`.
+with WRFRun("./config.toml", init_workspace=True) as wrf_run:
+    wrf_run.record_simulation(output_path="./outputs/example.replay")
+
+    geogrid()
+    ungrib()
+    metgrid()
+    real()
+    wrf()
+
+# 2. Replay the simulation in a different directory or on a different machine.
+with WRFRun("./config.toml", init_workspace=True) as wrf_run:
+    wrf_run.replay_simulation("./example.replay")
+```
+
+### ⚙️ Simplified Configuration
+
+Manage all simulation settings in TOML files: A main config file, and model config files.
+
+For more information about the configuration file, check [config](wrfrun/res/config).
+
+```toml
+# main config file: config.toml
+work_dir = "./.wrfrun"
+
+input_data_path = ""
+output_path = "./outputs"
+log_path = "./logs"
+
+server_host = "localhost"
+server_port = 54321
+
+core_num = 36
+
+[job_scheduler]
+job_scheduler = "pbs"
+
+queue_name = ""
+node_num = 1
+env_settings = {}
+python_interpreter = "/usr/bin/python3"     # or just "python3"
+
+[model]
+[model.wrf]
+use = false
+include = "./configs/wrf.toml"
+```
+
+`wrfrun` remains compatible with original `namelist` inputs, just set namelist file path in the model config.
+
+```toml
+# WRF model config file: wrf.toml
+wps_path = '/path/to/your/WPS/folder'
+wrf_path = '/path/to/your/WRF/folder'
+wrfda_path = ''		# WRFDA is optional.
+geog_data_path = '/path/to/your/geog/data'
+user_wps_namelist = ''		# set your own namelist file here
+user_real_namelist = ''		# set your own namelist file here
+user_wrf_namelist = ''		# set your own namelist file here
+user_wrfda_namelist = ''	# set your own namelist file here
+restart_mode = false
+debug_level = 100
+
+[time]
+start_date = 2021-03-24T12:00:00Z   # or [2021-03-24T12:00:00Z, 2021-03-24T12:00:00Z]
+end_date = 2021-03-26T00:00:00Z     # or [2021-03-26T00:00:00Z, 2021-03-24T12:00:00Z]
+input_data_interval = 10800
+output_data_interval = 180
+time_step = 120
+parent_time_step_ratio = [1, 3, 4]
+restart_interval = -1
+
+
+[domain]
+domain_num = 3
+parent_grid_ratio = [1, 3, 9]
+i_parent_start = [1, 17, 72]
+j_parent_start = [1, 17, 36]
+e_we = [120, 250, 1198]
+e_sn = [120, 220, 1297]
+dx = 9000
+dy = 9000
+map_proj = 'lambert'
+truelat1 = 34.0
+truelat2 = 40.0
+ref_lat = 37.0
+ref_lon = 120.5
+stand_lon = 120.5
+
+
+[scheme]
+long_wave_scheme = { name = "rrtm", option = {} }
+short_wave_scheme = { name = "rrtmg", option = {} }
+cumulus_scheme = { name = "kf", option = {} }
+pbl_scheme = { name = "ysu", option = { ysu_topdown_pblmix = 1} }
+land_surface_scheme = { name = "noah", option = {} }
+surface_layer_scheme = { name = "mm5", option = {} }
+```
+
+### 💻 Job Scheduling Integration
+
+Automatically submit jobs to supported schedulers:
+
+- PBS
+- Slurm
+- LSF
+
+`wrfrun` takes care of resource requests and queue management automatically.
+
+```python
+from wrfrun import WRFRun
+from wrfrun.model.wrf import geogrid, metgrid, real, ungrib, wrf
+
+# just set submit_job=True
+with WRFRun("./config.toml", init_workspace=True, submit_job=True) as wrf_run:
+    geogrid()
+    ungrib()
+    metgrid()
+    real()
+    wrf()
+```
+
+### 📡 Real-time Monitoring
+
+`wrfrun` can parse model log files and start a lightweight socket server to report simulation progress.
+
+```python
+from wrfrun import WRFRun
+from wrfrun.model.wrf import geogrid, metgrid, real, ungrib, wrf
+
+# just set start_server=True
+with WRFRun("./config.toml", init_workspace=True, start_server=True) as wrf_run:
+    geogrid()
+    ungrib()
+    metgrid()
+    real()
+    wrf()
+```
+
+## 🌍 Current Capabilities
+
+- Automated ERA5 data download (requires `cdsapi` authentication)
+- Real-time progress reporting via socket interface
+- Partial WRF support:
+  - Full support for WPS
+  - Wrapped execution for `real` and `wrf`
+- Job submission on PBS, Slurm, and LSF
+- `record` / `replay` reproducibility for all compliant interfaces
+
+## 🧭 TODO
+
+- [ ] Full WRF model integration.
+- [ ] Broaden model support.
+- [ ] Enhanced progress visualization dashboard.
+
+## 🤝 Contributing
+
+This project is currently for personal and research use. If you have ideas or feature requests, feel free to open an issue.

{wrfrun-0.2.0 → wrfrun-0.3.0}/meson.build

@@ -1,8 +1,8 @@
 project(
     'wrfrun',
     'c',
-    version: '0.2.0',
-    license: 'BSD-3',
+    version: '0.3.0',
+    license: 'GPL-3',
     meson_version: '>=0.64.0',
     default_options: ['warning_level=2'],
 )

{wrfrun-0.2.0 → wrfrun-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ requires = ["meson-python"]
 
 [project]
 name = "wrfrun"
-version = "0.2.0"
+version = "0.3.0"
 readme = "README.md"
 license = { text = "GPL-3.0-or-later" }
 requires-python = '>=3.10'
@@ -24,11 +24,11 @@ dependencies = [
     "cdsapi",
     "matplotlib",
     "Cartopy",
-    "wrf-python",
-    "cfgrib",
     "tomli",
     "tomli-w",
-    "haversine"
+    "haversine",
+    "wrf-python>=1.3.1",
+    "cfgrib>=0.9.15.1",
 ]
 
 [project.urls]
@@ -53,4 +53,19 @@ dev = [
     "sphinx-design",
     "meson-python",
     "sphinx-autobuild",
+    "ruff>=0.14.3",
+    "pyright>=1.1.407",
 ]
+
+[tool.ruff]
+line-length = 130
+
+[tool.ruff.lint]
+select = ["D", "E", "F", "I"]
+ignore = ["F403", "D211", "D212", "D200", "D203", "D401", "D404", "D205", "D400", "D415"]
+
+[tool.pyright]
+include = ["wrfrun"]
+pythonVersion = "3.10"
+pythonPlatform = "Linux"
+reportArgumentType = false

wrfrun-0.3.0/wrfrun/__init__.py

@@ -0,0 +1,8 @@
+"""
+wrfrun
+######
+"""
+
+from . import core, data, extension, model, plot, res, run, scheduler, utils, workspace
+
+__all__ = ["core", "data", "extension", "model", "plot", "res", "run", "scheduler", "utils", "workspace"]

wrfrun-0.3.0/wrfrun/cli.py

@@ -0,0 +1,171 @@
+"""
+wrfrun.cli
+##########
+
+.. autosummary::
+    :toctree: generated/
+
+    main_entry
+
+``wrfrun`` command line interface (CLI).
+
+**This is an experimental tool, NOT BEING STABLE YET.**
+
+After installing ``wrfrun``, you can use command ``wrfrun`` to create a simulation project.
+Get more information by running ``wrfrun --help``.
+"""
+
+import argparse
+import sys
+from os import listdir, makedirs
+from os.path import abspath, dirname, exists
+from shutil import copyfile
+
+import tomli
+import tomli_w
+
+from .core import WRFRunConfig
+from .log import logger
+from .res import CONFIG_MAIN_TOML_TEMPLATE, CONFIG_PALM_TOML_TEMPLATE, CONFIG_WRF_TOML_TEMPLATE, _register_res_uri
+
+MODEL_MAP = {
+    "wrf": CONFIG_WRF_TOML_TEMPLATE,
+    "palm": CONFIG_PALM_TOML_TEMPLATE,
+}
+
+# need some mannual calls to make cli work without a config file.
+wrfrun_config = WRFRunConfig("./.wrfrun")
+_register_res_uri(wrfrun_config)
+
+
+def _entry_init(args: argparse.Namespace):
+    """
+    Initialize a wrfrun project.
+
+    :param args: Arguments namespace.
+    :type args: argparse.Namespace
+    """
+    configs = vars(args)
+
+    project_name = configs["name"]
+    models = configs["models"]
+
+    if project_name is None:
+        project_name = "."
+
+    if exists(project_name):
+        # we need to check if this directory isn't empty.
+        files = listdir()
+        # exclude:
+        exclude_target = [".venv", ".git", ".gitignore", ".gitattribute", "pyproject.toml", "uv.lock"]
+        files = [x for x in files if x not in exclude_target]
+
+        if len(files) != 0:
+            logger.error(f"{project_name} isn't empty, choose an empty directory, or backup and delete your files first.")
+            exit(1)
+
+    makedirs(f"{project_name}/configs")
+    makedirs(f"{project_name}/data")
+    namelist_path = f"{project_name}/namelists"
+    makedirs(namelist_path)
+
+    copyfile(wrfrun_config.parse_resource_uri(CONFIG_MAIN_TOML_TEMPLATE), f"{project_name}/config.toml")
+
+    if models is not None:
+        for _model in models:
+            src_path = wrfrun_config.parse_resource_uri(MODEL_MAP[_model])
+            copyfile(src_path, f"{project_name}/configs/{_model}.toml")
+            makedirs(f"{namelist_path}/{_model}")
+
+    logger.info(f"Created project {project_name}")
+    logger.info(f"All your configs should be placed in {project_name}/configs.")
+    logger.info(f"All your data should be placed in {project_name}/data.")
+    logger.info(f"All your namelist files should be placed in {namelist_path}")
+    logger.info(f"Make sure to set `use = True` to enable models in {project_name}/config.toml .")
+    logger.info("It is recommanded to use git to track your wrfrun and model configs.")
+    logger.info("Use command `wrfrun add MODEL_NAME` to add a new model to project.")
+
+
+def _entry_model(args: argparse.Namespace):
+    """
+    Manage models used by wrfrun project.
+
+    :param args: Arguments namespace.
+    :type args: argparse.Namespace
+    """
+    configs = vars(args)
+    new_models = configs["add"]
+    config_path = configs["config"]
+
+    if not exists(config_path):
+        logger.error(f"Can't find '{config_path}', initialize this project first.")
+        exit(1)
+
+    for _new_model in new_models:
+        if _new_model not in MODEL_MAP:
+            logger.error(f"Unknow model type: '{_new_model}'")
+            exit(1)
+
+    config_dir_path = f"{abspath(dirname(config_path))}/configs"
+
+    if not exists(config_dir_path):
+        makedirs(config_dir_path)
+
+    with open(config_path, "rb") as f:
+        main_config = tomli.load(f)
+
+    for _new_model in new_models:
+        if _new_model not in main_config["model"]:
+            main_config["model"][_new_model] = {
+                "note": (
+                    "Config of this model is generated by wrfrun cli command. "
+                    "DO NOT ADD CUSTOM CONFIG IN THIS SECTION because they may be overwrite by wrfrun cli tools."
+                ),
+                "use": True,
+                "include": f"./configs/{_new_model}.toml",
+            }
+
+        else:
+            if not ("use" in main_config["model"] and main_config["model"]["use"]):
+                main_config["model"][_new_model] = {
+                    "note": (
+                        "Config of this model is generated by wrfrun cli command. "
+                        "DO NOT ADD CUSTOM CONFIG IN THIS SECTION because they may be overwrite by wrfrun cli tools."
+                    ),
+                    "use": True,
+                    "include": f"./configs/{_new_model}.toml",
+                }
+
+    for _new_model in new_models:
+        copyfile(wrfrun_config.parse_resource_uri(MODEL_MAP[_new_model]), f"{config_dir_path}/{_new_model}.toml")
+
+    with open(config_path, "wb") as f:
+        tomli_w.dump(main_config, f)
+
+    logger.info(f"Added models: {new_models}")
+
+
+def main_entry():
+    """
+    CLI entry point.
+    """
+    args_parser = argparse.ArgumentParser()
+    subparsers = args_parser.add_subparsers(title="Subcommands", description="Valid Subcommands", help="Subcommands")
+
+    init_parser = subparsers.add_parser("init", help="Initialize a wrfrun project.", add_help=True)
+    init_parser.add_argument("-n", "--name", type=str, help="Name of the wrfrun project.")
+    init_parser.add_argument("--models", nargs="*", type=str, help="List of models to use.", choices=["wrf", "palm"])
+    init_parser.set_defaults(func=_entry_init)
+
+    model_parser = subparsers.add_parser("model", help="Manage models used by wrfrun project.", add_help=True)
+    model_parser.add_argument("-c", "--config", type=str, default="config.toml", help="Path of the main config file.")
+    model_parser.add_argument(
+        "-a", "--add", nargs="+", required=True, type=str, help="Add models to the project.", choices=["wrf", "palm"]
+    )
+    model_parser.set_defaults(func=_entry_model)
+
+    args = args_parser.parse_args(args=None if sys.argv[1:] else ["--help"])
+    args.func(args)
+
+
+__all__ = ["main_entry"]