ece4-exp 1.0.0__tar.gz

ece4_exp-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024-2026 Barcelona Supercomputing Center (BSC)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ece4_exp-1.0.0/MANIFEST.in

@@ -0,0 +1,18 @@
+# Include recipe and platform files
+recursive-include recipes *.yml *.yaml
+recursive-include platforms *.yml *.yaml
+
+# Include documentation
+include README.md
+include LICENSE
+include QUICK_DEMO.sh
+
+# Include scripts
+include scripts/ece4-exp-completion.sh
+
+# Exclude development files
+exclude ece4-exp
+exclude setup.sh
+exclude SIMPLIFICATION_ANALYSIS.md
+recursive-exclude autosubmit *
+global-exclude *.pyc __pycache__

ece4_exp-1.0.0/PKG-INFO

@@ -0,0 +1,205 @@
+Metadata-Version: 2.1
+Name: ece4-exp
+Version: 1.0.0
+Summary: EC-Earth4 experiment configuration tool with YAML overlay system
+Author-email: Vladimir Lapin <vladimir.lapin@bsc.es>
+Project-URL: Homepage, https://github.com/vlap/ece4-exp
+Project-URL: Bug Tracker, https://github.com/vlap/ece4-exp/issues
+Project-URL: Documentation, https://github.com/vlap/ece4-exp/blob/main/README.md
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Atmospheric Science
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: ruamel.yaml>=0.17.0
+Requires-Dist: jsonschema>=4.0.0
+Provides-Extra: highlight
+Requires-Dist: pygments>=2.10.0; extra == "highlight"
+
+# ece4-exp: EC-Earth4 Experiment Configuration Tool
+
+**Generate EC-Earth4 configs in 30 seconds instead of 2-4 hours.**
+
+One command creates production-ready configurations with correct components, calculated node layouts, and validated parameters.
+
+## Quick Start
+
+### Install
+
+```bash
+# Via pip (recommended)
+pip install ece4-exp
+
+# Or from source
+git clone https://github.com/vlap/ece4-exp.git
+cd ece4-exp
+pip install -e .
+```
+
+### Generate Your First Config
+
+```bash
+ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 --expid myexp \
+  --platform bsc-marenostrum5 --account bsc32
+```
+
+**Optional:** Save defaults to avoid typing platform/account every time:
+
+```bash
+ece4-exp init-user
+# Edit ~/.config/ece4-exp/defaults.yml
+```
+
+Then just:
+
+```bash
+ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 --expid myexp
+```
+
+## Pre-Built Recipes
+
+| Recipe | Description | Components | Procs (MN5) |
+|--------|-------------|------------|-------------|
+| `gcm-sr.yml` | Coupled atmosphere-ocean | OIFS + NEMO + XIOS + OASIS | 1120 (10 nodes) |
+| `omip-sr.yml` | Ocean-only | NEMO + XIOS | 224 (2 nodes) |
+| `amip-sr.yml` | Atmosphere-only | OIFS + XIOS + AMIPFR | 896 (8 nodes) |
+| `ccycle-sr.yml` | Carbon cycle | OIFS + NEMO + LPJG + XIOS | 1120+ |
+
+## Common Commands
+
+```bash
+# List available recipes
+ece4-exp list
+
+# Generate experiment config
+ece4-exp generate --recipe <recipe.yml> --sim-procs <N> --expid <id>
+
+# Preview without creating file
+ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 --dry-run
+
+# Validate configuration
+ece4-exp validate myexp.yml
+
+# Save customized config as new recipe
+ece4-exp save --expid myexp --recipe gcm-sr.yml -o my-recipe.yml
+
+# Initialize user defaults (optional)
+ece4-exp init-user
+
+# Show help
+ece4-exp --help
+ece4-exp generate --help
+```
+
+## Examples
+
+### Ocean-Only Experiment
+
+```bash
+ece4-exp generate --recipe omip-sr.yml --sim-procs 224 --expid omip001 \
+  --platform bsc-marenostrum5 --account bsc32
+```
+
+### Custom Recipe Workflow
+
+```bash
+# 1. Generate base config
+ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 -o test.yml
+
+# 2. Edit manually
+vim test.yml
+
+# 3. Validate
+ece4-exp validate test.yml
+
+# 4. Save as new recipe (includes base + your changes)
+ece4-exp save --expid test --recipe gcm-sr.yml -o my-tuned-gcm.yml
+
+# 5. Reuse
+ece4-exp generate --recipe my-tuned-gcm.yml --sim-procs 1120 --expid exp002
+```
+
+### Batch Generation
+
+```bash
+for id in {001..010}; do
+  ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 \
+    --expid exp$id --quiet
+done
+```
+
+## How It Works
+
+**YAML overlay system** merges configurations in this order:
+
+1. Base config (from EC-Earth4 repo)
+2. Platform launcher settings
+3. Recipe (experiment pattern)
+4. User defaults (~/.config/ece4-exp/defaults.yml)
+5. CLI flags
+
+Each layer overrides the previous, giving you full control.
+
+## Run with EC-Earth4
+
+```bash
+cd /path/to/ecearth4/scripts/runtime
+se user.yml platform.yml experiment.yml scriptlib/main.yml
+```
+
+Where `experiment.yml` is generated by ece4-exp.
+
+## Documentation
+
+- **Try the demo:** `./QUICK_DEMO.sh` (3-minute interactive walkthrough)
+- **Detailed guide:** [DEMO.md](DEMO.md)
+- **Presentation:** [docs/presentation/](docs/presentation/) (slides)
+- **Changelog:** [docs/CHANGES.md](docs/CHANGES.md)
+
+## Platform Support
+
+**Currently supported:**
+- BSC MareNostrum 5
+- ECMWF HPC2020
+
+**Adding a new platform:**  
+Create `platforms/<platform>.yml` with launcher templates (~50 lines).
+
+## Requirements
+
+- Python 3.8+
+- ruamel.yaml >= 0.17.0
+- jsonschema >= 4.0.0
+- pygments >= 2.10.0 (optional, for syntax highlighting)
+
+## Features
+
+✅ Pre-tested recipes  
+✅ Auto node layout calculation  
+✅ Config validation  
+✅ Upstream EC-Earth4 sync  
+✅ Custom recipe creation  
+✅ Script-friendly (--quiet mode)  
+✅ Cross-platform (Linux, macOS, Windows)
+
+## Autosubmit Compatibility
+
+For auto-ecearth4 users, backward compatibility is maintained:
+
+```bash
+ece4-exp generate --expdef expdef_ID.yml --jobs jobs_ID.yml
+```
+
+## License & Support
+
+**License:** MIT - See [LICENSE](LICENSE)
+
+**Questions?** vladimir.lapin@bsc.es  
+**Issues:** https://github.com/vlap/ece4-exp/issues
+
+## Contributing
+
+See [SIMPLIFICATION_ANALYSIS.md](SIMPLIFICATION_ANALYSIS.md) for development roadmap.

ece4_exp-1.0.0/QUICK_DEMO.sh

@@ -0,0 +1,106 @@
+#!/usr/bin/env bash
+# Quick 2-minute demo of ece4-exp
+
+set +e  # Don't exit on errors
+
+# Colors
+CYAN='\033[0;36m'
+GREEN='\033[0;32m'
+YELLOW='\033[1;33m'
+NC='\033[0m'
+
+echo "╔══════════════════════════════════════════════════════════╗"
+echo "║       ece4-exp: EC-Earth4 Made Easy - Quick Demo        ║"
+echo "╚══════════════════════════════════════════════════════════╝"
+echo ""
+
+# Step 1: List recipes
+echo -e "${CYAN}📋 Step 1: What recipes are available?${NC}"
+echo "$ ece4-exp list"
+echo ""
+ece4-exp list
+echo ""
+read -p "Press Enter to continue..."
+echo ""
+
+# Step 2: Generate GCM experiment
+echo -e "${CYAN}🚀 Step 2: Generate a coupled GCM experiment${NC}"
+echo "$ ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 --expid demo-gcm"
+echo ""
+if ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 --expid demo-gcm -o demo-gcm.yml; then
+    echo -e "${GREEN}✓ Created demo-gcm.yml${NC}"
+else
+    echo -e "${YELLOW}✗ Generation failed${NC}"
+    exit 1
+fi
+echo ""
+read -p "Press Enter to continue..."
+echo ""
+
+# Step 3: Modify and save as custom recipe
+echo -e "${CYAN}💾 Step 3: Customize and save as new recipe${NC}"
+echo "$ ece4-exp save - Save your modifications as a reusable recipe"
+echo ""
+echo "Let's modify the config to run 10 legs instead of 1:"
+sed -i 's/nlegs: 1/nlegs: 10/' demo-gcm.yml
+echo "Changed: nlegs: 1 → nlegs: 10"
+echo ""
+# Rename for save command (expects {expid}_experiment.yml format)
+cp demo-gcm.yml demo-gcm_experiment.yml
+cp ece4_exp/demo-gcm_pristine.yml ece4_exp/demo-gcm_experiment_pristine.yml
+echo "$ python3 -m ece4_exp.save_recipe_from_diff --expid demo-gcm --recipe gcm-sr.yml -o my-10legs.yml"
+if python3 -m ece4_exp.save_recipe_from_diff --expid demo-gcm --recipe gcm-sr.yml -o my-10legs.yml 2>&1 | grep -v "^$"; then
+    echo ""
+    if [ -f recipes/my-10legs.yml ]; then
+        echo -e "${GREEN}✓ Saved as recipes/my-10legs.yml${NC}"
+        echo "Recipe content:"
+        cat recipes/my-10legs.yml
+    fi
+fi
+echo ""
+read -p "Press Enter to continue..."
+echo ""
+
+# Step 4: Validate
+echo -e "${CYAN}✅ Step 4: Validate the configuration${NC}"
+echo "$ ece4-exp validate demo-gcm.yml"
+echo ""
+ece4-exp validate demo-gcm.yml
+echo ""
+read -p "Press Enter to continue..."
+echo ""
+
+# Step 5: Ocean-only example
+echo -e "${CYAN}🌊 Step 5: Ocean-only experiment (different setup)${NC}"
+echo "$ ece4-exp generate --recipe omip-sr.yml --sim-procs 224 --expid demo-omip"
+echo ""
+if ece4-exp generate --recipe omip-sr.yml --sim-procs 224 --expid demo-omip -o demo-omip.yml 2>&1 | tail -5; then
+    echo ""
+    echo "Components (no atmosphere!):"
+    grep "components:" demo-omip.yml
+    echo -e "${GREEN}✓ Created demo-omip.yml${NC}"
+fi
+echo ""
+
+echo "╔══════════════════════════════════════════════════════════╗"
+echo "║                      Demo Complete!                      ║"
+echo "╚══════════════════════════════════════════════════════════╝"
+echo ""
+echo -e "${GREEN}⏱️  Time: ~3 minutes${NC}"
+echo ""
+echo "What you saw:"
+echo "  1. List available recipes"
+echo "  2. Generate coupled GCM config"
+echo "  3. Modify config and save as new recipe"
+echo "  4. Validate config"
+echo "  5. Generate ocean-only config"
+echo ""
+echo "Next steps:"
+echo "  • Run: ece4-exp init-user (set your defaults)"
+echo "  • Read: README.md (full documentation)"
+echo "  • View: docs/presentation/ece4-exp-intro.pdf (slides)"
+echo ""
+echo -e "${CYAN}Cleaning up demo files...${NC}"
+rm -f demo-*.yml recipes/my-*.yml ece4_exp/demo-*_pristine.yml
+echo -e "${GREEN}✓ Demo files cleaned${NC}"
+echo ""

ece4_exp-1.0.0/README.md

@@ -0,0 +1,184 @@
+# ece4-exp: EC-Earth4 Experiment Configuration Tool
+
+**Generate EC-Earth4 configs in 30 seconds instead of 2-4 hours.**
+
+One command creates production-ready configurations with correct components, calculated node layouts, and validated parameters.
+
+## Quick Start
+
+### Install
+
+```bash
+# Via pip (recommended)
+pip install ece4-exp
+
+# Or from source
+git clone https://github.com/vlap/ece4-exp.git
+cd ece4-exp
+pip install -e .
+```
+
+### Generate Your First Config
+
+```bash
+ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 --expid myexp \
+  --platform bsc-marenostrum5 --account bsc32
+```
+
+**Optional:** Save defaults to avoid typing platform/account every time:
+
+```bash
+ece4-exp init-user
+# Edit ~/.config/ece4-exp/defaults.yml
+```
+
+Then just:
+
+```bash
+ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 --expid myexp
+```
+
+## Pre-Built Recipes
+
+| Recipe | Description | Components | Procs (MN5) |
+|--------|-------------|------------|-------------|
+| `gcm-sr.yml` | Coupled atmosphere-ocean | OIFS + NEMO + XIOS + OASIS | 1120 (10 nodes) |
+| `omip-sr.yml` | Ocean-only | NEMO + XIOS | 224 (2 nodes) |
+| `amip-sr.yml` | Atmosphere-only | OIFS + XIOS + AMIPFR | 896 (8 nodes) |
+| `ccycle-sr.yml` | Carbon cycle | OIFS + NEMO + LPJG + XIOS | 1120+ |
+
+## Common Commands
+
+```bash
+# List available recipes
+ece4-exp list
+
+# Generate experiment config
+ece4-exp generate --recipe <recipe.yml> --sim-procs <N> --expid <id>
+
+# Preview without creating file
+ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 --dry-run
+
+# Validate configuration
+ece4-exp validate myexp.yml
+
+# Save customized config as new recipe
+ece4-exp save --expid myexp --recipe gcm-sr.yml -o my-recipe.yml
+
+# Initialize user defaults (optional)
+ece4-exp init-user
+
+# Show help
+ece4-exp --help
+ece4-exp generate --help
+```
+
+## Examples
+
+### Ocean-Only Experiment
+
+```bash
+ece4-exp generate --recipe omip-sr.yml --sim-procs 224 --expid omip001 \
+  --platform bsc-marenostrum5 --account bsc32
+```
+
+### Custom Recipe Workflow
+
+```bash
+# 1. Generate base config
+ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 -o test.yml
+
+# 2. Edit manually
+vim test.yml
+
+# 3. Validate
+ece4-exp validate test.yml
+
+# 4. Save as new recipe (includes base + your changes)
+ece4-exp save --expid test --recipe gcm-sr.yml -o my-tuned-gcm.yml
+
+# 5. Reuse
+ece4-exp generate --recipe my-tuned-gcm.yml --sim-procs 1120 --expid exp002
+```
+
+### Batch Generation
+
+```bash
+for id in {001..010}; do
+  ece4-exp generate --recipe gcm-sr.yml --sim-procs 1120 \
+    --expid exp$id --quiet
+done
+```
+
+## How It Works
+
+**YAML overlay system** merges configurations in this order:
+
+1. Base config (from EC-Earth4 repo)
+2. Platform launcher settings
+3. Recipe (experiment pattern)
+4. User defaults (~/.config/ece4-exp/defaults.yml)
+5. CLI flags
+
+Each layer overrides the previous, giving you full control.
+
+## Run with EC-Earth4
+
+```bash
+cd /path/to/ecearth4/scripts/runtime
+se user.yml platform.yml experiment.yml scriptlib/main.yml
+```
+
+Where `experiment.yml` is generated by ece4-exp.
+
+## Documentation
+
+- **Try the demo:** `./QUICK_DEMO.sh` (3-minute interactive walkthrough)
+- **Detailed guide:** [DEMO.md](DEMO.md)
+- **Presentation:** [docs/presentation/](docs/presentation/) (slides)
+- **Changelog:** [docs/CHANGES.md](docs/CHANGES.md)
+
+## Platform Support
+
+**Currently supported:**
+- BSC MareNostrum 5
+- ECMWF HPC2020
+
+**Adding a new platform:**  
+Create `platforms/<platform>.yml` with launcher templates (~50 lines).
+
+## Requirements
+
+- Python 3.8+
+- ruamel.yaml >= 0.17.0
+- jsonschema >= 4.0.0
+- pygments >= 2.10.0 (optional, for syntax highlighting)
+
+## Features
+
+✅ Pre-tested recipes  
+✅ Auto node layout calculation  
+✅ Config validation  
+✅ Upstream EC-Earth4 sync  
+✅ Custom recipe creation  
+✅ Script-friendly (--quiet mode)  
+✅ Cross-platform (Linux, macOS, Windows)
+
+## Autosubmit Compatibility
+
+For auto-ecearth4 users, backward compatibility is maintained:
+
+```bash
+ece4-exp generate --expdef expdef_ID.yml --jobs jobs_ID.yml
+```
+
+## License & Support
+
+**License:** MIT - See [LICENSE](LICENSE)
+
+**Questions?** vladimir.lapin@bsc.es  
+**Issues:** https://github.com/vlap/ece4-exp/issues
+
+## Contributing
+
+See [SIMPLIFICATION_ANALYSIS.md](SIMPLIFICATION_ANALYSIS.md) for development roadmap.