rc-qlc 0.3.26__cp311-cp311-macosx_11_0_arm64.whl → 0.3.27__cp311-cp311-macosx_11_0_arm64.whl

qlc/doc/README.md

@@ -10,6 +10,24 @@ The suite streamlines the entire post-processing workflow, from data retrieval a
 
 ---
 
+## What's New in v0.3.27
+
+This release focuses on improving the out-of-the-box installation experience, especially for HPC environments, and significantly expanding the user documentation.
+
+-   **Installer Overhaul**: The `qlc-install` script is now more robust.
+    -   It automatically creates the `qlc` -> `qlc_latest` -> `qlc_vX.Y.Z/<mode>` symlink structure, removing the need for manual setup.
+    -   It now provides clear, actionable instructions on how to update your `PATH` if needed.
+-   **Enhanced HPC & Batch Job Support**:
+    -   The batch submission script (`sqlc`) is more reliable, no longer using hardcoded paths.
+    -   Shell scripts are now more compatible with typical HPC environments that may only have a `python3` executable.
+-   **Expanded Documentation**:
+    -   The `USAGE.md` guide now includes comprehensive, exhaustive lists of currently available plotting regions, observation datasets, and supported chemical/meteorological variables.
+    -   A new "Advanced Workflow" section has been added to `USAGE.md`, explaining the underlying shell script pipeline, the `param/ncvar/myvar` variable mapping system, and how to use your own data with the `qlc-py` engine.
+    -   Added a note on the future integration with the GHOST database.
+-   **Dependency Fix**: The `adjustText` library is now included as a core dependency.
+
+---
+
 ## What's New in v0.3.26
 
 This version introduces a completely new, high-performance Python processing engine and a more robust installation system.
@@ -56,18 +74,43 @@ qlc b2ro b2rn 2018-12-01 2018-12-21
 
 ---
 
+## Prerequisites
+
+Before running the QLC suite, please ensure the following system-level software is installed and accessible in your environment's `PATH`:
+
+- **`pdflatex`**: Required for generating the final PDF reports. It is part of the **TeX Live** distribution.
+- **`CDO` (Climate Data Operators)**: Used for processing NetCDF data.
+- **`eccodes`**: The ECMWF library for decoding and encoding GRIB files.
+- **`netcdf`**: The core NetCDF libraries.
+
+On HPC systems, these tools are typically made available by loading the appropriate modules (e.g., `module load cdo`). On personal machines, they can be installed using system package managers like `apt-get` (Debian/Ubuntu), `yum` (Red Hat/CentOS), or `brew` (macOS).
+
+---
+
 ## Installation and Configuration
 
 ### Standard Installation
 
 QLC is installed from PyPI. After the `pip install`, you **must** run `qlc-install` to set up the necessary local directory structure.
 
+**First-Time Installation**
+```bash
+pip install rc-qlc
+```
+
+**Upgrading an Existing Installation**
+To ensure you have the latest version, always use the `--upgrade` flag:
+```bash
+pip install --upgrade rc-qlc
+```
+
+After installing, set up your desired environment:
 ```bash
 # For a standalone test environment with example data
-pip install rc-qlc && qlc-install --mode test
+qlc-install --mode test
 
 # For an operational CAMS environment
-pip install rc-qlc && qlc-install --mode cams
+qlc-install --mode cams
 ```
 
 ### Installation in Restricted Environments (HPC/ATOS)
@@ -143,10 +186,95 @@ For advanced development, you can also use `--mode interactive`, which requires
 qlc-install --mode interactive --config /path/to/your/custom_qlc.conf
 ```
 
+## Advanced Topics
+
+### Installing PyFerret for Global Plots
+
+The `qlc_C5.sh` script, which generates global map plots, requires the `pyferret` library. This is an optional dependency.
+
+-   **To install with `pyferret` support:**
+    ```bash
+    pip install "rc-qlc[ferret]"
+    ```
+-   **If you do not need these plots**, you can either skip the `pyferret` installation or, if it's already installed, disable the script by commenting out `"C5"` in the `SUBSCRIPT_NAMES` array in your `$HOME/qlc/config/qlc.conf` file.
+-   **For HPC environments**, `pyferret` is often available as a module that can be loaded (e.g., `module load ferret/7.6.3`).
+
+### Manual PyFerret Installation for macOS / Apple Silicon
+
+If you are using a Mac with Apple Silicon (M1/M2/M3) or if the standard installation fails, `pyferret` may require a manual setup using a dedicated `conda` environment. `pip` installations are not recommended for this package on macOS as they may not work correctly with the ARM architecture.
+
+The most reliable method is to use `conda` with the Rosetta 2 translation layer.
+
+**1. (If needed) Install Conda**
+If you do not have `conda` installed, we recommend **Miniforge**, which is a minimal installer that is optimized for Apple Silicon and includes the high-performance `mamba` package manager.
+```bash
+# Download and run the installer for Apple Silicon
+curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh"
+bash Miniforge3-MacOSX-arm64.sh
+# Follow the prompts and restart your terminal after installation
+```
+
+**2. Create a Dedicated x86_64 Environment for PyFerret**
+This command creates a new `conda` environment named `pyferret_env` and installs the Intel (`x86_64`) version of `pyferret`, which will run seamlessly on Apple Silicon via Rosetta 2. It also pins `numpy` to a version older than 2.0 to ensure compatibility.
+
+```bash
+CONDA_SUBDIR=osx-64 conda create -n pyferret_env -c conda-forge pyferret ferret_datasets "numpy<2" --yes
+```
+
+**3. Configure QLC to Use the New Environment**
+The QLC scripts need to know where to find this new `pyferret` installation. You can achieve this by modifying the `qlc_C5.sh` script to activate the environment.
+
+Open the file `$HOME/qlc/sh/qlc_C5.sh` and add the following lines near the top, after `source $FUNCTIONS`:
+
+```bash
+# ... after 'source $FUNCTIONS'
+# Activate the dedicated conda environment for pyferret
+if [ -f "$HOME/miniforge3/bin/activate" ]; then
+    . "$HOME/miniforge3/bin/activate"
+    conda activate pyferret_env
+fi
+# ... rest of the script
+```
+*Note: The path to the activate script may differ if you installed Anaconda/Miniforge in a custom location.*
+
+### MARS Data Retrieval
+The `qlc_A1.sh` script is responsible for retrieving data from the ECMWF MARS archive. It uses a mapping system to associate the experiment prefix with a MARS `class`. 
+
+By default, the script is configured for `nl` (Netherlands), `be` (Belgium), and `rd` (Research Department) experiments. If you are working with data from other classes (e.g., `fr` for France, `de` for Germany), you will need to manually edit `$HOME/qlc/sh/qlc_A1.sh` and uncomment / edit the corresponding `XCLASS` line to ensure data is retrieved correctly.
+
+---
+
+## Troubleshooting
+
+### macOS "Permission Denied" or Quarantine Issues
+
+On macOS, the Gatekeeper security feature may "quarantine" files, including shell scripts that have been downloaded or modified. This can prevent them from being executed, sometimes with a "Permission Denied" error, even if the file has the correct execute permissions (`+x`).
+
+This is most likely to occur if you manually edit the `qlc` shell scripts (`.sh` files) directly in their `site-packages` installation directory.
+
+To resolve this, you can manually remove the quarantine attribute from the script directory using the `xattr` command in your terminal.
+
+1.  **First, find the exact location of the `qlc` package:**
+    ```bash
+    pip show rc-qlc
+    ```
+    Look for the `Location:` line in the output. This is your `site-packages` path.
+
+2.  **Then, use the `xattr` command to remove the quarantine flag:**
+    Use the path from the previous step to build the full path to the `qlc/sh` directory.
+    ```bash
+    # The path will depend on your Python installation. Use the location from 'pip show'.
+    xattr -rd com.apple.quarantine /path/to/your/site-packages/qlc/sh/
+    ```
+
+This should immediately resolve the execution issues.
+
 ---
 
 ## License
 
 © ResearchConcepts io GmbH  
 Contact: [contact@researchconcepts.io](mailto:contact@researchconcepts.io)  
-MIT-compatible, source-restricted under private release until publication.
+MIT-compatible, source-restricted under private release until publication.
+
+---

qlc/doc/USAGE.md

@@ -95,3 +95,201 @@ You can customize this workflow by editing the variables in your main configurat
 | `EXP_LABELS` | Comma-separated labels for experiments, used in plot legends. Must match the order of experiments passed to `qlc`. | `"MyExp,MyREF"` |
 | `PLOTEXTENSION` | The file format for the output plots (e.g., `pdf`, `png`, ...). | `"png"` |
 
+---
+
+## Available `qlc-py` Options
+
+The Python engine is highly configurable through the `qlc.conf` file or a custom JSON configuration. Below is an overview of the most common options available.
+
+### Plotting Regions (`REGION`)
+
+You can set the `REGION` variable to any of the following codes to automatically set the map bounds for plots.
+
+| Category | Region Codes |
+| --- | --- |
+| **Continents & Global** | `Globe`, `EU` (Europe), `ASIA`, `AFRICA`, `NA` (North America), `SA` (South America), `OC` (Oceania), `ANT` (Antarctica), `NP` (North Pole) |
+| **Oceans & Water Bodies** | `PAC` (Pacific), `ATLA` (Atlantic), `INDO` (Indian), `ARC` (Arctic), `SOU` (Southern) |
+| **Major Deserts** | `SAH` (Sahara), `ARA` (Arabian), `GOBI`, `OUTBACK` (Australian), `ATACAMA` |
+| **Key Countries** | `GB` (Great Britain), `US` (United States), `CN` (China), `JP` (Japan), `SA` (Saudi Arabia), `IR` (Iran), `EG` (Egypt), `MA` (Morocco), `NG` (Nigeria), `KE` (Kenya), `ZA` (South Africa), `IS` (Iceland) |
+| **European Countries**| `DE` (Germany), `FR` (France), `IT` (Italy), `ES` (Spain), `PL` (Poland), `SE` (Sweden), `FI` (Finland), `NO` (Norway), `NL` (Netherlands), `BE` (Belgium), `AT` (Austria), `CH` (Switzerland), `CZ` (Czech Rep.), `GR` (Greece) |
+| **US States** | `CA-US` (California), `NY-US` (New York), `TX-US` (Texas), `FL-US` (Florida), `IL-US` (Illinois), `WA-US` (Washington), `CO-US` (Colorado), `AZ-US` (Arizona) |
+| **Specific Regions** | `MENA` (Middle East/North Africa), `SSA` (Sub-Saharan Africa) |
+| **Major Cities** | `LA-US`, `NYC-US`, `SHA-CN` (Shanghai), `BEI-CN` (Beijing), `TOK-JP` (Tokyo), `FR-PAR` (Paris), `DE-MUC` (Munich), `DE-FRA` (Frankfurt), `IT-ROM` (Rome), `ES-MAD` (Madrid), `PL-WAW` (Warsaw), `NL-AMS` (Amsterdam), `BE-BRU` (Brussels), `AT-VIE` (Vienna), `CZ-PRG` (Prague), `GR-ATH` (Athens), `CH-ZUR` (Zurich) |
+| **German Regions** | `BW-DE` (Baden-Württemberg), `B-DE` (Berlin), `HH-DE` (Hamburg), `FR-DE` (Freiburg) |
+
+### Time Averaging (`TIME_AVERAGE`)
+
+The `TIME_AVERAGE` variable controls the temporal aggregation of the time series data.
+
+- **`raw`**: No averaging is applied.
+- **`mean`**: The mean over the entire time period is calculated.
+- **Time Frequencies**: `1min`, `10min`, `30min`, `hourly`, `3hourly`, `6hourly`, `12hourly`, `daily`, `weekly`, `monthly`, `annual`, `seasonal`, `decadal`.
+
+### Observation Datasets (`OBS_DATASET_TYPE`)
+
+The `OBS_DATASET_TYPE` variable specifies which observation network data to use. The following datasets are supported:
+
+- `ebas` / `ebas_hourly` / `ebas_daily` (EBAS - European Monitoring and Evaluation Programme)
+- `airbase` / `airbase_ineris` (European air quality database)
+- `airnow` (U.S. EPA's AirNow program)
+- `castnet` (Clean Air Status and Trends Network)
+- `AMoN` (Ammonia Monitoring Network)
+- `NNDMN` (National Network of Deposition Monitoring in the Netherlands)
+- `china_gsv` / `china_aq` (Chinese air quality data)
+
+### Supported Variables (`variable`)
+
+The `qlc` pipeline automatically discovers variables from your data files. The system supports a wide range of chemical species and physical properties. The list below contains all variables recognized by the EBAS observation dataset mapping, which is the most extensive. Other datasets may support a subset of these.
+
+-   **Gases**: `NO2`, `SO2`, `SO4`, `HNO3`, `NO3`, `NH3`, `NH4`, `NO`, `NOx`, `O3`, `CO`, `HONO` (Nitrous Acid), `ethanal`, `methanol`, `ethene`, `ethyne`, `propene`, `benzene`, `ethane`, `propane`, `ethylbenzene`, `m-p-xylene`, `o-xylene`, `toluene`.
+-   **Mole Fractions**: `SO2_mf`, `NH3_mf`, `NO_mf`, `NO2_mf`, `NOx_mf`, `O3_mf`, `CO_mf`, `ethanal_mf`, `methanol_mf`, `ethene_mf`, `ethyne_mf`, `propene_mf`, `benzene_mf`, `ethane_mf`, `propane_mf`, `ethylbenzene_mf`, `m-p-xylene_mf`, `o-xylene_mf`, `toluene_mf`.
+-   **Halocarbons (Mole Fractions)**: `CCl4_mf`, `CH3Cl_mf`, `CH2Br2_mf`, `CH2Cl2_mf`, `CHCl3_mf`.
+-   **Greenhouse Gases (Mole Fractions)**: `carbon_dioxide_mf`, `methane_mf`, `hydrogen_mf`, `nitrous_oxide_mf`.
+-   **Aerosol Properties**:
+    -   **Mass Density**: `PM1`, `PM2.5`, `PM10`.
+    -   **Composition (Dry Aerosol)**: `Dry_Nitrate`, `Dry_Ammonium`, `Dry_Chloride`, `Dry_Calcium`, `Dry_Sodium`, `Dry_Iron`, `Dry_Sulphate_Corrected`.
+    -   **Composition (PM2.5)**: `PM2.5_Nitrate`, `PM2.5_Sodium`, `PM2.5_Calcium`, `PM2.5_Ammonium`, `PM2.5_Chloride`, `PM2.5_Total_Sulphate`, `PM2.5_Sulphate_Corrected`, `PM2.5_EC` (Elemental Carbon), `PM2.5_OC` (Organic Carbon), `PM2.5_TC` (Total Carbon).
+    -   **Composition (PM10)**: `PM10_Nitrate`, `PM10_Sodium`, `PM10_Calcium`, `PM10_Ammonium`, `PM10_Chloride`, `PM10_Lead`, `PM10_Iron`, `PM10_Manganese`, `PM10_Total_Sulphate`, `PM10_Sulphate_Corrected`, `PM10_EC`, `PM10_OC`, `PM10_TC`.
+    -   **Number Concentration**: `Dry_NA_NumConc`.
+-   **Optical Properties**:
+    -   **Aerosol Optical Depth (AOD)**: `AOD_380`, `AOD_500`, `AOD_675` (and many other wavelengths).
+    -   **Scattering Coefficient**: `Scatt_450`, `Scatt_525`, `Scatt_550`, `Scatt_635`, `Scatt_700`.
+    -   **Absorption Coefficient**: `Abs_370`, `Abs_470`, `Abs_520`, `Abs_660`, `Abs_880` (and many other wavelengths between 370nm and 950nm).
+    -   **Backscattering**: `Backscatt_700`.
+-   **Meteorology**: `Pressure`, `Temperature`.
+
+#### Model-Specific Variables
+
+The following variables are specifically mapped for model (`mod`) data types. These often include different aerosol size bins or speciated components.
+
+-   **Aerosol Mass**: `PM1`, `PM2.5`, `PM10`
+-   **Aerosol Number Concentration**: `N`, `N_ks` (nucleation mode), `N_as` (aitken mode), `N_cs` (coarse mode)
+-   **Sulphate Species**: `SO4`, `SO4_ks`, `SO4_as`, `SO4_cs`
+-   **Ammonium Species**: `NH4`, `NH4_ks`, `NH4_as`, `NH4_cs`
+-   **Nitrate Species**: `NO3`, `NO3a`, `NO3b`, `NO3_ks`, `NO3_as`, `NO3_cs`
+-   **Gases (Mass Mixing Ratios)**: `NH3`, `HNO3`, `NO2`, `SO2`, `CO`, `O3`
+
+*Note: The list of supported variables is actively being expanded. Future releases will include a more comprehensive mapping and direct integration with the GHOST (Globally Harmonised Observations in Space and Time) database [[Bowdalo et al., 2024]](https://essd.copernicus.org/articles/16/4417/2024/).*
+
+---
+
+## Advanced `qlc-py` Configuration
+
+The `qlc-py` engine offers several advanced configuration options for more complex analysis workflows.
+
+### Using Custom Station Lists
+
+For targeted analysis, you can provide a custom list of stations via the `station_file` parameter in your configuration. This should be a path to a CSV file containing station metadata (e.g., ID, name, latitude, longitude).
+
+-   This is useful for focusing on specific station types (e.g., urban, rural) or networks.
+-   If a station from your list is not found in the observation dataset for a given period, it will still be included in the model-only analysis and plots.
+-   In addition to plots showing the average across all stations, you can configure `qlc-py` to generate plots and statistics for each individual station in the list.
+
+### Multi-Entry and Parallel Processing
+
+The JSON configuration file passed to `qlc-py` can be a single JSON object or an array of multiple objects. This enables powerful and flexible workflow designs.
+
+-   **Serial Processing**: If you provide an array of configuration objects, `qlc-py` will process them sequentially. This is useful for workflows with distinct steps, such as:
+    1.  An entry for processing observations only.
+    2.  An entry for processing model results only.
+    3.  A final entry that uses both outputs for a combined collocation analysis.
+-   **Parallel Processing**: Within a single configuration entry, you can enable parallel processing for time-consuming tasks (like loading and processing many model files at once) by setting `"multiprocessing": true`.
+
+### Example: Multi-Variable Observation Configuration
+
+Below is an example of a single configuration entry for processing daily EBAS observations for two variables (`NH3` and `NH4_as`). It also includes the optional `global_attributes` block, which allows you to embed custom metadata into the output NetCDF files.
+
+```json
+{
+    "name": "CAMS",
+    "logdir": "./log",
+    "workdir": "./run",
+    "output_base_name": "$HOME/qlc/Plots/PY",
+    "station_file": "$HOME/qlc/obs/data/ebas_station-locations.csv",
+    "obs_path": "$HOME/qlc/obs/data/ver0d",
+    "obs_dataset_type": "ebas_daily",
+    "obs_dataset_version": "latest",
+    "start_date": "2018-12-01",
+    "end_date": "2018-12-21",
+    "variable": "NH3,NH4_as",
+    "station_radius_deg": 0.5,
+    "plot_type": "",
+    "plot_region": "EU",
+    "time_average": "daily",
+    "station_plot_group_size": 5,
+    "show_stations": false,
+    "show_min_max": true,
+    "log_y_axis": false,
+    "fix_y_axis": true,
+    "show_station_map": true,
+    "load_station_timeseries_obs": true,
+    "show_station_timeseries_obs": true,
+    "show_station_timeseries_mod": false,
+    "show_station_timeseries_com": false,
+    "save_plot_format": "pdf",
+    "save_data_format": "nc",
+    "multiprocessing": false,
+    "n_threads": "20",
+    "debug": false,
+    "global_attributes": {
+      "title": "Air pollutants over Europe, SO2,SO4,HNO3,NO3,NH3,NH4",
+      "summary": "Custom summary for netCDF output: Ebas daily observations for selected EU stations.",
+      "author": "Swen Metzger, sm@researchconcepts.io",
+      "history": "Processed for CAMS2_35bis (qlc_v0.3.27)",
+      "Conventions": "CF-1.8"
+    }
+}
+```
+
+---
+
+## Advanced Workflow: Data Processing and Configuration
+
+For advanced users, it is helpful to understand the underlying data processing pipeline, which is controlled by a series of shell scripts and configured via `qlc.conf`. This allows for significant customization of data retrieval and analysis.
+
+### The Shell Script Pipeline
+
+When you run the main `qlc` command, it executes a chain of scripts to process data. The scripts to be run are defined by the `SUBSCRIPT_NAMES` array in `$HOME/qlc/config/qlc.conf`. A typical workflow is:
+
+1.  **`qlc_A1.sh`**: Handles data retrieval from the MARS archive. It fetches the required variables in GRIB format. The specific variables are defined by the `MARS_RETRIEVALS` array in `qlc.conf`, which must correspond to entries in the `nml/mars_*.nml` namelist files.
+2.  **`qlc_B1a.sh`**: Converts the retrieved GRIB files into NetCDF format.
+3.  **`qlc_B2.sh`**: Performs post-processing and variable substitution on the NetCDF files. This step is crucial as it renames the variables to the user-friendly names (`myvar_*`) expected by the plotting scripts.
+4.  **`qlc_D1.sh`**: Drives the station time-series analysis by generating a configuration and calling `qlc-py`.
+5.  **`qlc_C5.sh`**: Generates global overview plots, including 3D surface maps, vertical integrals (burden), and zonal/meridional means for selected variables.
+
+The raw data retrieved from MARS is stored in the `$HOME/qlc/Results` directory, while the final, post-processed NetCDF files used for analysis are placed in `$HOME/qlc/Analysis`.
+
+### Variable Mapping Explained
+
+QLC uses a flexible three-part system defined in `qlc.conf` to map variables from the MARS archive to user-defined names for plotting.
+
+-   **`param_*`**: This is the official ECMWF parameter ID from the GRIB tables (e.g., `param_A1_sfc="73.210"`). This value is required by MARS for data retrieval and must be correct.
+-   **`ncvar_*`**: This is the short name that is automatically assigned to the variable when the GRIB file is converted to NetCDF (e.g., `ncvar_A1_sfc="var73"`). This name can differ depending on the data type (surface, pressure levels, model levels).
+-   **`myvar_*`**: This is the final, user-defined name for the variable (e.g., `myvar_A1_sfc="PM25"`). This is the name that will be used in plot labels, titles, and filenames throughout the QLC system.
+
+This system allows you to work with consistent, human-readable variable names (`PM25`) while ensuring the underlying retrieval from MARS uses the correct, official parameter codes.
+
+### Automatic Unit Conversion and Collocation
+
+During the collocation step (comparing model data to observations), `qlc-py` automatically handles variable mapping and unit conversion. For example, when comparing model output (`mod`) to `castnet` observations:
+
+-   If a model variable named `SO4` or `SO4_as` (in units of `kg/kg`) is being compared to a `castnet` observation variable also named `SO4` (in units of `ug/m3`), the system will automatically convert the model data to `ug/m3` before calculating statistics. The observational unit is always treated as the target for conversion.
+
+### Using Custom Data with `qlc-py`
+
+While the shell script pipeline is designed for a seamless workflow, you can also use `qlc-py` as a standalone tool with your own NetCDF data. Simply provide absolute paths to your data files in the `mod_path` and `obs_path` fields of your JSON configuration. The data must be CF-compliant, but this allows you to bypass the MARS retrieval and processing steps entirely.
+
+### Example Report
+
+An example of the final PDF report generated by the `qlc` pipeline can be found in the `$HOME/qlc/doc/` directory. This provides a complete overview of all selected plots and analyses produced. The `$HOME/qlc/Presentations/` directory contains the report of a successful run.
+
+### Accessing Raw Outputs for Custom Analysis
+
+The QLC pipeline generates a wide range of outputs that can be used for further analysis outside of the main workflow.
+
+-   **Comprehensive Plots**: The `$HOME/qlc/Plots` directory contains all of the individual plots created during the run, e.g., in higher resolution or with more detail than what is included in the final summary report.
+-   **Exportable Data**: You can configure `qlc-py` to save the intermediate, collocated data in either NetCDF (`.nc`) or CSV (`.csv`) format.
+-   **Multiple Plot Formats**: Plots can be saved in various formats, including `.pdf`, `.png`, and `.jpg`.
+
+This flexibility allows you to easily import QLC-processed data and graphics into your own analysis scripts, presentations, or reports.
+

qlc/install.py

@@ -386,23 +386,28 @@ def setup(mode: str, version: str, debug: bool = False, config_file: str = None)
     generic_config_path = config_dst / "qlc.conf"
     update_qlc_version(generic_config_path, version)
 
-    # --- Setup master symlinks to point to this installation (relative) ---
-    # Link qlc_latest to the new version-specific root
-    qlc_latest_link = Path(user_home) / "qlc_latest"
-    safe_move_and_link(root, qlc_latest_link, relative=True, backup=False)
-
-    # Ensure $HOME/qlc points to qlc_latest
-    qlc_stable_link = Path(user_home) / "qlc"
-    if not qlc_stable_link.exists() or not qlc_stable_link.is_symlink():
-         safe_move_and_link(qlc_latest_link, qlc_stable_link, relative=True, backup=False)
-    else:
-        try:
-            if not qlc_stable_link.resolve() == qlc_latest_link.resolve():
-                safe_move_and_link(qlc_latest_link, qlc_stable_link, relative=True, backup=False)
-            else:
-                print(f"[SKIP] {qlc_stable_link} already links to {qlc_latest_link.name}")
-        except FileNotFoundError: # Handle broken link
-            safe_move_and_link(qlc_latest_link, qlc_stable_link, relative=True, backup=False)
+    # --- Setup master symlinks to point to this installation ---
+
+    qlc_latest_link = user_home / "qlc_latest"
+    qlc_stable_link = user_home / "qlc"
+
+    # Forcefully remove existing links to ensure a clean state
+    print(f"[LINK] Removing existing master links if they exist: {qlc_stable_link.name}, {qlc_latest_link.name}")
+    qlc_stable_link.unlink(missing_ok=True)
+    qlc_latest_link.unlink(missing_ok=True)
+
+    # Create qlc_latest -> qlc_vX.Y.Z/<mode> (relative link)
+    # Target path is relative to the link's location ($HOME)
+    latest_target = os.path.relpath(root, user_home)
+    print(f"[LINK] Creating master link: {qlc_latest_link} -> {latest_target}")
+    qlc_latest_link.symlink_to(latest_target, target_is_directory=True)
+
+    # Create qlc -> qlc_latest (relative link)
+    # Target path is relative to the link's location ($HOME)
+    stable_target = os.path.relpath(qlc_latest_link, user_home)
+    print(f"[LINK] Creating stable link: {qlc_stable_link} -> {stable_target}")
+    qlc_stable_link.symlink_to(stable_target, target_is_directory=True)
+
 
     # Write install info
     info = {
@@ -418,8 +423,14 @@ def setup(mode: str, version: str, debug: bool = False, config_file: str = None)
     update_qlc_version(qlc_stable_link / "config" / "qlc.conf", version)
 
     print("\n[INFO] QLC installation complete.")
-    print("[INFO] To get started, you may need to open a new terminal or run 'rehash'.")
     print("[INFO] The following commands are now available: qlc, qlc-py, sqlc, qlc-install")
+    print("\n[ACTION REQUIRED]")
+    print("To make these commands available, you may need to add the local bin directory to your PATH.")
+    print("For most bash users, you can do this by running the following command:")
+    print("  echo 'export PATH=\"$HOME/.local/bin:$PATH\"' >> ~/.bashrc")
+    print("\nFor other shells (like zsh), you may need to add this line to '~/.zshrc' instead.")
+    print("After running the command, please open a new terminal or run 'source ~/.bashrc' to apply the changes.")
+
 
 if __name__ == "__main__":
     parser = argparse.ArgumentParser(description="Install QLC runtime structure")

qlc/sh/qlc_A1.sh

@@ -38,24 +38,25 @@ for exp in $exps ; do
   # This is required for retrieving data from the MARS archive.
   EXPCLASS=$(echo "${exp}" | cut -c 1)
  
+  # For experiments other than 'nl', 'be', or 'rd', please uncomment the corresponding XCLASS line.
   case "${EXPCLASS}" in
     a) XCLASS="be" ;;  # Belgium
     b) XCLASS="nl" ;;  # Netherlands
-    c) XCLASS="fr" ;;  # France
-    d) XCLASS="de" ;;  # Germany
-    e) XCLASS="es" ;;  # Spain
-    f) XCLASS="fi" ;;  # Finland
-    g) XCLASS="gr" ;;  # Greece
-    h) XCLASS="hu" ;;  # Hungary
-    i) XCLASS="it" ;;  # Italy
-    k) XCLASS="dk" ;;  # Denmark
-    l) XCLASS="pt" ;;  # Portugal
-    m) XCLASS="at" ;;  # Austria
-    n) XCLASS="no" ;;  # Norway
-    s) XCLASS="se" ;;  # Sweden
-    t) XCLASS="tr" ;;  # Turkey
-    u) XCLASS="uk" ;;  # United Kingdom
-    w) XCLASS="ch" ;;  # Switzerland
+#     c) XCLASS="fr" ;;  # France
+#     d) XCLASS="de" ;;  # Germany
+#     e) XCLASS="es" ;;  # Spain
+#     f) XCLASS="fi" ;;  # Finland
+#     g) XCLASS="gr" ;;  # Greece
+#     h) XCLASS="hu" ;;  # Hungary
+#     i) XCLASS="it" ;;  # Italy
+#     k) XCLASS="dk" ;;  # Denmark
+#     l) XCLASS="pt" ;;  # Portugal
+#     m) XCLASS="at" ;;  # Austria
+#     n) XCLASS="no" ;;  # Norway
+#     s) XCLASS="se" ;;  # Sweden
+#     t) XCLASS="tr" ;;  # Turkey
+#     u) XCLASS="uk" ;;  # United Kingdom
+#     w) XCLASS="ch" ;;  # Switzerland
     *) XCLASS="rd" ;;  # Default to Research Department
   esac
   log "${exp}" "${XCLASS}"

qlc/sh/qlc_C5.sh

@@ -23,21 +23,56 @@ done
 log "$0 ANALYSIS_DIRECTORY = $ANALYSIS_DIRECTORY"
 
 # module load for ATOS
-myOS="`uname -s`"
-HOST=`hostname -s  | awk '{printf $1}' | cut -c 1`
-log   "HOST = ${HOST} | `hostname -s`"
-log   "myOS = ${myOS}"
-if [  "${HOST}" == "a" ] && [ "${myOS}" != "Darwin" ]; then
-	module load ferret/7.6.3
+myOS="$(uname -s)"
+HOST="$(hostname -s | cut -c 1)"
+log "HOST = ${HOST} | $(hostname -s)"
+log "myOS = ${myOS}"
+
+if [ "${HOST}" = "a" ] && [ "${myOS}" != "Darwin" ]; then
+    # Only on ATOS-like hosts (non-macOS, short host starting with 'a')
+    if command -v module >/dev/null 2>&1; then
+        module load ferret/7.6.3
+    else
+        log "[WARN] 'module' command not found; skipping ferret module load"
+    fi
+else
+    # Conda activation for pyferret_env
+    if [ -f "$HOME/miniforge3/etc/profile.d/conda.sh" ]; then
+        . "$HOME/miniforge3/etc/profile.d/conda.sh"
+        conda activate pyferret_env
+    elif [ -f "$HOME/miniforge3/bin/activate" ]; then
+        . "$HOME/miniforge3/bin/activate" "pyferret_env"
+    else
+        log "[WARN] Conda activation scripts not found; relying on PATH for pyferret"
+    fi
 fi
 
-# Check if pyferret exists
-if ! command_exists pyferret; then
-  log  "Error: pyferret command not found" >&2
-  exit 1
-else
-  log  "Success: pyferret command found"
-  which pyferret
+# Determine the pyferret command to use.
+# First, check if pyferret is in the PATH (e.g., from "pip install 'rc-qlc[ferret]'").
+# If not, fall back to the direct path in the conda environment as per the README.
+PYFERRET_CMD=""
+if command -v pyferret &> /dev/null; then
+    log "Using 'pyferret' found in system PATH."
+    PYFERRET_CMD="pyferret"
+elif [ -x "$HOME/miniforge3/envs/pyferret_env/bin/pyferret" ]; then
+    log "Using 'pyferret' from dedicated conda environment."
+    PYFERRET_CMD="$HOME/miniforge3/envs/pyferret_env/bin/pyferret"
+fi
+
+# Check if we found a valid pyferret command
+if [ -z "$PYFERRET_CMD" ]; then
+  log  "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
+  log  "WARNING: pyferret command not found."
+  log  "The qlc_C5.sh script requires pyferret for generating global map plots."
+  log  "You can install it as an optional dependency with:"
+  log  "    pip install 'rc-qlc[ferret]'"
+  log  "For more detailed instructions (e.g., for macOS), please see the"
+  log  "'Advanced Topics' section in the main README.md file."
+  log  "Alternatively, you can disable this script by commenting out 'C5' in the"
+  log  "SUBSCRIPT_NAMES array in your qlc.conf file."
+  log  "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"
+  log  "Skipping pyferret plots and exiting script gracefully."
+  exit 0
 fi
 
 # Create output directory if not existent
@@ -612,51 +647,51 @@ EOF
 			if [ -f  "$ferret.jnl" ]; then
 			    cp -p $ferret.jnl  .
 				rm -f ${tfile}_burden.${ext} ${tfile}_burden_log.${ext}
-				log "pyferret -nodisplay -script $ferret.jnl ${ifile} ${pvar} ${MODEL_RESOLUTION} ${exp} ${tfile}_burden ${ext}"
-					 pyferret -nodisplay -script $ferret.jnl ${ifile} ${pvar} ${MODEL_RESOLUTION} ${exp} ${tfile}_burden ${ext}
+				log "$PYFERRET_CMD -nodisplay -script $ferret.jnl ${ifile} ${pvar} ${MODEL_RESOLUTION} ${exp} ${tfile}_burden ${ext}"
+					 $PYFERRET_CMD -nodisplay -script $ferret.jnl ${ifile} ${pvar} ${MODEL_RESOLUTION} ${exp} ${tfile}_burden ${ext}
 			else
 			 	# ferret plots for default journal file
 				rm -f ${tfile}_burden.${ext} ${tfile}_burden_log.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_burden_1x1.jnl"
-					 pyferret -nodisplay -script  ${tfile}_burden_1x1.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_burden_1x1.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_burden_1x1.jnl
 
 				rm -f ${tfile}_zonal.${ext} ${tfile}_zonal_log.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_zonal_1x1.jnl"
-					 pyferret -nodisplay -script  ${tfile}_zonal_1x1.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_zonal_1x1.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_zonal_1x1.jnl
 
 				rm -f ${tfile}_meridional.${ext}  ${tfile}_meridional_log.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_meridional_1x1.jnl"
-					 pyferret -nodisplay -script  ${tfile}_meridional_1x1.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_meridional_1x1.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_meridional_1x1.jnl
 
 				rm -f ${tfile}_surface.${ext}     ${tfile}_surface_log.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_surface_1x1.jnl"
-					 pyferret -nodisplay -script  ${tfile}_surface_1x1.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_surface_1x1.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_surface_1x1.jnl
 
 				rm -f ${tfile}_utls.${ext}        ${tfile}_utls_log.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_utls_1x1.jnl"
-					 pyferret -nodisplay -script  ${tfile}_utls_1x1.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_utls_1x1.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_utls_1x1.jnl
 			fi
 					 
 			if [ "${exp}" == "${exp1}" ]; then
 				rm -f ${tfile}_burden_diff.${ext} ${tfile}_burden_log_diff.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_burden_1x1_diff.jnl"
-					 pyferret -nodisplay -script  ${tfile}_burden_1x1_diff.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_burden_1x1_diff.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_burden_1x1_diff.jnl
 
 				rm -f ${tfile}_zonal_diff.${ext}  ${tfile}_zonal_log_diff.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_zonal_1x1_diff.jnl"
-					 pyferret -nodisplay -script  ${tfile}_zonal_1x1_diff.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_zonal_1x1_diff.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_zonal_1x1_diff.jnl
 
 				rm -f ${tfile}_meridional_diff.${ext} ${tfile}_meridional_log_diff.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_meridional_1x1_diff.jnl"
-					 pyferret -nodisplay -script  ${tfile}_meridional_1x1_diff.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_meridional_1x1_diff.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_meridional_1x1_diff.jnl
 
 				rm -f ${tfile}_surface_diff.${ext} ${tfile}_surface_log_diff.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_surface_1x1_diff.jnl"
-					 pyferret -nodisplay -script  ${tfile}_surface_1x1_diff.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_surface_1x1_diff.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_surface_1x1_diff.jnl
 
 				rm -f ${tfile}_utls_diff.${ext}   ${tfile}_utls_log_diff.${ext}
-				log "pyferret -nodisplay -script  ${tfile}_utls_1x1_diff.jnl"
-					 pyferret -nodisplay -script  ${tfile}_utls_1x1_diff.jnl
+				log "$PYFERRET_CMD -nodisplay -script  ${tfile}_utls_1x1_diff.jnl"
+					 $PYFERRET_CMD -nodisplay -script  ${tfile}_utls_1x1_diff.jnl
 
 #				files=("${tfile}" "${tfile}_log" "${tfile}_diff" "${tfile}_log_diff")
 				files=("${tfile}_surface"    "${tfile}_surface_log"    "${tfile}_surface_diff"    "${tfile}_surface_log_diff"    \
@@ -801,6 +836,7 @@ cat >> ${texFile} <<EOF
 EOF
 fi
 done # plot
+
 log  "----------------------------------------------------------------------------------------"
 log "${texFile}"
 cat  ${texFile}

qlc/sh/qlc_D1.sh

@@ -31,15 +31,6 @@ if [  "${HOST}" == "a" ] && [ "${myOS}" != "Darwin" ]; then
 module load python3/3.10.10-01
 fi
 
-# Check if python exists
-if ! command_exists python; then
-  log  "Error: python command not found" >&2
-  exit 1
-else
-  log  "Success: python command found"
-  which python
-fi
-
 # Check if qlc-py exists
 if ! command_exists qlc-py; then
   log  "Error: qlc-py command not found" >&2

qlc/sh/qlc_batch.sh

@@ -40,9 +40,9 @@ cat > $HOME/qlc/run/qlc_batch.sh$$<<EOF
 #SBATCH --output=log-%J.out
 #SBATCH --error=err-%J.out
 #SBATCH --export=ALL
-$HOME/qlc/bin/qlc $1 $2 $3 $4 $5
+qlc $1 $2 $3 $4 $5
 echo "SLURM_JOB_ID = ${jobid}"
-sbatch --dependency=afterok:${jobid} --mail-user=$USER@ecmwf.int $HOME/qlc/bin/qlc $1 $2 $3 $4
+sbatch --dependency=afterok:${jobid} --mail-user=$USER@ecmwf.int qlc $1 $2 $3 $4
 EOF
 else
 cat > $HOME/qlc/run/qlc_batch.sh$$<<EOF
@@ -50,7 +50,7 @@ cat > $HOME/qlc/run/qlc_batch.sh$$<<EOF
 #SBATCH --job-name=$HOME/qlc/run/qlc_batch.sh$$
 #SBATCH --output=log-%J.out
 #SBATCH --error=err-%J.out
-$HOME/qlc/bin/qlc $1 $2 $3 $4
+qlc $1 $2 $3 $4
 EOF
 fi
 sbatch $HOME/qlc/run/qlc_batch.sh$$

qlc/sh/qlc_main.sh

@@ -1,6 +1,23 @@
 #!/bin/bash -e
 umask 0022
 
+# --- Start: Environment Setup ---
+# Find the Python executable that runs this tool.
+# This ensures that any executables installed in the same environment (like pyferret) are found.
+# Fallback to 'python3' if 'qlc' is not in the path (e.g., during development).
+PYTHON_CMD=$(which python3)
+if command -v qlc >/dev/null 2>&1; then
+    QLC_PATH=$(which qlc)
+    PYTHON_CMD=$(head -n 1 "$QLC_PATH" | sed 's/^#!//')
+fi
+
+# Get the directory of the Python executable.
+PYTHON_BIN_DIR=$(dirname "$PYTHON_CMD")
+
+# Prepend this directory to the PATH for this script and all subscripts.
+export PATH="$PYTHON_BIN_DIR:$PATH"
+# --- End: Environment Setup ---
+
 ARCH="`uname -m`"
 myOS="`uname -s`"
 HOST="`hostname -s`"

{rc_qlc-0.3.26.dist-info → rc_qlc-0.3.27.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: rc-qlc
-Version: 0.3.26
+Version: 0.3.27
 Summary: Quick Look Content (QLC): Model–Observation Comparison Suite for Use with CAMS
 Home-page: https://pypi.org/project/qlc/
 Author-email: ResearchConcepts io GmbH <contact@researchconcepts.io>
@@ -16,10 +16,13 @@ Requires-Dist: netCDF4
 Requires-Dist: scipy
 Requires-Dist: cartopy
 Requires-Dist: tqdm
+Requires-Dist: adjustText
 Provides-Extra: build
 Requires-Dist: tomli; extra == "build"
 Provides-Extra: test
 Requires-Dist: pytest; extra == "test"
+Provides-Extra: ferret
+Requires-Dist: pyferret; extra == "ferret"
 Provides-Extra: cams
 Dynamic: home-page
 Dynamic: license-file
@@ -36,6 +39,24 @@ The suite streamlines the entire post-processing workflow, from data retrieval a
 
 ---
 
+## What's New in v0.3.27
+
+This release focuses on improving the out-of-the-box installation experience, especially for HPC environments, and significantly expanding the user documentation.
+
+-   **Installer Overhaul**: The `qlc-install` script is now more robust.
+    -   It automatically creates the `qlc` -> `qlc_latest` -> `qlc_vX.Y.Z/<mode>` symlink structure, removing the need for manual setup.
+    -   It now provides clear, actionable instructions on how to update your `PATH` if needed.
+-   **Enhanced HPC & Batch Job Support**:
+    -   The batch submission script (`sqlc`) is more reliable, no longer using hardcoded paths.
+    -   Shell scripts are now more compatible with typical HPC environments that may only have a `python3` executable.
+-   **Expanded Documentation**:
+    -   The `USAGE.md` guide now includes comprehensive, exhaustive lists of currently available plotting regions, observation datasets, and supported chemical/meteorological variables.
+    -   A new "Advanced Workflow" section has been added to `USAGE.md`, explaining the underlying shell script pipeline, the `param/ncvar/myvar` variable mapping system, and how to use your own data with the `qlc-py` engine.
+    -   Added a note on the future integration with the GHOST database.
+-   **Dependency Fix**: The `adjustText` library is now included as a core dependency.
+
+---
+
 ## What's New in v0.3.26
 
 This version introduces a completely new, high-performance Python processing engine and a more robust installation system.
@@ -82,18 +103,43 @@ qlc b2ro b2rn 2018-12-01 2018-12-21
 
 ---
 
+## Prerequisites
+
+Before running the QLC suite, please ensure the following system-level software is installed and accessible in your environment's `PATH`:
+
+- **`pdflatex`**: Required for generating the final PDF reports. It is part of the **TeX Live** distribution.
+- **`CDO` (Climate Data Operators)**: Used for processing NetCDF data.
+- **`eccodes`**: The ECMWF library for decoding and encoding GRIB files.
+- **`netcdf`**: The core NetCDF libraries.
+
+On HPC systems, these tools are typically made available by loading the appropriate modules (e.g., `module load cdo`). On personal machines, they can be installed using system package managers like `apt-get` (Debian/Ubuntu), `yum` (Red Hat/CentOS), or `brew` (macOS).
+
+---
+
 ## Installation and Configuration
 
 ### Standard Installation
 
 QLC is installed from PyPI. After the `pip install`, you **must** run `qlc-install` to set up the necessary local directory structure.
 
+**First-Time Installation**
+```bash
+pip install rc-qlc
+```
+
+**Upgrading an Existing Installation**
+To ensure you have the latest version, always use the `--upgrade` flag:
+```bash
+pip install --upgrade rc-qlc
+```
+
+After installing, set up your desired environment:
 ```bash
 # For a standalone test environment with example data
-pip install rc-qlc && qlc-install --mode test
+qlc-install --mode test
 
 # For an operational CAMS environment
-pip install rc-qlc && qlc-install --mode cams
+qlc-install --mode cams
 ```
 
 ### Installation in Restricted Environments (HPC/ATOS)
@@ -169,6 +215,89 @@ For advanced development, you can also use `--mode interactive`, which requires
 qlc-install --mode interactive --config /path/to/your/custom_qlc.conf
 ```
 
+## Advanced Topics
+
+### Installing PyFerret for Global Plots
+
+The `qlc_C5.sh` script, which generates global map plots, requires the `pyferret` library. This is an optional dependency.
+
+-   **To install with `pyferret` support:**
+    ```bash
+    pip install "rc-qlc[ferret]"
+    ```
+-   **If you do not need these plots**, you can either skip the `pyferret` installation or, if it's already installed, disable the script by commenting out `"C5"` in the `SUBSCRIPT_NAMES` array in your `$HOME/qlc/config/qlc.conf` file.
+-   **For HPC environments**, `pyferret` is often available as a module that can be loaded (e.g., `module load ferret/7.6.3`).
+
+### Manual PyFerret Installation for macOS / Apple Silicon
+
+If you are using a Mac with Apple Silicon (M1/M2/M3) or if the standard installation fails, `pyferret` may require a manual setup using a dedicated `conda` environment. `pip` installations are not recommended for this package on macOS as they may not work correctly with the ARM architecture.
+
+The most reliable method is to use `conda` with the Rosetta 2 translation layer.
+
+**1. (If needed) Install Conda**
+If you do not have `conda` installed, we recommend **Miniforge**, which is a minimal installer that is optimized for Apple Silicon and includes the high-performance `mamba` package manager.
+```bash
+# Download and run the installer for Apple Silicon
+curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh"
+bash Miniforge3-MacOSX-arm64.sh
+# Follow the prompts and restart your terminal after installation
+```
+
+**2. Create a Dedicated x86_64 Environment for PyFerret**
+This command creates a new `conda` environment named `pyferret_env` and installs the Intel (`x86_64`) version of `pyferret`, which will run seamlessly on Apple Silicon via Rosetta 2. It also pins `numpy` to a version older than 2.0 to ensure compatibility.
+
+```bash
+CONDA_SUBDIR=osx-64 conda create -n pyferret_env -c conda-forge pyferret ferret_datasets "numpy<2" --yes
+```
+
+**3. Configure QLC to Use the New Environment**
+The QLC scripts need to know where to find this new `pyferret` installation. You can achieve this by modifying the `qlc_C5.sh` script to activate the environment.
+
+Open the file `$HOME/qlc/sh/qlc_C5.sh` and add the following lines near the top, after `source $FUNCTIONS`:
+
+```bash
+# ... after 'source $FUNCTIONS'
+# Activate the dedicated conda environment for pyferret
+if [ -f "$HOME/miniforge3/bin/activate" ]; then
+    . "$HOME/miniforge3/bin/activate"
+    conda activate pyferret_env
+fi
+# ... rest of the script
+```
+*Note: The path to the activate script may differ if you installed Anaconda/Miniforge in a custom location.*
+
+### MARS Data Retrieval
+The `qlc_A1.sh` script is responsible for retrieving data from the ECMWF MARS archive. It uses a mapping system to associate the experiment prefix with a MARS `class`. 
+
+By default, the script is configured for `nl` (Netherlands), `be` (Belgium), and `rd` (Research Department) experiments. If you are working with data from other classes (e.g., `fr` for France, `de` for Germany), you will need to manually edit `$HOME/qlc/sh/qlc_A1.sh` and uncomment / edit the corresponding `XCLASS` line to ensure data is retrieved correctly.
+
+---
+
+## Troubleshooting
+
+### macOS "Permission Denied" or Quarantine Issues
+
+On macOS, the Gatekeeper security feature may "quarantine" files, including shell scripts that have been downloaded or modified. This can prevent them from being executed, sometimes with a "Permission Denied" error, even if the file has the correct execute permissions (`+x`).
+
+This is most likely to occur if you manually edit the `qlc` shell scripts (`.sh` files) directly in their `site-packages` installation directory.
+
+To resolve this, you can manually remove the quarantine attribute from the script directory using the `xattr` command in your terminal.
+
+1.  **First, find the exact location of the `qlc` package:**
+    ```bash
+    pip show rc-qlc
+    ```
+    Look for the `Location:` line in the output. This is your `site-packages` path.
+
+2.  **Then, use the `xattr` command to remove the quarantine flag:**
+    Use the path from the previous step to build the full path to the `qlc/sh` directory.
+    ```bash
+    # The path will depend on your Python installation. Use the location from 'pip show'.
+    xattr -rd com.apple.quarantine /path/to/your/site-packages/qlc/sh/
+    ```
+
+This should immediately resolve the execution issues.
+
 ---
 
 ## License
@@ -176,3 +305,5 @@ qlc-install --mode interactive --config /path/to/your/custom_qlc.conf
 © ResearchConcepts io GmbH  
 Contact: [contact@researchconcepts.io](mailto:contact@researchconcepts.io)  
 MIT-compatible, source-restricted under private release until publication.
+
+---

{rc_qlc-0.3.26.dist-info → rc_qlc-0.3.27.dist-info}/RECORD

@@ -1,32 +1,26 @@
-rc_qlc-0.3.26.dist-info/RECORD,,
-rc_qlc-0.3.26.dist-info/WHEEL,sha256=qxQkdhERtGxJzqnVOBnucx1aUmU2n3HmuzYdln_LyOw,109
-rc_qlc-0.3.26.dist-info/entry_points.txt,sha256=haCqahVrnFhVkfBt0HoOuVpKGV1KJOKpC1v75lOBl_s,191
-rc_qlc-0.3.26.dist-info/top_level.txt,sha256=Czv4KZhpli6RXvgvtCB1g87eRNEM2R-8iHplel1Qp5Q,4
-rc_qlc-0.3.26.dist-info/METADATA,sha256=_7gmYNJCGMHKD4tdeNNYoWYSeJ_XwxmlyHxpwVUaH5s,7849
-rc_qlc-0.3.26.dist-info/licenses/LICENSE,sha256=1pYHFCPjSx9_0weOUVJ-SpSin1dm6nSMzeO8Jj_qKuo,1081
 qlc/__init__.py,sha256=cmqybybgtiIwlVJIUfWOtP65yM1WzykLc1QRgdLANtI,208
-qlc/install.py,sha256=AgA7OAsRqk5LRq9yuQ4TyZhz_IT0jk-Ecbut2OSOiJw,18534
-qlc/py/style.cpython-311-darwin.so,sha256=5fgn3eTjJDPPqnN5dhvomQsguOCSXGZKj-WXMf_vB9Y,300152
-qlc/py/bias_plots.cpython-311-darwin.so,sha256=5lrmQksD_CKNTqFgFYKRigUa9fWu7rVyX7IaKRxs1Qc,107344
-qlc/py/control.cpython-311-darwin.so,sha256=PWMGvfboE0EzP1u9VmSbyEaz6akcd238QrX7SkJKuQY,250224
-qlc/py/__main__.cpython-311-darwin.so,sha256=Rbn2mC5GzA2zyP2A5vQB38r6EJS5Yhb4yo5APaRSsp0,80496
-qlc/py/loadobs.cpython-311-darwin.so,sha256=6d9DyEexGn92z7QwN-gdUuGPn3CWk-zLzJhC4RUgQv4,180800
-qlc/py/processing.cpython-311-darwin.so,sha256=Y96CRcLQGLONcANTPFTmhX2Erl0nFMlg7XFz7e67928,123328
-qlc/py/stations.cpython-311-darwin.so,sha256=v770ygfJuiB7ZLznC5Ag3tYNaNBXZzmsZ0_IU1Y9M1Q,693136
-qlc/py/averaging.cpython-311-darwin.so,sha256=YcEik_1d-FRRGNOumAf-BED07DvHxsbGy0Zp_JJJk00,82960
-qlc/py/version.cpython-311-darwin.so,sha256=wAhRhE74wUNter8qz90e4K1QXBsQ6tMR_JrJ9MI4_Kg,54256
-qlc/py/logging_utils.cpython-311-darwin.so,sha256=gtzx6xZ9LEkf4C8xhESyxS287AaGnVv80dAnKHVm51A,123632
-qlc/py/utils.cpython-311-darwin.so,sha256=YMdHgDBC64LIP2VwWa9R1JTYNHujpCtjxIrMQ21sCko,543320
-qlc/py/matched.cpython-311-darwin.so,sha256=qgYJh3pY26nEjj1-5O0LJUEmos63pDEjeTfPkOhUbi0,101056
-qlc/py/plugin_loader.cpython-311-darwin.so,sha256=Rgtmt8uKJdKcWurZTBQXzyGcg-_CSbPwGsYqrsSUizE,83408
-qlc/py/statistics.cpython-311-darwin.so,sha256=wlwOx7HDqpD4sci_OQZXvupjv8iICf8doS_Mu8yzxEs,252288
-qlc/py/map_plots.cpython-311-darwin.so,sha256=qWcT9YLwsA3vX57C6L2Dd7QWXr9UWOvdZEH5_I9Pawk,205456
-qlc/py/plot_config.cpython-311-darwin.so,sha256=AOrV9XcvnOfXDGGn8O1YOF43mvF6vFm9vc65zU1wBRg,79824
-qlc/py/loadmod.cpython-311-darwin.so,sha256=MiTG5tAtLxaK2JduCBKX-yCoVhsz4fN9SNULxqJeB3w,198208
-qlc/py/plotting.cpython-311-darwin.so,sha256=MOSH8XsjfyJ5aH-NirNfbPBVC0fvO8DO8dc0uHitPfI,179072
-qlc/py/timeseries_plots.cpython-311-darwin.so,sha256=lhBC4Jp88EoeQbLMYVp-Yhr6bTPlyJmzRfT5srbIw-g,621144
-qlc/py/io.cpython-311-darwin.so,sha256=J824JZDFPSWBQ_pyL-4dEQtEU8d7XxzniFs3aghuUtY,144408
-qlc/py/scatter_plots.cpython-311-darwin.so,sha256=l8KhXgVqBRf2Sbs_aQ3ziaSVfeAkSH-RMe2HfQk0DlQ,106160
+qlc/install.py,sha256=BjMoE-K8sll8K118BT12dcV6rarY0XuyKemZuTgBuTA,19059
+qlc/py/style.cpython-311-darwin.so,sha256=mzZimECUioVlCmFQQ9rcyv8tJYqOuQEyF_Q6WrUxfV8,300152
+qlc/py/bias_plots.cpython-311-darwin.so,sha256=BWeaGyktuYOGwtC5RemH2LUOzFNAHM_ahfbiG168liY,107344
+qlc/py/control.cpython-311-darwin.so,sha256=QTq5tAj9Z_iPTfLv7pshFYJxgcqg7T30FLd_YVP64WE,250224
+qlc/py/__main__.cpython-311-darwin.so,sha256=0kaP5SuOpgnt4-2QF4oyI0bbRazF2qoKbCJ38CIjzf4,80496
+qlc/py/loadobs.cpython-311-darwin.so,sha256=Q6DkAhpLnEdbB1JdzhrWPHuMjsxDOSLdvoGxONKqWAw,180800
+qlc/py/processing.cpython-311-darwin.so,sha256=oTxjs4zbaHkJIgjSRR3ypPk2XBK_TdItzf6pzhy6u3Y,123328
+qlc/py/stations.cpython-311-darwin.so,sha256=J7WC7fPDavNC1mfeXCxO1GG9fH2dBV0DTiT9L9LyiMs,693136
+qlc/py/averaging.cpython-311-darwin.so,sha256=CZim_fwJJt5RIqRXxl2jnql0v61tUhGx3qstu1FnjRA,82960
+qlc/py/version.cpython-311-darwin.so,sha256=eX-EP-29mNp95sqNTixZ7tdly6qferrTU2ZxqkSjpkw,54256
+qlc/py/logging_utils.cpython-311-darwin.so,sha256=2Awdq6z54Ic4e5KPgf6BbBhWHwBzlRLKzEx50NJsyyQ,123632
+qlc/py/utils.cpython-311-darwin.so,sha256=Q_kfJju2QYlZlXba3Suzhr8gL0f-Rn7vejqI7Uo56ZE,543320
+qlc/py/matched.cpython-311-darwin.so,sha256=e3hGQUSALahV1Xd1KrAPSQsU3EhdDlPDhHZtn84QACY,101056
+qlc/py/plugin_loader.cpython-311-darwin.so,sha256=oruJ9s_pTXXoKMLzGwSUGvCFlnKXWNVJ4MTcfDFquoU,83408
+qlc/py/statistics.cpython-311-darwin.so,sha256=B1crNqpc40rV6hB6b-XXZu7_1_2ZtDhminTUB3t5fTE,252288
+qlc/py/map_plots.cpython-311-darwin.so,sha256=cYw-0EuTJiBXpVnkQlJD1j3G0DYUcr8mCCO8AwaDTyE,205456
+qlc/py/plot_config.cpython-311-darwin.so,sha256=6CeNh3B6PZsn_0i5vLobQRE1I72BIA2Tg79ovKlbLoY,79824
+qlc/py/loadmod.cpython-311-darwin.so,sha256=B6e4jE8pI02UQH0XSrj3AAjpKH_AWjoguy_4oesMcGc,198208
+qlc/py/plotting.cpython-311-darwin.so,sha256=koPqpRV5tx673OPfnrtgai4wCyjzo1pXI65pRqS1ty0,179072
+qlc/py/timeseries_plots.cpython-311-darwin.so,sha256=_i94fobIMb2RH3-IWLV2wNO5M89x4mB-_Wg2f2PLcc4,621144
+qlc/py/io.cpython-311-darwin.so,sha256=47xjeUTxj9clT-pfuZua0Rd4EAt1yt_wkk_E8zBf04Y,144408
+qlc/py/scatter_plots.cpython-311-darwin.so,sha256=MSg8OsJNVKLcXV1z6Lc8b3joHD6bBmHqL5JaMLJqPu8,106160
 qlc/config/qlc.conf,sha256=NjQ-GwuXiT2sAsKtUPTkano1h_0xe1XSr4OTr-jFlkg,8542
 qlc/config/nml/mars_A2_sfc.nml,sha256=83Gkdr8vmlzif41EClWdjiJopZHDk9p-80-_l3fkeDE,638
 qlc/config/nml/mars_B1_pl.nml,sha256=j-WUUNqFh3a1m3uh4UVtd23kP1E9vuYDvfZN1BfYVxE,641
@@ -48,14 +42,14 @@ qlc/cli/__init__.py,sha256=5BRttT9IRcyoQx_dD9S0g-wBfz1ufakpLUGEGYZV808,4219
 qlc/cli/installer.py,sha256=xe1Dzwmd_suc_-d63k50v5c9WNOljtVx7cZ9ofVJGd8,1335
 qlc/cli/qlc_py_main.py,sha256=9L8P0mPiw35I37Drdqajydyb8IojMQkPAbOenwunTkI,2720
 qlc/sh/qlc_B1a.sh,sha256=iwqgH84xoDNQtWRPq4048oBwhIIjKoV7oqv6sa-exIg,3110
-qlc/sh/qlc_D1.sh,sha256=7lEBVJP3EeI2s8F0QPnRn3a_CImcBCSmghhPWEsvx6M,13583
-qlc/sh/qlc_A1.sh,sha256=v8IzABCMBRxYb0cwVAh90mkYeGsBLPOItpKeFwTBl_E,5331
+qlc/sh/qlc_D1.sh,sha256=oesshJ5K2XLm2-mHtPZkOK77B8KRMTs8WpCCMz5gwig,13408
+qlc/sh/qlc_A1.sh,sha256=U4YjpGAFFDgUjAwZhqujASGC-awLQQYwImVmDRnTTZ0,5461
 qlc/sh/qlc_common_functions.sh,sha256=ozBEOfnsjy7j-FTyvdyBWf5xpewNQ6SVzA9uwWVIvGs,5355
-qlc/sh/qlc_C5.sh,sha256=YunNdShkFNaRjqro0HALc1WmFmnKUioX-aIEXBWD_vI,36136
-qlc/sh/qlc_main.sh,sha256=yjeLRilD9JNxrXiR8LL1a-64hMafcK2S8l2P6P00frw,4885
+qlc/sh/qlc_C5.sh,sha256=SxCDw2CvHqPP7knLJWe7nDGZDivIzf-IAUsQpgb9Uyg,38066
+qlc/sh/qlc_main.sh,sha256=yvZXrxuDA-RG6zC7lpn--yOGHJgjGIy5Rj3D0WfCnJE,5531
 qlc/sh/qlc_Z1.sh,sha256=GzWhyGU5nWIGU-adTMEaXzAnYaDpy6pVE46j_Vt5htA,5473
 qlc/sh/qlc_B2.sh,sha256=6K76ItijPZzpHkwR4dy-pyb_ivI433gxRZtg8LDTE-s,11173
-qlc/sh/qlc_batch.sh,sha256=i_dyEPmelvlhb_J00AmkWBu5EoTNXSAgwptd5xkNrrg,2415
+qlc/sh/qlc_batch.sh,sha256=I8HJNEeRpxHSZwPa-ZmgxDqU3Ltw4ROMWyjtFX9hRDw,2373
 qlc/sh/tex_template/beamercolorthemeCAMS2_35.sty,sha256=1wN5BAXnBQfxbz-PnWDTD2RXvU42wB5cOQp_qDD67Mo,2420
 qlc/sh/tex_template/subcaption.sty,sha256=qfztB7wGra-v-P7BsqkgPb6iaUt1LJ29N7YpeeJUG0Q,5937
 qlc/sh/tex_template/template.tex,sha256=Xx77W9tDp68zoTShmpg0LBHp1Qc8FfKOmbU8ifXUP5w,4660
@@ -97,6 +91,13 @@ qlc/examples/cams_case_1/mod/b2rn/2018/b2rn_20181201-20181221_C1_sfc.grb,sha256=
 qlc/examples/cams_case_1/mod/b2rn/2018/b2rn_20181201-20181221_B1_pl.grb,sha256=amWL5pi5OpxmkcejAleW22n57EK0nVeU_kl671bdASU,41073480
 qlc/examples/cams_case_1/mod/b2ro/2018/b2ro_20181201-20181221_B1_pl.grb,sha256=WKK2bZf3i9VDj0K1uoZaToXxVZJXPxXEGm52rrLGVSg,41073480
 qlc/examples/cams_case_1/mod/b2ro/2018/b2ro_20181201-20181221_C1_sfc.grb,sha256=FlWm-BFu9PTmAx--3AbUhs99tuWkNO6yEdFjn5UXRGA,1570901
-qlc/doc/USAGE.md,sha256=SC0HCvLLkzJoI_N4owMLiXR5HsXrqP5uXZbnRljVqpk,4232
-qlc/doc/README.md,sha256=Dt16C6xRPO9g4WEBcD0X9GxiFBu3aKWYRb2kjK9a0UQ,7136
+qlc/doc/USAGE.md,sha256=PmaGPIk2jtn95dpT4ffJoeCz7vDNwedQakC6YQkkLHc,18303
+qlc/doc/README.md,sha256=tabHFCBGpLyzpFXkA9BBsjkcJMiIiEly_2d2Bm2H1yo,13912
+qlc/doc/CAMS_b2ro-b2rn_20181201-20181221_qlc_Z1-png_202509090814.pdf,sha256=kltQSosZRDQ2UvX367OUnZq0_WUvWswt-Rr3_I5Pk4w,11807466
 qlc/doc/CONTRIBUTING.md,sha256=IXYwAIdAfiQUQOVJcP0RTVpWL9Q-dW2ljG6hs4WkHTY,2607
+rc_qlc-0.3.27.dist-info/RECORD,,
+rc_qlc-0.3.27.dist-info/WHEEL,sha256=qxQkdhERtGxJzqnVOBnucx1aUmU2n3HmuzYdln_LyOw,109
+rc_qlc-0.3.27.dist-info/entry_points.txt,sha256=haCqahVrnFhVkfBt0HoOuVpKGV1KJOKpC1v75lOBl_s,191
+rc_qlc-0.3.27.dist-info/top_level.txt,sha256=Czv4KZhpli6RXvgvtCB1g87eRNEM2R-8iHplel1Qp5Q,4
+rc_qlc-0.3.27.dist-info/METADATA,sha256=oHV9ZrAMzNKvnSg7uQajUfgkWcV1MDbgzXSWEMbjCZ8,14716
+rc_qlc-0.3.27.dist-info/licenses/LICENSE,sha256=1pYHFCPjSx9_0weOUVJ-SpSin1dm6nSMzeO8Jj_qKuo,1081