hera-matlab 1.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- hera_matlab-1.1.0/LICENSE +21 -0
- hera_matlab-1.1.0/PKG-INFO +329 -0
- hera_matlab-1.1.0/README.md +287 -0
- hera_matlab-1.1.0/hera_matlab/__init__.py +373 -0
- hera_matlab-1.1.0/hera_matlab/hera_matlab.ctf +0 -0
- hera_matlab-1.1.0/hera_matlab/install_runtime.py +130 -0
- hera_matlab-1.1.0/hera_matlab.egg-info/PKG-INFO +329 -0
- hera_matlab-1.1.0/hera_matlab.egg-info/SOURCES.txt +12 -0
- hera_matlab-1.1.0/hera_matlab.egg-info/dependency_links.txt +1 -0
- hera_matlab-1.1.0/hera_matlab.egg-info/top_level.txt +1 -0
- hera_matlab-1.1.0/pyproject.toml +6 -0
- hera_matlab-1.1.0/setup.cfg +4 -0
- hera_matlab-1.1.0/setup.py +141 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Lukas von Erdmannsdorff
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,329 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: hera-matlab
|
|
3
|
+
Version: 1.1.0
|
|
4
|
+
Summary: A Python interface to hera_matlab
|
|
5
|
+
Home-page: https://github.com/lerdmann1601/HERA-Matlab
|
|
6
|
+
Author: Lukas von Erdmannsdorff
|
|
7
|
+
License: MIT
|
|
8
|
+
Project-URL: Bug Tracker, https://github.com/lerdmann1601/HERA-Matlab/issues
|
|
9
|
+
Project-URL: Source Code, https://github.com/lerdmann1601/HERA-Matlab
|
|
10
|
+
Project-URL: Documentation, https://github.com/lerdmann1601/HERA-Matlab/tree/main/docs
|
|
11
|
+
Keywords: ranking,benchmarking,statistics,effect-size,bootstrapping,significance-testing,matlab-interface,hierarchical-compensatory,scientific-computing
|
|
12
|
+
Platform: Linux
|
|
13
|
+
Platform: Windows
|
|
14
|
+
Platform: macOS
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
21
|
+
Classifier: Operating System :: OS Independent
|
|
22
|
+
Classifier: Topic :: Scientific/Engineering
|
|
23
|
+
Classifier: Topic :: Scientific/Engineering :: Mathematics
|
|
24
|
+
Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
|
|
25
|
+
Classifier: Intended Audience :: Science/Research
|
|
26
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
27
|
+
Requires-Python: >=3.9, <3.13
|
|
28
|
+
Description-Content-Type: text/markdown
|
|
29
|
+
License-File: LICENSE
|
|
30
|
+
Dynamic: author
|
|
31
|
+
Dynamic: classifier
|
|
32
|
+
Dynamic: description
|
|
33
|
+
Dynamic: description-content-type
|
|
34
|
+
Dynamic: home-page
|
|
35
|
+
Dynamic: keywords
|
|
36
|
+
Dynamic: license
|
|
37
|
+
Dynamic: license-file
|
|
38
|
+
Dynamic: platform
|
|
39
|
+
Dynamic: project-url
|
|
40
|
+
Dynamic: requires-python
|
|
41
|
+
Dynamic: summary
|
|
42
|
+
|
|
43
|
+
|
|
44
|
+
<div align="center">
|
|
45
|
+
|
|
46
|
+
<img src="https://raw.githubusercontent.com/lerdmann1601/HERA-Matlab/main/assets/hera_logo.svg" alt="HERA Logo" width="300"/>
|
|
47
|
+
|
|
48
|
+
# HERA: Hierarchical-Compensatory, Effect-Size-Driven and Non-Parametric Ranking Algorithm
|
|
49
|
+
|
|
50
|
+
[](https://www.mathworks.com/products/matlab.html)
|
|
51
|
+
[](https://www.mathworks.com/products/statistics.html)
|
|
52
|
+
[](https://www.mathworks.com/products/parallel-computing.html)
|
|
53
|
+
[](https://opensource.org/licenses/MIT)
|
|
54
|
+
[](https://github.com/lerdmann1601/HERA-Matlab)
|
|
55
|
+
[](https://github.com/lerdmann1601/HERA-Matlab/issues)
|
|
56
|
+
[](https://doi.org/10.5281/zenodo.18274871)
|
|
57
|
+
|
|
58
|
+
[](https://orcid.org/0009-0009-3758-7363)
|
|
59
|
+
|
|
60
|
+
**Made for Scientific Benchmarking**
|
|
61
|
+
[Key Features](#key-features) • [Installation](#installation) • [Quick Start](#quick-start) • [Documentation](#documentation) • [Citation](#citation)
|
|
62
|
+
|
|
63
|
+
</div>
|
|
64
|
+
|
|
65
|
+
---
|
|
66
|
+
|
|
67
|
+
## Overview
|
|
68
|
+
|
|
69
|
+
**HERA** is a MATLAB toolbox designed to automate the objective comparison of
|
|
70
|
+
algorithms, experimental conditions, or datasets across multiple quality
|
|
71
|
+
metrics. Unlike traditional ranking methods that rely solely on mean values or
|
|
72
|
+
p-values, HERA employs a **hierarchical-compensatory logic** that integrates:
|
|
73
|
+
|
|
74
|
+
* **Significance Testing**: Wilcoxon signed-rank tests for paired data.
|
|
75
|
+
* **Effect Sizes**: Cliff's Delta and Relative Mean Difference for practical relevance.
|
|
76
|
+
* **Bootstrapping**: Data-driven thresholds and BCa confidence intervals.
|
|
77
|
+
|
|
78
|
+
This ensures that a "win" is only counted if it is both **statistically
|
|
79
|
+
significant** and **practically relevant**, providing a robust and nuanced
|
|
80
|
+
ranking system.
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## Key Features
|
|
85
|
+
|
|
86
|
+
* **Hierarchical Logic**: Define primary and secondary metrics. Secondary
|
|
87
|
+
metrics can act as tie-breakers or rank correctors (e.g., `M1_M2`,
|
|
88
|
+
`M1_M2_M3`).
|
|
89
|
+
* **Data-Driven Thresholds**: Automatically calculates adaptive effect size
|
|
90
|
+
thresholds using Percentile Bootstrapping.
|
|
91
|
+
* **Robustness**: Utilizes Bias-Corrected and Accelerated (BCa) confidence
|
|
92
|
+
intervals and Cluster Bootstrapping for rank stability.
|
|
93
|
+
* **Automated Reporting**: Generates PDF reports, Win-Loss Matrices, Sankey
|
|
94
|
+
Diagrams, and machine-readable JSON/CSV exports.
|
|
95
|
+
* **Reproducibility**: Supports fixed-seed execution and configuration
|
|
96
|
+
file-based workflows.
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## Installation
|
|
101
|
+
|
|
102
|
+
### Requirements
|
|
103
|
+
|
|
104
|
+
* **MATLAB** (R2024a or later Required)
|
|
105
|
+
* **Statistics and Machine Learning Toolbox** (Required)
|
|
106
|
+
* **Parallel Computing Toolbox** (Required for performance)
|
|
107
|
+
|
|
108
|
+
### Setup
|
|
109
|
+
|
|
110
|
+
#### Option A: MATLAB Toolbox (Recommended)
|
|
111
|
+
|
|
112
|
+
1. Download the latest `HERA_vX.Y.Z.mltbx` from the
|
|
113
|
+
[Releases](https://github.com/lerdmann1601/HERA-Matlab/releases) page.
|
|
114
|
+
2. Double-click the file to install it.
|
|
115
|
+
3. Done! HERA is now available as a command (`HERA.start_ranking`) in MATLAB.
|
|
116
|
+
|
|
117
|
+
#### Option B: Git Clone (for Developers)
|
|
118
|
+
|
|
119
|
+
1. **Clone the repository:**
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
git clone https://github.com/lerdmann1601/HERA-Matlab.git
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
2. **Install/Configure Path:**
|
|
126
|
+
|
|
127
|
+
Navigate to the repository folder and run the setup script to add HERA to
|
|
128
|
+
your MATLAB path.
|
|
129
|
+
|
|
130
|
+
```matlab
|
|
131
|
+
cd HERA-Matlab
|
|
132
|
+
setup_HERA
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
👉 [Standalone Runtime](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Standalone_Runtime.md)
|
|
136
|
+
|
|
137
|
+
👉 [Python Integration](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Python_Integration.md)
|
|
138
|
+
|
|
139
|
+
👉 [Automated Build (GitHub Actions)](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Automated_Build.md)
|
|
140
|
+
|
|
141
|
+
---
|
|
142
|
+
|
|
143
|
+
## Quick Start
|
|
144
|
+
|
|
145
|
+
### 1. Interactive Mode (Recommended for Beginners)
|
|
146
|
+
|
|
147
|
+
The interactive command-line interface guides you through every step of the configuration,
|
|
148
|
+
from data selection to statistical parameters.
|
|
149
|
+
If you are new to HERA, this is the recommended mode.
|
|
150
|
+
At any point, you can exit the interface by typing `exit` or `quit` or `q`.
|
|
151
|
+
|
|
152
|
+
```matlab
|
|
153
|
+
HERA.start_ranking()
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
### 2. Batch Mode (Reproducible / Server)
|
|
157
|
+
|
|
158
|
+
For automated analysis or reproducible research, use a JSON configuration file.
|
|
159
|
+
|
|
160
|
+
```matlab
|
|
161
|
+
HERA.start_ranking('configFile', 'config.json')
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### 3. Unit Test Mode
|
|
165
|
+
|
|
166
|
+
Run the built-in validation suite to ensure HERA is working correctly on your system.
|
|
167
|
+
|
|
168
|
+
```matlab
|
|
169
|
+
% Run tests and save log to default location
|
|
170
|
+
HERA.start_ranking('runtest', 'true')
|
|
171
|
+
|
|
172
|
+
% Run tests and save log to a specific folder
|
|
173
|
+
HERA.start_ranking('runtest', 'true', 'logPath', '/path/to/logs')
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
> **Note:** Example use cases with synthetic datasets and results are
|
|
177
|
+
> provided in the `data/examples` directory. See [data/README.md](https://github.com/lerdmann1601/HERA-Matlab/blob/main/data/README.md) for a
|
|
178
|
+
> walkthrough of the example use cases and visual examples of the ranking
|
|
179
|
+
> outputs.
|
|
180
|
+
>
|
|
181
|
+
> **Note:** HERA is designed for high-performance scientific computing, featuring
|
|
182
|
+
> **fully parallelized bootstrap procedures** and **automatic memory management**
|
|
183
|
+
> to optimize efficiency. However, specifically due to the extensive use of
|
|
184
|
+
> bootstrapping, it remains a **CPU-intensive application**. Please ensure you
|
|
185
|
+
> have access to enough CPU cores for reasonable performance.
|
|
186
|
+
---
|
|
187
|
+
|
|
188
|
+
## Documentation
|
|
189
|
+
|
|
190
|
+
👉 [Repository Structure](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Repository_Structure.md)
|
|
191
|
+
|
|
192
|
+
👉 [Ranking Modes Explained](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Ranking_Modes_Explained.md)
|
|
193
|
+
|
|
194
|
+
👉 [Input Data Specification](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Input_Data_Specification.md)
|
|
195
|
+
|
|
196
|
+
👉 [Methodological Guidelines & Limitations](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Methodological_Guidelines_&_Limitations.md)
|
|
197
|
+
|
|
198
|
+
👉 [Configuration Reference](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Configuration_Reference.md)
|
|
199
|
+
|
|
200
|
+
👉 [Bootstrap Configuration](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Bootstrap_Configuration.md)
|
|
201
|
+
|
|
202
|
+
👉 [Convergence Modes](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Convergence_Modes.md)
|
|
203
|
+
|
|
204
|
+
👉 [Advanced Usage (Developer Mode)](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Advanced_Usage.md)
|
|
205
|
+
|
|
206
|
+
👉 [Results Structure Reference](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Results_Structure_Reference.md)
|
|
207
|
+
|
|
208
|
+
---
|
|
209
|
+
|
|
210
|
+
## Outputs
|
|
211
|
+
|
|
212
|
+
HERA generates a timestamped directory containing:
|
|
213
|
+
|
|
214
|
+
```text
|
|
215
|
+
Ranking_<Timestamp>/
|
|
216
|
+
├── Output/
|
|
217
|
+
│ ├── results_*.csv % Final ranking table (Mean ± SD of metrics and rank CI)
|
|
218
|
+
│ ├── data_*.json % Complete analysis record (Inputs, Config, Stats, Results)
|
|
219
|
+
│ ├── log_*.csv % Detailed log of pairwise comparisons and logic
|
|
220
|
+
│ ├── sensitivity_details_*.csv % Results of the Borda sensitivity analysis
|
|
221
|
+
│ ├── BCa_Correction_Factors_*.csv % Correction factors (Bias/Skewness) for BCa CIs
|
|
222
|
+
│ └── bootstrap_rank_*.csv % Complete distribution of bootstrapped ranks
|
|
223
|
+
├── Graphics/ % High-res PNGs organized in subfolders
|
|
224
|
+
│ ├── Ranking/
|
|
225
|
+
│ ├── Detail_Comparison/
|
|
226
|
+
│ ├── CI_Histograms/
|
|
227
|
+
│ └── Threshold_Analysis/
|
|
228
|
+
├── PDF/ % Specialized reports
|
|
229
|
+
│ ├── Ranking_Report.pdf
|
|
230
|
+
│ ├── Convergence_Report.pdf
|
|
231
|
+
│ └── Bootstrap_Report.pdf
|
|
232
|
+
├── Final_Ranking_*.png % Summary graphic of ranking result
|
|
233
|
+
├── Final_Report_*.pdf % Consolidated graphical report of the main results
|
|
234
|
+
├── Ranking_*.txt % Complete console log of the session
|
|
235
|
+
└── configuration.json % Reusable configuration file to reproduce the run
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
---
|
|
239
|
+
|
|
240
|
+
## Testing
|
|
241
|
+
|
|
242
|
+
HERA includes a comprehensive validation framework (`run_unit_test.m`)
|
|
243
|
+
comprising **46 test cases** organized into four suites:
|
|
244
|
+
|
|
245
|
+
1. **Unit Tests** (19 cases): Checks individual components, helper functions, and
|
|
246
|
+
execution logic (Run/Start packages) to ensure specific parts of the code work
|
|
247
|
+
correctly.
|
|
248
|
+
2. **Statistical Tests** (5 cases): Verifies the core mathematical functions
|
|
249
|
+
(e.g., Jackknife, Cliff's Delta) and ensures the performance optimizations
|
|
250
|
+
(hybrid switching) work as intended.
|
|
251
|
+
3. **Scientific Tests** (19 cases): Comprehensive validation of ranking logic,
|
|
252
|
+
statistical accuracy, and robustness against edge cases (e.g., zero
|
|
253
|
+
variance, outliers).
|
|
254
|
+
4. **System Tests** (3 cases): Runs the entire HERA pipeline from start to
|
|
255
|
+
finish to ensure that the JSON configuration (batch), Developer API and NaN
|
|
256
|
+
Data handling are working correctly.
|
|
257
|
+
|
|
258
|
+
### Running Tests
|
|
259
|
+
|
|
260
|
+
You can run the test suite in three ways:
|
|
261
|
+
|
|
262
|
+
1. **Auto-Log Mode (Default)**
|
|
263
|
+
Automatically finds a writable folder (e.g., Documents) to save the log
|
|
264
|
+
file.
|
|
265
|
+
|
|
266
|
+
```matlab
|
|
267
|
+
HERA.run_unit_test()
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
2. **Interactive Mode**
|
|
271
|
+
Opens a dialog to select where to save the log file.
|
|
272
|
+
|
|
273
|
+
```matlab
|
|
274
|
+
HERA.run_unit_test('interactive')
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
3. **Custom Path Mode**
|
|
278
|
+
Saves the log file to a specific directory.
|
|
279
|
+
|
|
280
|
+
```matlab
|
|
281
|
+
HERA.run_unit_test('/path/to/my/logs')
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
### GitHub Actions (Cloud Testing)
|
|
285
|
+
|
|
286
|
+
For reviewers or users without a local MATLAB license, you can run the test
|
|
287
|
+
suite directly on GitHub:
|
|
288
|
+
|
|
289
|
+
1. Go to the **Actions** tab in this repository.
|
|
290
|
+
2. Select **Testing HERA** from the left sidebar.
|
|
291
|
+
3. Click **Run workflow**.
|
|
292
|
+
|
|
293
|
+
---
|
|
294
|
+
|
|
295
|
+
## Contributing
|
|
296
|
+
|
|
297
|
+
We welcome contributions! Please see [CONTRIBUTING.md](https://github.com/lerdmann1601/HERA-Matlab/blob/main/CONTRIBUTING.md) for details.
|
|
298
|
+
|
|
299
|
+
1. Fork the repository.
|
|
300
|
+
2. Create a feature branch.
|
|
301
|
+
3. Commit your changes.
|
|
302
|
+
4. Open a Pull Request.
|
|
303
|
+
|
|
304
|
+
---
|
|
305
|
+
|
|
306
|
+
## Citation
|
|
307
|
+
|
|
308
|
+
If you use HERA in your research, please cite:
|
|
309
|
+
|
|
310
|
+
```bibtex
|
|
311
|
+
@software{HERA_Matlab,
|
|
312
|
+
author = {von Erdmannsdorff, Lukas},
|
|
313
|
+
title = {HERA: A Hierarchical-Compensatory, Effect-Size Driven and Non-parametric
|
|
314
|
+
Ranking Algorithm using Data-Driven Thresholds and Bootstrap Validation},
|
|
315
|
+
url = {https://github.com/lerdmann1601/HERA-Matlab},
|
|
316
|
+
version = {1.1.0},
|
|
317
|
+
doi = {10.5281/zenodo.18274871},
|
|
318
|
+
year = {2026}
|
|
319
|
+
}
|
|
320
|
+
```
|
|
321
|
+
|
|
322
|
+
---
|
|
323
|
+
|
|
324
|
+
## License
|
|
325
|
+
|
|
326
|
+
This project is licensed under the MIT License - see the [LICENSE](https://github.com/lerdmann1601/HERA-Matlab/blob/main/LICENSE) file
|
|
327
|
+
for details.
|
|
328
|
+
|
|
329
|
+
---
|
|
@@ -0,0 +1,287 @@
|
|
|
1
|
+
|
|
2
|
+
<div align="center">
|
|
3
|
+
|
|
4
|
+
<img src="https://raw.githubusercontent.com/lerdmann1601/HERA-Matlab/main/assets/hera_logo.svg" alt="HERA Logo" width="300"/>
|
|
5
|
+
|
|
6
|
+
# HERA: Hierarchical-Compensatory, Effect-Size-Driven and Non-Parametric Ranking Algorithm
|
|
7
|
+
|
|
8
|
+
[](https://www.mathworks.com/products/matlab.html)
|
|
9
|
+
[](https://www.mathworks.com/products/statistics.html)
|
|
10
|
+
[](https://www.mathworks.com/products/parallel-computing.html)
|
|
11
|
+
[](https://opensource.org/licenses/MIT)
|
|
12
|
+
[](https://github.com/lerdmann1601/HERA-Matlab)
|
|
13
|
+
[](https://github.com/lerdmann1601/HERA-Matlab/issues)
|
|
14
|
+
[](https://doi.org/10.5281/zenodo.18274871)
|
|
15
|
+
|
|
16
|
+
[](https://orcid.org/0009-0009-3758-7363)
|
|
17
|
+
|
|
18
|
+
**Made for Scientific Benchmarking**
|
|
19
|
+
[Key Features](#key-features) • [Installation](#installation) • [Quick Start](#quick-start) • [Documentation](#documentation) • [Citation](#citation)
|
|
20
|
+
|
|
21
|
+
</div>
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Overview
|
|
26
|
+
|
|
27
|
+
**HERA** is a MATLAB toolbox designed to automate the objective comparison of
|
|
28
|
+
algorithms, experimental conditions, or datasets across multiple quality
|
|
29
|
+
metrics. Unlike traditional ranking methods that rely solely on mean values or
|
|
30
|
+
p-values, HERA employs a **hierarchical-compensatory logic** that integrates:
|
|
31
|
+
|
|
32
|
+
* **Significance Testing**: Wilcoxon signed-rank tests for paired data.
|
|
33
|
+
* **Effect Sizes**: Cliff's Delta and Relative Mean Difference for practical relevance.
|
|
34
|
+
* **Bootstrapping**: Data-driven thresholds and BCa confidence intervals.
|
|
35
|
+
|
|
36
|
+
This ensures that a "win" is only counted if it is both **statistically
|
|
37
|
+
significant** and **practically relevant**, providing a robust and nuanced
|
|
38
|
+
ranking system.
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## Key Features
|
|
43
|
+
|
|
44
|
+
* **Hierarchical Logic**: Define primary and secondary metrics. Secondary
|
|
45
|
+
metrics can act as tie-breakers or rank correctors (e.g., `M1_M2`,
|
|
46
|
+
`M1_M2_M3`).
|
|
47
|
+
* **Data-Driven Thresholds**: Automatically calculates adaptive effect size
|
|
48
|
+
thresholds using Percentile Bootstrapping.
|
|
49
|
+
* **Robustness**: Utilizes Bias-Corrected and Accelerated (BCa) confidence
|
|
50
|
+
intervals and Cluster Bootstrapping for rank stability.
|
|
51
|
+
* **Automated Reporting**: Generates PDF reports, Win-Loss Matrices, Sankey
|
|
52
|
+
Diagrams, and machine-readable JSON/CSV exports.
|
|
53
|
+
* **Reproducibility**: Supports fixed-seed execution and configuration
|
|
54
|
+
file-based workflows.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Installation
|
|
59
|
+
|
|
60
|
+
### Requirements
|
|
61
|
+
|
|
62
|
+
* **MATLAB** (R2024a or later Required)
|
|
63
|
+
* **Statistics and Machine Learning Toolbox** (Required)
|
|
64
|
+
* **Parallel Computing Toolbox** (Required for performance)
|
|
65
|
+
|
|
66
|
+
### Setup
|
|
67
|
+
|
|
68
|
+
#### Option A: MATLAB Toolbox (Recommended)
|
|
69
|
+
|
|
70
|
+
1. Download the latest `HERA_vX.Y.Z.mltbx` from the
|
|
71
|
+
[Releases](https://github.com/lerdmann1601/HERA-Matlab/releases) page.
|
|
72
|
+
2. Double-click the file to install it.
|
|
73
|
+
3. Done! HERA is now available as a command (`HERA.start_ranking`) in MATLAB.
|
|
74
|
+
|
|
75
|
+
#### Option B: Git Clone (for Developers)
|
|
76
|
+
|
|
77
|
+
1. **Clone the repository:**
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
git clone https://github.com/lerdmann1601/HERA-Matlab.git
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
2. **Install/Configure Path:**
|
|
84
|
+
|
|
85
|
+
Navigate to the repository folder and run the setup script to add HERA to
|
|
86
|
+
your MATLAB path.
|
|
87
|
+
|
|
88
|
+
```matlab
|
|
89
|
+
cd HERA-Matlab
|
|
90
|
+
setup_HERA
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
👉 [Standalone Runtime](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Standalone_Runtime.md)
|
|
94
|
+
|
|
95
|
+
👉 [Python Integration](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Python_Integration.md)
|
|
96
|
+
|
|
97
|
+
👉 [Automated Build (GitHub Actions)](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Automated_Build.md)
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
## Quick Start
|
|
102
|
+
|
|
103
|
+
### 1. Interactive Mode (Recommended for Beginners)
|
|
104
|
+
|
|
105
|
+
The interactive command-line interface guides you through every step of the configuration,
|
|
106
|
+
from data selection to statistical parameters.
|
|
107
|
+
If you are new to HERA, this is the recommended mode.
|
|
108
|
+
At any point, you can exit the interface by typing `exit` or `quit` or `q`.
|
|
109
|
+
|
|
110
|
+
```matlab
|
|
111
|
+
HERA.start_ranking()
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
### 2. Batch Mode (Reproducible / Server)
|
|
115
|
+
|
|
116
|
+
For automated analysis or reproducible research, use a JSON configuration file.
|
|
117
|
+
|
|
118
|
+
```matlab
|
|
119
|
+
HERA.start_ranking('configFile', 'config.json')
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
### 3. Unit Test Mode
|
|
123
|
+
|
|
124
|
+
Run the built-in validation suite to ensure HERA is working correctly on your system.
|
|
125
|
+
|
|
126
|
+
```matlab
|
|
127
|
+
% Run tests and save log to default location
|
|
128
|
+
HERA.start_ranking('runtest', 'true')
|
|
129
|
+
|
|
130
|
+
% Run tests and save log to a specific folder
|
|
131
|
+
HERA.start_ranking('runtest', 'true', 'logPath', '/path/to/logs')
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
> **Note:** Example use cases with synthetic datasets and results are
|
|
135
|
+
> provided in the `data/examples` directory. See [data/README.md](https://github.com/lerdmann1601/HERA-Matlab/blob/main/data/README.md) for a
|
|
136
|
+
> walkthrough of the example use cases and visual examples of the ranking
|
|
137
|
+
> outputs.
|
|
138
|
+
>
|
|
139
|
+
> **Note:** HERA is designed for high-performance scientific computing, featuring
|
|
140
|
+
> **fully parallelized bootstrap procedures** and **automatic memory management**
|
|
141
|
+
> to optimize efficiency. However, specifically due to the extensive use of
|
|
142
|
+
> bootstrapping, it remains a **CPU-intensive application**. Please ensure you
|
|
143
|
+
> have access to enough CPU cores for reasonable performance.
|
|
144
|
+
---
|
|
145
|
+
|
|
146
|
+
## Documentation
|
|
147
|
+
|
|
148
|
+
👉 [Repository Structure](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Repository_Structure.md)
|
|
149
|
+
|
|
150
|
+
👉 [Ranking Modes Explained](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Ranking_Modes_Explained.md)
|
|
151
|
+
|
|
152
|
+
👉 [Input Data Specification](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Input_Data_Specification.md)
|
|
153
|
+
|
|
154
|
+
👉 [Methodological Guidelines & Limitations](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Methodological_Guidelines_&_Limitations.md)
|
|
155
|
+
|
|
156
|
+
👉 [Configuration Reference](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Configuration_Reference.md)
|
|
157
|
+
|
|
158
|
+
👉 [Bootstrap Configuration](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Bootstrap_Configuration.md)
|
|
159
|
+
|
|
160
|
+
👉 [Convergence Modes](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Convergence_Modes.md)
|
|
161
|
+
|
|
162
|
+
👉 [Advanced Usage (Developer Mode)](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Advanced_Usage.md)
|
|
163
|
+
|
|
164
|
+
👉 [Results Structure Reference](https://github.com/lerdmann1601/HERA-Matlab/blob/main/docs/Results_Structure_Reference.md)
|
|
165
|
+
|
|
166
|
+
---
|
|
167
|
+
|
|
168
|
+
## Outputs
|
|
169
|
+
|
|
170
|
+
HERA generates a timestamped directory containing:
|
|
171
|
+
|
|
172
|
+
```text
|
|
173
|
+
Ranking_<Timestamp>/
|
|
174
|
+
├── Output/
|
|
175
|
+
│ ├── results_*.csv % Final ranking table (Mean ± SD of metrics and rank CI)
|
|
176
|
+
│ ├── data_*.json % Complete analysis record (Inputs, Config, Stats, Results)
|
|
177
|
+
│ ├── log_*.csv % Detailed log of pairwise comparisons and logic
|
|
178
|
+
│ ├── sensitivity_details_*.csv % Results of the Borda sensitivity analysis
|
|
179
|
+
│ ├── BCa_Correction_Factors_*.csv % Correction factors (Bias/Skewness) for BCa CIs
|
|
180
|
+
│ └── bootstrap_rank_*.csv % Complete distribution of bootstrapped ranks
|
|
181
|
+
├── Graphics/ % High-res PNGs organized in subfolders
|
|
182
|
+
│ ├── Ranking/
|
|
183
|
+
│ ├── Detail_Comparison/
|
|
184
|
+
│ ├── CI_Histograms/
|
|
185
|
+
│ └── Threshold_Analysis/
|
|
186
|
+
├── PDF/ % Specialized reports
|
|
187
|
+
│ ├── Ranking_Report.pdf
|
|
188
|
+
│ ├── Convergence_Report.pdf
|
|
189
|
+
│ └── Bootstrap_Report.pdf
|
|
190
|
+
├── Final_Ranking_*.png % Summary graphic of ranking result
|
|
191
|
+
├── Final_Report_*.pdf % Consolidated graphical report of the main results
|
|
192
|
+
├── Ranking_*.txt % Complete console log of the session
|
|
193
|
+
└── configuration.json % Reusable configuration file to reproduce the run
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
---
|
|
197
|
+
|
|
198
|
+
## Testing
|
|
199
|
+
|
|
200
|
+
HERA includes a comprehensive validation framework (`run_unit_test.m`)
|
|
201
|
+
comprising **46 test cases** organized into four suites:
|
|
202
|
+
|
|
203
|
+
1. **Unit Tests** (19 cases): Checks individual components, helper functions, and
|
|
204
|
+
execution logic (Run/Start packages) to ensure specific parts of the code work
|
|
205
|
+
correctly.
|
|
206
|
+
2. **Statistical Tests** (5 cases): Verifies the core mathematical functions
|
|
207
|
+
(e.g., Jackknife, Cliff's Delta) and ensures the performance optimizations
|
|
208
|
+
(hybrid switching) work as intended.
|
|
209
|
+
3. **Scientific Tests** (19 cases): Comprehensive validation of ranking logic,
|
|
210
|
+
statistical accuracy, and robustness against edge cases (e.g., zero
|
|
211
|
+
variance, outliers).
|
|
212
|
+
4. **System Tests** (3 cases): Runs the entire HERA pipeline from start to
|
|
213
|
+
finish to ensure that the JSON configuration (batch), Developer API and NaN
|
|
214
|
+
Data handling are working correctly.
|
|
215
|
+
|
|
216
|
+
### Running Tests
|
|
217
|
+
|
|
218
|
+
You can run the test suite in three ways:
|
|
219
|
+
|
|
220
|
+
1. **Auto-Log Mode (Default)**
|
|
221
|
+
Automatically finds a writable folder (e.g., Documents) to save the log
|
|
222
|
+
file.
|
|
223
|
+
|
|
224
|
+
```matlab
|
|
225
|
+
HERA.run_unit_test()
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
2. **Interactive Mode**
|
|
229
|
+
Opens a dialog to select where to save the log file.
|
|
230
|
+
|
|
231
|
+
```matlab
|
|
232
|
+
HERA.run_unit_test('interactive')
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
3. **Custom Path Mode**
|
|
236
|
+
Saves the log file to a specific directory.
|
|
237
|
+
|
|
238
|
+
```matlab
|
|
239
|
+
HERA.run_unit_test('/path/to/my/logs')
|
|
240
|
+
```
|
|
241
|
+
|
|
242
|
+
### GitHub Actions (Cloud Testing)
|
|
243
|
+
|
|
244
|
+
For reviewers or users without a local MATLAB license, you can run the test
|
|
245
|
+
suite directly on GitHub:
|
|
246
|
+
|
|
247
|
+
1. Go to the **Actions** tab in this repository.
|
|
248
|
+
2. Select **Testing HERA** from the left sidebar.
|
|
249
|
+
3. Click **Run workflow**.
|
|
250
|
+
|
|
251
|
+
---
|
|
252
|
+
|
|
253
|
+
## Contributing
|
|
254
|
+
|
|
255
|
+
We welcome contributions! Please see [CONTRIBUTING.md](https://github.com/lerdmann1601/HERA-Matlab/blob/main/CONTRIBUTING.md) for details.
|
|
256
|
+
|
|
257
|
+
1. Fork the repository.
|
|
258
|
+
2. Create a feature branch.
|
|
259
|
+
3. Commit your changes.
|
|
260
|
+
4. Open a Pull Request.
|
|
261
|
+
|
|
262
|
+
---
|
|
263
|
+
|
|
264
|
+
## Citation
|
|
265
|
+
|
|
266
|
+
If you use HERA in your research, please cite:
|
|
267
|
+
|
|
268
|
+
```bibtex
|
|
269
|
+
@software{HERA_Matlab,
|
|
270
|
+
author = {von Erdmannsdorff, Lukas},
|
|
271
|
+
title = {HERA: A Hierarchical-Compensatory, Effect-Size Driven and Non-parametric
|
|
272
|
+
Ranking Algorithm using Data-Driven Thresholds and Bootstrap Validation},
|
|
273
|
+
url = {https://github.com/lerdmann1601/HERA-Matlab},
|
|
274
|
+
version = {1.1.0},
|
|
275
|
+
doi = {10.5281/zenodo.18274871},
|
|
276
|
+
year = {2026}
|
|
277
|
+
}
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
---
|
|
281
|
+
|
|
282
|
+
## License
|
|
283
|
+
|
|
284
|
+
This project is licensed under the MIT License - see the [LICENSE](https://github.com/lerdmann1601/HERA-Matlab/blob/main/LICENSE) file
|
|
285
|
+
for details.
|
|
286
|
+
|
|
287
|
+
---
|