graphroute 0.1.0__tar.gz

graphroute-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Brianna Mueller
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

graphroute-0.1.0/MANIFEST.in

@@ -0,0 +1,2 @@
+include quickstart.py run_experiments.py model_registry.py
+recursive-include configs *.yaml

graphroute-0.1.0/PKG-INFO

@@ -0,0 +1,221 @@
+Metadata-Version: 2.4
+Name: graphroute
+Version: 0.1.0
+Summary: Graph-based dynamic ensembling
+Author: Brianna Mueller
+License-Expression: MIT
+Project-URL: Repository, https://github.com/briannamueller/GraphRoute
+Project-URL: Issues, https://github.com/briannamueller/GraphRoute/issues
+Requires-Python: <3.13,>=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: torch>=2.0.0
+Requires-Dist: pydantic>=2
+Requires-Dist: torch_geometric>=2.4.0
+Requires-Dist: numpy<2
+Requires-Dist: scikit-learn
+Requires-Dist: scipy
+Requires-Dist: filelock
+Requires-Dist: pyyaml>=6
+Requires-Dist: pytorch-minimize
+Dynamic: license-file
+
+# GraphRoute
+
+GraphRoute is a graph-based dynamic ensemble selection framework. Given a pool
+of candidate models, a GNN trained over a sample similarity graph learns which
+models to trust for individual cases. Because different models have different
+inductive biases, their reliability varies across the input space. GraphRoute
+learns sample representations where proximity better reflects shared model
+competence. This approach is particularly valuable for problems where models
+optimized for aggregate performance tend to fail on rare edge cases, and where
+failures on such cases are the most consequential.
+
+GraphRoute operates in three stages:
+
+1. Train a pool of diverse models.
+2. Construct a graph where nodes represent samples and edges encode sample similarity.
+3. Train a GNN to produce per-classifier competence scores.
+
+- [Installation](#installation)
+- [Quickstart](#quickstart)
+- [Data and model interface](#data-and-model-interface)
+- [Configure Experiments](#configure-experiments)
+- [Configuration](#configuration)
+- [Reusing a trained pool](#reusing-a-trained-pool)
+- [Development](#development)
+
+## Installation
+
+```bash
+pip install graphroute
+```
+
+GraphRoute requires Python 3.10–3.12, PyTorch 2.0 or newer, and PyTorch
+Geometric 2.4 or newer.
+
+The quickstart and experiment runner are repository-level scripts. To use them,
+clone the repository and follow the [development installation](#development-installation).
+
+## Quickstart
+
+```bash
+python quickstart.py
+```
+
+The quickstart trains a pool of four lightweight models on synthetic data, fits
+a graph attention network (GAT), and compares the performance of the resulting
+GNN's dynamic selection with the individual classifiers and fixed ensemble
+baselines.
+
+## Data and model interface
+
+GraphRoute accepts PyTorch Dataset objects whose samples are (inputs, target) pairs. Datasets with
+custom sample structures can provide a `collate_fn`; see the `fit_graphroute`
+function documentation in
+[`graphroute/run.py`](https://github.com/briannamueller/GraphRoute/blob/main/graphroute/run.py)
+for the required interface. Supply the candidate model instances through the
+ordered `models` list.
+
+## Configure Experiments
+
+[`run_experiments.py`](https://github.com/briannamueller/GraphRoute/blob/main/run_experiments.py)
+reads the experiment configuration from
+[`configs/experiment.yaml`](https://github.com/briannamueller/GraphRoute/blob/main/configs/experiment.yaml)
+and model factories from
+[`model_registry.py`](https://github.com/briannamueller/GraphRoute/blob/main/model_registry.py).
+Register candidate models in `MODEL_REGISTRY`, specify their names in
+`base.models`, and run:
+
+```bash
+python run_experiments.py --config configs/experiment.yaml
+```
+
+The provided YAML defines one experiment. Uncomment its optional `sweep`
+section to run the Cartesian product of the listed values. Each completed
+configuration and its metrics are saved as a separate JSON file under
+`results/<dataset>/`. Repeating the command skips completed configurations.
+Use `--force` to rerun completed configurations.
+
+With the default `data_dir="data"`, GraphRoute reads `train.pt` and `test.pt`
+from `data/<dataset>/`; each file must contain an `(inputs, targets)` tuple saved
+using `torch.save`. `validation.pt` is optional. When it is absent, GraphRoute
+derives validation data from the training set using `val_ratio`.
+
+## Configuration
+
+Every available setting and its default is defined in
+[`graphroute/config.py`](https://github.com/briannamueller/GraphRoute/blob/main/graphroute/config.py).
+The YAML file uses the same field names. The tables below focus on settings
+whose options require an understanding of GraphRoute itself.
+
+### General
+
+| Argument | Meaning | Available options |
+| --- | --- | --- |
+| `loss_target` | Sets the GNN training objective. | `"meta_labels"`: minimizes the loss between predicted competence scores and targets that encode each model’s competence.<br>`"ensemble"`: minimizes the loss between the combined prediction and each sample’s ground-truth class label or regression target. |
+
+### Model pool training (`base`)
+
+| Argument | Meaning | Available options |
+| --- | --- | --- |
+| `base.models` | Names the ordered model pool using entries in the experiment registry. | Nonempty list of registered model names |
+| `base.split_mode` | Chooses how the pool produces out-of-sample predictions for GNN training. | `"oof_stacking"`, `"split_train"` |
+| `base.oof_folds` | Sets the number of folds used for OOF pool training. | Integer of at least `2` |
+
+
+To use all of the training data to train both the base classifiers and the GNN without the optimistic bias caused by evaluating models on their own training samples, base.split_mode=`"oof_stacking"` uses cross-validation to generate out-of-fold predictions for GNN training. The final classifiers used for inference are then trained on the full training set.
+
+`split_train` is less computationally expensive because each base classifier is trained only once. It divides the training data into two parts: one is used to train the base classifiers, and the other is used to train the GNN.
+
+### Graph construction (`graph`)
+
+<table>
+  <thead>
+    <tr>
+      <th>Argument</th>
+      <th>Meaning</th>
+      <th>Available options</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><code>graph.node_feature_source</code></td>
+      <td>Selects the sample representation supplied to the GNN.</td>
+      <td rowspan="2">
+        <code>"decision_space"</code>: concatenated pool predictions for the sample.<br>
+        <code>"feature_space"</code>: original features (flattened if not tabular already).<br>
+        <code>"embedding_mean"</code>: averages internal representation each model produces for the sample immediately before its final layer (requires same size embeddings).<br>
+        <code>"embedding_concat"</code>: concatenates internal representation each model produces for the sample immediately before its final layer (embedding sizes may differ).<br>
+        <code>"hybrid"</code>: decision-space representation and original features concatenated.
+      </td>
+    </tr>
+    <tr>
+      <td><code>graph.edge_feature_source</code></td>
+      <td>Selects the representation used to measure similarity when constructing graph edges.</td>
+    </tr>
+    <tr>
+      <td><code>graph.k</code></td>
+      <td>Sets the number of neighbors per sample.</td>
+      <td>Positive integer</td>
+    </tr>
+    <tr>
+      <td><code>graph.neighbor_mode</code></td>
+      <td>Selects ordinary nearest neighbors or class-balanced neighbors.</td>
+      <td><code>"knn"</code>, <code>"class_balanced"</code></td>
+    </tr>
+    <tr>
+      <td><code>graph.weight_mode</code></td>
+      <td>Determines how sample-to-sample edge weights are calculated.</td>
+      <td><code>"softmax"</code>, <code>"uniform"</code>, <code>"inverse_distance"</code>, <code>"cmdw"</code></td>
+    </tr>
+    <tr>
+      <td><code>graph.pool_calibrate</code></td>
+      <td>Enables or disables classification-pool calibration.</td>
+      <td><code>True</code>, <code>False</code></td>
+    </tr>
+    <tr>
+      <td><code>graph.calib_method</code></td>
+      <td>Selects the calibration method.</td>
+      <td><code>"ts-mix"</code>, <code>"logistic"</code></td>
+    </tr>
+  </tbody>
+</table>
+
+When oof_stacking is combined with embedding-based representations, GraphRoute uses out-of-fold predictions for GNN training, but extracts embeddings from the final base classifiers.
+
+### GNN training and dynamic selection (`gnn`)
+
+| Argument | Meaning | Available options |
+| --- | --- | --- |
+| `gnn.arch` | Selects the architecture used to learn the dynamic selection rule. | `"gat"`, `"graph_gps"`, `"mlp"` |
+| `gnn.loss` | Selects the GNN training loss. | `"bce"`, `"focal_bce"`, `"soft_bce"`, `"regression"` |
+| `gnn.ens_combination_mode` | Determines how model scores form the final prediction. | `"soft_weighted_voting"`, `"hard_weighted_voting"`, `"soft_voting"`, `"hard_voting"`, `"weighted_mean"` for regression |
+| `gnn.voting_weight_space` | Selects how GNN scores become voting weights; when omitted, GraphRoute chooses based on `loss_target`. | `None`, `"logit"`, `"sig"` |
+| `gnn.fallback` | Selects the fallback rule when no model receives a positive selection weight. | `"uniform"`, `"wacc"`, `"acc"`, `"bacc"` |
+
+
+## Reusing a trained pool
+
+GraphRoute automatically caches the results of the computation-heavy base-model training stage under pool_cache/<dataset>/pool_<pool-configuration-hash>/. The cache includes the final trained pool models and the model outputs required to construct the graph and train the GNN. The pool-configuration-hash identifies the specified model architectures and configuration for training base classifiers. Changing only the configuration for graph construction or GNN training leaves the hash unchanged, allowing the trained pool to be reused. Because `dataset` is part of the pool-cache path, use a different dataset name or delete the existing cache when the underlying data or preprocessing changes. Otherwise, GraphRoute may reuse stale models or predictions.
+
+## Development
+
+### Development installation
+
+```bash
+git clone https://github.com/briannamueller/GraphRoute.git
+cd GraphRoute
+pip install -e .
+```
+
+```bash
+pytest -q
+```
+
+The quickstart and test suite run offline on CPU.
+
+## License
+
+MIT -- see
+[LICENSE](https://github.com/briannamueller/GraphRoute/blob/main/LICENSE).

graphroute-0.1.0/README.md

@@ -0,0 +1,199 @@
+# GraphRoute
+
+GraphRoute is a graph-based dynamic ensemble selection framework. Given a pool
+of candidate models, a GNN trained over a sample similarity graph learns which
+models to trust for individual cases. Because different models have different
+inductive biases, their reliability varies across the input space. GraphRoute
+learns sample representations where proximity better reflects shared model
+competence. This approach is particularly valuable for problems where models
+optimized for aggregate performance tend to fail on rare edge cases, and where
+failures on such cases are the most consequential.
+
+GraphRoute operates in three stages:
+
+1. Train a pool of diverse models.
+2. Construct a graph where nodes represent samples and edges encode sample similarity.
+3. Train a GNN to produce per-classifier competence scores.
+
+- [Installation](#installation)
+- [Quickstart](#quickstart)
+- [Data and model interface](#data-and-model-interface)
+- [Configure Experiments](#configure-experiments)
+- [Configuration](#configuration)
+- [Reusing a trained pool](#reusing-a-trained-pool)
+- [Development](#development)
+
+## Installation
+
+```bash
+pip install graphroute
+```
+
+GraphRoute requires Python 3.10–3.12, PyTorch 2.0 or newer, and PyTorch
+Geometric 2.4 or newer.
+
+The quickstart and experiment runner are repository-level scripts. To use them,
+clone the repository and follow the [development installation](#development-installation).
+
+## Quickstart
+
+```bash
+python quickstart.py
+```
+
+The quickstart trains a pool of four lightweight models on synthetic data, fits
+a graph attention network (GAT), and compares the performance of the resulting
+GNN's dynamic selection with the individual classifiers and fixed ensemble
+baselines.
+
+## Data and model interface
+
+GraphRoute accepts PyTorch Dataset objects whose samples are (inputs, target) pairs. Datasets with
+custom sample structures can provide a `collate_fn`; see the `fit_graphroute`
+function documentation in
+[`graphroute/run.py`](https://github.com/briannamueller/GraphRoute/blob/main/graphroute/run.py)
+for the required interface. Supply the candidate model instances through the
+ordered `models` list.
+
+## Configure Experiments
+
+[`run_experiments.py`](https://github.com/briannamueller/GraphRoute/blob/main/run_experiments.py)
+reads the experiment configuration from
+[`configs/experiment.yaml`](https://github.com/briannamueller/GraphRoute/blob/main/configs/experiment.yaml)
+and model factories from
+[`model_registry.py`](https://github.com/briannamueller/GraphRoute/blob/main/model_registry.py).
+Register candidate models in `MODEL_REGISTRY`, specify their names in
+`base.models`, and run:
+
+```bash
+python run_experiments.py --config configs/experiment.yaml
+```
+
+The provided YAML defines one experiment. Uncomment its optional `sweep`
+section to run the Cartesian product of the listed values. Each completed
+configuration and its metrics are saved as a separate JSON file under
+`results/<dataset>/`. Repeating the command skips completed configurations.
+Use `--force` to rerun completed configurations.
+
+With the default `data_dir="data"`, GraphRoute reads `train.pt` and `test.pt`
+from `data/<dataset>/`; each file must contain an `(inputs, targets)` tuple saved
+using `torch.save`. `validation.pt` is optional. When it is absent, GraphRoute
+derives validation data from the training set using `val_ratio`.
+
+## Configuration
+
+Every available setting and its default is defined in
+[`graphroute/config.py`](https://github.com/briannamueller/GraphRoute/blob/main/graphroute/config.py).
+The YAML file uses the same field names. The tables below focus on settings
+whose options require an understanding of GraphRoute itself.
+
+### General
+
+| Argument | Meaning | Available options |
+| --- | --- | --- |
+| `loss_target` | Sets the GNN training objective. | `"meta_labels"`: minimizes the loss between predicted competence scores and targets that encode each model’s competence.<br>`"ensemble"`: minimizes the loss between the combined prediction and each sample’s ground-truth class label or regression target. |
+
+### Model pool training (`base`)
+
+| Argument | Meaning | Available options |
+| --- | --- | --- |
+| `base.models` | Names the ordered model pool using entries in the experiment registry. | Nonempty list of registered model names |
+| `base.split_mode` | Chooses how the pool produces out-of-sample predictions for GNN training. | `"oof_stacking"`, `"split_train"` |
+| `base.oof_folds` | Sets the number of folds used for OOF pool training. | Integer of at least `2` |
+
+
+To use all of the training data to train both the base classifiers and the GNN without the optimistic bias caused by evaluating models on their own training samples, base.split_mode=`"oof_stacking"` uses cross-validation to generate out-of-fold predictions for GNN training. The final classifiers used for inference are then trained on the full training set.
+
+`split_train` is less computationally expensive because each base classifier is trained only once. It divides the training data into two parts: one is used to train the base classifiers, and the other is used to train the GNN.
+
+### Graph construction (`graph`)
+
+<table>
+  <thead>
+    <tr>
+      <th>Argument</th>
+      <th>Meaning</th>
+      <th>Available options</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><code>graph.node_feature_source</code></td>
+      <td>Selects the sample representation supplied to the GNN.</td>
+      <td rowspan="2">
+        <code>"decision_space"</code>: concatenated pool predictions for the sample.<br>
+        <code>"feature_space"</code>: original features (flattened if not tabular already).<br>
+        <code>"embedding_mean"</code>: averages internal representation each model produces for the sample immediately before its final layer (requires same size embeddings).<br>
+        <code>"embedding_concat"</code>: concatenates internal representation each model produces for the sample immediately before its final layer (embedding sizes may differ).<br>
+        <code>"hybrid"</code>: decision-space representation and original features concatenated.
+      </td>
+    </tr>
+    <tr>
+      <td><code>graph.edge_feature_source</code></td>
+      <td>Selects the representation used to measure similarity when constructing graph edges.</td>
+    </tr>
+    <tr>
+      <td><code>graph.k</code></td>
+      <td>Sets the number of neighbors per sample.</td>
+      <td>Positive integer</td>
+    </tr>
+    <tr>
+      <td><code>graph.neighbor_mode</code></td>
+      <td>Selects ordinary nearest neighbors or class-balanced neighbors.</td>
+      <td><code>"knn"</code>, <code>"class_balanced"</code></td>
+    </tr>
+    <tr>
+      <td><code>graph.weight_mode</code></td>
+      <td>Determines how sample-to-sample edge weights are calculated.</td>
+      <td><code>"softmax"</code>, <code>"uniform"</code>, <code>"inverse_distance"</code>, <code>"cmdw"</code></td>
+    </tr>
+    <tr>
+      <td><code>graph.pool_calibrate</code></td>
+      <td>Enables or disables classification-pool calibration.</td>
+      <td><code>True</code>, <code>False</code></td>
+    </tr>
+    <tr>
+      <td><code>graph.calib_method</code></td>
+      <td>Selects the calibration method.</td>
+      <td><code>"ts-mix"</code>, <code>"logistic"</code></td>
+    </tr>
+  </tbody>
+</table>
+
+When oof_stacking is combined with embedding-based representations, GraphRoute uses out-of-fold predictions for GNN training, but extracts embeddings from the final base classifiers.
+
+### GNN training and dynamic selection (`gnn`)
+
+| Argument | Meaning | Available options |
+| --- | --- | --- |
+| `gnn.arch` | Selects the architecture used to learn the dynamic selection rule. | `"gat"`, `"graph_gps"`, `"mlp"` |
+| `gnn.loss` | Selects the GNN training loss. | `"bce"`, `"focal_bce"`, `"soft_bce"`, `"regression"` |
+| `gnn.ens_combination_mode` | Determines how model scores form the final prediction. | `"soft_weighted_voting"`, `"hard_weighted_voting"`, `"soft_voting"`, `"hard_voting"`, `"weighted_mean"` for regression |
+| `gnn.voting_weight_space` | Selects how GNN scores become voting weights; when omitted, GraphRoute chooses based on `loss_target`. | `None`, `"logit"`, `"sig"` |
+| `gnn.fallback` | Selects the fallback rule when no model receives a positive selection weight. | `"uniform"`, `"wacc"`, `"acc"`, `"bacc"` |
+
+
+## Reusing a trained pool
+
+GraphRoute automatically caches the results of the computation-heavy base-model training stage under pool_cache/<dataset>/pool_<pool-configuration-hash>/. The cache includes the final trained pool models and the model outputs required to construct the graph and train the GNN. The pool-configuration-hash identifies the specified model architectures and configuration for training base classifiers. Changing only the configuration for graph construction or GNN training leaves the hash unchanged, allowing the trained pool to be reused. Because `dataset` is part of the pool-cache path, use a different dataset name or delete the existing cache when the underlying data or preprocessing changes. Otherwise, GraphRoute may reuse stale models or predictions.
+
+## Development
+
+### Development installation
+
+```bash
+git clone https://github.com/briannamueller/GraphRoute.git
+cd GraphRoute
+pip install -e .
+```
+
+```bash
+pytest -q
+```
+
+The quickstart and test suite run offline on CPU.
+
+## License
+
+MIT -- see
+[LICENSE](https://github.com/briannamueller/GraphRoute/blob/main/LICENSE).

graphroute-0.1.0/configs/experiment.yaml

@@ -0,0 +1,33 @@
+dataset: my_dataset
+data_dir: data
+num_classes: 10
+seed: 0
+
+base:
+  models: [mlp64, mlp128, mlp256]
+  split_mode: oof_stacking
+  oof_folds: 5
+  epochs: 300
+  batch_size: 32
+  lr: 0.0005
+
+graph:
+  k: 5
+  node_feature_source: decision_space
+  edge_feature_source: decision_space
+  pool_calibrate: true
+
+gnn:
+  arch: gat
+  hidden_dim: 128
+  epochs: 300
+  patience: 20
+  es_metric: val_acc
+  ens_combination_mode: hard_weighted_voting
+  voting_weight_space: sig
+
+# Uncomment this section to run every combination of the listed values.
+# sweep:
+#   seed: [0, 1, 2]
+#   graph.k: [3, 5, 10]
+#   gnn.arch: [gat, graph_gps]

graphroute-0.1.0/graphroute/__init__.py

@@ -0,0 +1 @@
+"""GraphRoute: graph-based dynamic ensembling."""