ndscape 0.1.1__tar.gz → 0.1.2__tar.gz

ndscape-0.1.2/MANIFEST.in

@@ -0,0 +1 @@
+prune tests

{ndscape-0.1.1 → ndscape-0.1.2}/PKG-INFO

@@ -1,132 +1,136 @@
-Metadata-Version: 2.4
-Name: ndscape
-Version: 0.1.1
-Summary: Fit, score, and embed nested-dichotomy (ND) trees for multi-class classification.
-Author: Maxwell Dix-Matthews
-License-Expression: MIT
-Project-URL: Repository, https://github.com/res-lucid/ndscape
-Keywords: nested-dichotomy,multi-class,classification,scikit-learn
-Classifier: Programming Language :: Python :: 3
-Classifier: Intended Audience :: Science/Research
-Classifier: Topic :: Scientific/Engineering
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: numpy>=1.24
-Requires-Dist: scikit-learn>=1.3
-Requires-Dist: joblib>=1.2
-Provides-Extra: spatial
-Requires-Dist: esda>=2.6; extra == "spatial"
-Requires-Dist: libpysal>=4.10; extra == "spatial"
-Provides-Extra: plot
-Requires-Dist: matplotlib>=3.5; extra == "plot"
-Requires-Dist: bokeh>=3.0; extra == "plot"
-Dynamic: license-file
-
-# ndscape
-
-Fit, score, and embed nested-dichotomy (ND) trees for multi-class classification.
-
-A nested dichotomy reduces a C-class problem to a tree of binary splits
-(e.g. {0,1,2} vs {3,4}, then {0} vs {1,2}, ...). ndscape lets you fit one,
-score it, or place a whole population of candidate trees in a 2-D
-"tree-space" to see how a property (accuracy, variance, ...) varies across
-tree structures.
-
-## Install
-
-```
-pip install ndscape
-pip install ndscape[spatial]   # adds Moran's I support (esda, libpysal)
-pip install ndscape[plot]      # adds plotting (matplotlib, bokeh)
-```
-
-## Quickstart
-
-```python
-from sklearn.datasets import load_iris
-from sklearn.model_selection import train_test_split
-import ndscape as nds
-
-X, y = load_iris(return_X_y=True)
-classes = sorted(set(y))
-X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
-
-nd = nds.fit(X_train, y_train, classes=classes, base="lr")
-nd.predict(X_test)
-nd.score(X_test, y_test)   # {"accuracy": ..., "logloss": ...}
-```
-
-`classes` is the list of class labels in your `y` (`sorted(set(y))` works for
-integer or string labels). `nds.fit` samples one ND tree automatically; pass
-`tree=...` to use a specific one (see "A `tree` is..." below).
-
-## Use cases
-
-**You have a dataset and a binary classifier.**
-
-`base` can be the string `"lr"` or `"decisiontree"`, or your own unfitted
-scikit-learn estimator — a fresh clone of it is fit at every split.
-
-```python
-from sklearn.svm import SVC
-
-nd = nds.fit(X_train, y_train, classes=classes, base=SVC(probability=True, kernel="linear"))
-```
-
-**You have a train/test split and want a score.**
-
-```python
-nd = nds.ND(tree, classes).fit(X_train, y_train, base="lr")
-nd.score(X_test, y_test)   # {"accuracy": ..., "logloss": ...}
-```
-
-**You already trained the per-split models yourself.**
-
-```python
-# models in the same order as tree, or a {(left, right): model} dict — either works
-nd = nds.ND.from_trained(tree, classes, models=[fitted_model_1, fitted_model_2, ...])
-nd.predict_proba(X_test)
-```
-
-**You already scored a set of trees and want to see where they sit in tree-space.**
-
-```python
-trees, coords = nds.embed_trees(classes)
-nds.spatial_autocorrelation(my_scores, coords)   # {"I": ..., "p_sim": ...}
-```
-
-**You just want the whole picture: fit, score, and embed every candidate tree.**
-
-```python
-rows = nds.analyze(X_train, y_train, classes, X_test=X_test, y_test=y_test, base="lr")
-# [{"tree": ..., "accuracy": ..., "logloss": ..., "coord": array([...])}, ...]
-```
-
-**You want a picture of that tree-space.**
-
-```python
-nds.plot(rows, metric="accuracy", path="tree_space.png")        # static PNG/PDF
-nds.plot_interactive(rows, metric="accuracy", path="tree_space.html")  # pan/zoom/hover
-```
-
-Both color points by `metric` and mark the best tree with a black x. Needs
-`ndscape[plot]`.
-
-**The embedding is slow to recompute and you want to reuse it.**
-
-```python
-rows = nds.analyze(X_train, y_train, classes, cache="embedding.joblib")
-```
-
-The MDS step in `embed_trees`/`analyze` is the slow part. Pass `cache=` a
-`.joblib` path: the first call computes the embedding and saves it there,
-later calls with the same path just load it.
-
-A `tree` is a list of `(left, right)` tuples of class labels, e.g.
-`[((0, 1), (2, 3)), ((0,), (1,)), ((2,), (3,))]`. Use `nds.all_trees(classes)`
-(exhaustive, for small C) or `nds.sample_trees(classes, N)` (for larger C)
-to generate candidates.
-
-`base` accepts `"lr"`, `"decisiontree"`, or any unfitted scikit-learn
-estimator with `fit`/`predict_proba`.
+Metadata-Version: 2.4
+Name: ndscape
+Version: 0.1.2
+Summary: Fit, score, and embed nested-dichotomy (ND) trees for multi-class classification.
+Author: Maxwell Dix-Matthews
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/res-lucid/ndscape
+Project-URL: Repository, https://github.com/res-lucid/ndscape
+Project-URL: Issues, https://github.com/res-lucid/ndscape/issues
+Project-URL: Interactive maps, https://res-lucid.github.io/ndscape-plots/
+Project-URL: Data archive, https://doi.org/10.5281/zenodo.20456943
+Keywords: nested-dichotomy,multi-class,classification,scikit-learn
+Classifier: Programming Language :: Python :: 3
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy>=1.24
+Requires-Dist: scikit-learn>=1.3
+Requires-Dist: joblib>=1.2
+Provides-Extra: spatial
+Requires-Dist: esda>=2.6; extra == "spatial"
+Requires-Dist: libpysal>=4.10; extra == "spatial"
+Provides-Extra: plot
+Requires-Dist: matplotlib>=3.5; extra == "plot"
+Requires-Dist: bokeh>=3.0; extra == "plot"
+Dynamic: license-file
+
+# ndscape
+
+Fit, score, and embed nested-dichotomy (ND) trees for multi-class classification.
+
+A nested dichotomy reduces a C-class problem to a tree of binary splits
+(e.g. {0,1,2} vs {3,4}, then {0} vs {1,2}, ...). ndscape lets you fit one,
+score it, or place a whole population of candidate trees in a 2-D
+"tree-space" to see how a property (accuracy, variance, ...) varies across
+tree structures.
+
+## Install
+
+```
+pip install ndscape
+pip install ndscape[spatial]   # adds Moran's I support (esda, libpysal)
+pip install ndscape[plot]      # adds plotting (matplotlib, bokeh)
+```
+
+## Quickstart
+
+```python
+from sklearn.datasets import load_iris
+from sklearn.model_selection import train_test_split
+import ndscape as nds
+
+X, y = load_iris(return_X_y=True)
+classes = sorted(set(y))
+X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)
+
+nd = nds.fit(X_train, y_train, classes=classes, base="lr")
+nd.predict(X_test)
+nd.score(X_test, y_test)   # {"accuracy": ..., "logloss": ...}
+```
+
+`classes` is the list of class labels in your `y` (`sorted(set(y))` works for
+integer or string labels). `nds.fit` samples one ND tree automatically; pass
+`tree=...` to use a specific one (see "A `tree` is..." below).
+
+## Use cases
+
+**You have a dataset and a binary classifier.**
+
+`base` can be the string `"lr"` or `"decisiontree"`, or your own unfitted
+scikit-learn estimator — a fresh clone of it is fit at every split.
+
+```python
+from sklearn.svm import SVC
+
+nd = nds.fit(X_train, y_train, classes=classes, base=SVC(probability=True, kernel="linear"))
+```
+
+**You have a train/test split and want a score.**
+
+```python
+nd = nds.ND(tree, classes).fit(X_train, y_train, base="lr")
+nd.score(X_test, y_test)   # {"accuracy": ..., "logloss": ...}
+```
+
+**You already trained the per-split models yourself.**
+
+```python
+# models in the same order as tree, or a {(left, right): model} dict — either works
+nd = nds.ND.from_trained(tree, classes, models=[fitted_model_1, fitted_model_2, ...])
+nd.predict_proba(X_test)
+```
+
+**You already scored a set of trees and want to see where they sit in tree-space.**
+
+```python
+trees, coords = nds.embed_trees(classes)
+nds.spatial_autocorrelation(my_scores, coords)   # {"I": ..., "p_sim": ...}
+```
+
+**You just want the whole picture: fit, score, and embed every candidate tree.**
+
+```python
+rows = nds.analyze(X_train, y_train, classes, X_test=X_test, y_test=y_test, base="lr")
+# [{"tree": ..., "accuracy": ..., "logloss": ..., "coord": array([...])}, ...]
+```
+
+**You want a picture of that tree-space.**
+
+```python
+nds.plot(rows, metric="accuracy", path="tree_space.png")        # static PNG/PDF
+nds.plot_interactive(rows, metric="accuracy", path="tree_space.html")  # pan/zoom/hover
+```
+
+Both color points by `metric` and mark the best tree with a black x. Needs
+`ndscape[plot]`.
+
+**The embedding is slow to recompute and you want to reuse it.**
+
+```python
+rows = nds.analyze(X_train, y_train, classes, cache="embedding.joblib")
+```
+
+The MDS step in `embed_trees`/`analyze` is the slow part. Pass `cache=` a
+`.joblib` path: the first call computes the embedding and saves it there,
+later calls with the same path just load it.
+
+A `tree` is a list of `(left, right)` tuples of class labels, e.g.
+`[((0, 1), (2, 3)), ((0,), (1,)), ((2,), (3,))]`. Use `nds.all_trees(classes)`
+(exhaustive, for small C) or `nds.sample_trees(classes, N)` (for larger C)
+to generate candidates.
+
+`base` accepts `"lr"`, `"decisiontree"`, or any unfitted scikit-learn
+estimator with `fit`/`predict_proba`.

{ndscape-0.1.1 → ndscape-0.1.2}/README.md

@@ -1,46 +1,74 @@
-# Nested-Dichotomy Tree-Space Embedding
+# ndscape: Nested Dichotomy Tree-Space Embedding
 
-Code for the paper. Embeds the space of nested-dichotomy (ND) structures via depth-weighted tokenisation and metric MDS, then maps bias, variance, and held-out performance across that space.
+[![PyPI](https://img.shields.io/pypi/v/ndscape.svg)](https://pypi.org/project/ndscape/)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.20456943.svg)](https://doi.org/10.5281/zenodo.20456943)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 
-## Installation
+A classification model assigns an observation to one of several classes, and when there are more than two, the task becomes a multiclass problem. A nested dichotomy (ND) gives that multiclass problem a visible structure: it arranges the classes in a binary tree and fits a classifier at each split, so the final prediction is built from a sequence of simpler binary decisions.
 
-```bash
-pip install -r requirements.txt
-```
+Any binary classifier can be used at the nodes. With probabilistic classifiers, the probability of each class is the product of the branch probabilities along its path from root to leaf. And because the tree records which classes are grouped together, where they separate, and in what order, you can inspect the fitted model and compare its structure with what you already know about the problem.
+
+That structure also matters for prediction. There are (2M−3)!! possible NDs for M classes, each defining a different decomposition of the same classification task, and changing the tree can produce very different fitted models even when the data and binary classifier stay the same. This gives NDs a large, interpretable model space where changes in predictive behaviour can be related directly to changes in structure.
 
-Versions of `numpy`, `scipy`, and `scikit-learn` are pinned in `requirements.txt`. The MDS coordinates in `cache_nd/` were built with those exact versions; changing them will produce different (but statistically equivalent) coordinates.
+`ndscape` embeds ND tree structures using depth-weighted structural tokens and metric MDS, so structurally similar trees sit close together. It can be used as a Python package, and this repository also contains the code needed to reproduce the paper outputs.
 
-## Quick check (~30 s)
+Interactive maps from the paper: [res-lucid.github.io/ndscape-plots](https://res-lucid.github.io/ndscape-plots/)
+
+## Installation
+
+For package use:
 
 ```bash
-pytest tests/
+pip install ndscape
 ```
 
-To also run the slower integration tests (require pre-built C=6 and C=7 cache files, or rebuild from scratch):
-
+To reproduce the paper outputs, clone the repository and install the pinned requirements:
 ```bash
-pytest tests/ -m integration
+python -m venv venv
+source venv/bin/activate  # Windows: venv\Scripts\activate
+pip install -r requirements.txt
 ```
 
-## Reproduce paper outputs (fast — no re-simulation)
+Versions of `numpy`, `scipy`, and `scikit-learn` are pinned in `requirements.txt`. The MDS coordinates in `cache_nd/` were built with those exact versions, so changing numpy, scipy, or scikit-learn can produce slightly different coordinates.
+
+## How to use this repository
+
+| Goal | Command |
+|------|---------|
+| Use `ndscape` as a package | `pip install ndscape` |
+| Test the repository install | `pytest tests/` |
+| Recreate paper figures and tables from archived outputs | `python scripts/download_zenodo.py` then `python reproduce.py --paper` |
+| Run the full reproduction pipeline | `python reproduce.py` |
+| Run only selected steps | `python reproduce.py --steps artifacts` |
+| Score a new dataset | `python run_datasets.py --datasets my_dataset` |
+
+## Reproduce paper outputs from archived results
 
-Download the Zenodo deposit first, then regenerate all figures and tables in minutes:
+Download the Zenodo deposit, then regenerate the paper figures and tables without re-running the simulations:
 
 ```bash
 python scripts/download_zenodo.py
 python reproduce.py --paper
 ```
 
-`--paper` runs four scripts in order:
+<details>
+<summary>Scripts run by <code>python reproduce.py --paper</code></summary>
 
-1. `scripts/make_dataset_plots.py --cache zenodo --out paper_outputs/figures` — PDF and interactive HTML embedding maps for all 6 real datasets
-2. `scripts/make_simulation_plots.py --out paper_outputs/figures` — simulation bias/variance/depth PDF maps (Figs 2–5 and appendix)
-3. `scripts/make_ablation_summaries.py` — Moran spatial analysis for the tokeniser / BoW / RF ablation (reads `zenodo/raw_simulation_results/`)
-4. `scripts/make_paper_tables.py --all --out paper_outputs/tables` — all LaTeX tables; uses pre-built scores from `zenodo/simulation_scores/` and `zenodo/simulation_scores_dt/` if present, otherwise recomputes from `zenodo/raw_simulation_results{,_dt}/`
+1. `scripts/make_dataset_plots.py --cache zenodo --out paper_outputs/figures`
+   - PDF and interactive HTML embedding maps for all six real datasets.
+2. `scripts/make_simulation_plots.py --out paper_outputs/figures`
+   - Simulation bias, variance, and depth maps.
+3. `scripts/make_ablation_summaries.py`
+   - Moran spatial analysis for the tokeniser, BoW, and RF ablations.
+4. `scripts/make_paper_tables.py --all --out paper_outputs/tables`
+   - LaTeX tables.
 
-Output lands in `paper_outputs/figures/` and `paper_outputs/tables/`.
+Output is saved to `paper_outputs/figures/` and `paper_outputs/tables/`.
+</details>
 
-## Full reproduction from scratch
+## Full reproduction pipeline
+
+This rebuilds the derived outputs locally, using the archived data and configuration files downloaded into `zenodo/`.
 
 ```bash
 python reproduce.py
@@ -67,18 +95,24 @@ python reproduce.py --steps plots
 
 **Note on step 2:** `run_datasets.py` scores every ND tree on each real dataset and writes xlsx results. These are also available pre-built in `zenodo/tree_metrics/` and are used automatically by `make_dataset_plots.py` and `make_dataset_tables.py` when the xlsx files are absent.
 
-## Real-data tables and figures without re-scoring
+## Using your own data
 
-`make_dataset_plots.py` and `make_dataset_tables.py` fall back to `zenodo/tree_metrics/<dataset>.csv` when the step-2 xlsx files are not present. After running `python scripts/download_zenodo.py`, the `--paper` command produces all real-data figures and tables without running step 2.
+Drop a CSV in `zenodo/data/` with a `y` column (integer or string class labels) and any number of numeric feature columns. A `split` column (`"train"` / `"test"`) is used if present, otherwise a stratified 70/30 split is applied.
+
+```bash
+python run_datasets.py --datasets my_dataset
+```
+
+Output goes to `real-data-bias-analysis/my_dataset/v_my_dataset_lr_cluster_tables.xlsx`.
 
 ## Tests
 
 ```bash
 pytest tests/                  # unit + snapshot tests (fast)
-pytest tests/ -m integration   # integration tests (slow; requires pre-built C=6,7 artifacts)
+pytest tests/ -m integration   # integration tests (slow and requires pre-built C=6,7 artifacts)
 ```
 
-The `tests/fixtures/` directory holds MD5 hashes of the tree lists, tokeniser output, and per-dataset scores.
+The `tests/fixtures/` directory holds golden-master fixtures: MD5 hashes for the larger artifacts, and full tree lists and generator outputs for the smaller ones.
 
 ## Datasets
 
@@ -112,38 +146,18 @@ Or place CSV files (with a `y` column for the class label) in `zenodo/data/`:
 
 ## Key files
 
-```
+```text
+reproduce.py                          Main reproduction entry point
 core.py                               Tree enumeration, embedding, models, artifact building
-generate_data.py                      Synthetic data generation for the simulation
-run_datasets.py                       Score all trees on each real dataset; write xlsx results
-run_simulations.py                    Bias-variance simulation (numba-accelerated)
-reproduce.py                          Single-command entry point (--paper for fast path)
-
-nd/                                   ND algorithm implementations
-  NestedDichotomy.py                  Core tree data structure and training
-  RandomGeneration.py                 Uniform random tree sampling
-  BBoK.py                             Balanced best-of-K generator
-  BestOfK.py                          Best-of-K wrapper (K=10)
-  ACND.py, NDC.py, RPND.py, CBND.py  Four further construction heuristics
-
-scripts/make_artifacts.py             Build MDS embeddings from configs/artifacts.yml
-scripts/verify_artifacts.py           Check built artifacts against configs/artifacts.yml
-scripts/make_dataset_plots.py                 Embedding PDF + HTML maps per real dataset
-scripts/make_simulation_plots.py      Simulation bias/variance/depth PDF maps
-scripts/moran_simulation_analysis.py  Spatial statistics on simulation results
-scripts/make_dataset_tables.py                LaTeX real-data tables (spatial stats + heuristic placement)
-scripts/make_simulation_tables.py     LaTeX simulation tables
-scripts/make_ablation_summaries.py    Moran analysis for tokeniser/BoW/RF ablations
-scripts/make_ablation_tables.py       LaTeX ablation tables
-scripts/make_paper_tables.py          Orchestrates all tables → paper_outputs/tables/
-scripts/download_zenodo.py            Download pre-built data from Zenodo
-scripts/csv_to_configs.py             Convert zenodo/simulation_configs.csv to JSON
-
-configs/artifacts.yml                 Manifest of all artifacts to build
-cache_nd/                             Cached .joblib artifacts (auto-created by step 1)
-zenodo/                               Pre-built deposit data (see Zenodo section below)
-real-data-bias-analysis/              Scoring results (auto-created by step 2)
-bias-analysis/raw_simulation_results/ Simulation bootstrap CSVs (auto-created by step 3)
+run_datasets.py                       Score all trees on real datasets
+run_simulations.py                    Bias-variance simulation
+nd/                                   Nested dichotomy implementations
+scripts/                              Figure, table, artifact, download, and analysis scripts
+configs/artifacts.yml                 Artifact specifications
+cache_nd/                             Cached embedding artifacts
+zenodo/                               Downloaded Zenodo archive
+real-data-bias-analysis/              Real-data scoring outputs
+bias-analysis/raw_simulation_results/ Simulation bootstrap outputs
 ```
 
 ## Zenodo
@@ -202,22 +216,11 @@ To convert the LR CSV to JSON:
 python scripts/csv_to_configs.py --output simulation_configs.json
 ```
 
-`zenodo/simulation_configs_dt.csv` is committed as a static artifact; it was generated once from
-the original Postgres simulation DB and is not regenerated as part of this pipeline.
-
-## Using your own data
-
-Drop a CSV in `zenodo/data/` with a `y` column (integer or string class labels) and any number of numeric feature columns. A `split` column (`"train"` / `"test"`) is used if present; otherwise a stratified 70/30 split is applied.
-
-```bash
-python run_datasets.py --datasets my_dataset
-```
-
-Output goes to `real-data-bias-analysis/my_dataset/v_my_dataset_lr_cluster_tables.xlsx`.
+`zenodo/simulation_configs_dt.csv` is committed as a static artifact and is not regenerated as part of this pipeline.
 
 ## Artifacts (cache_nd/)
 
-The `cache_nd/` directory holds `.joblib` artifacts for C=4 through C=19. All artifacts are tracked via Git LFS (`.gitattributes`) and are also available from Zenodo:
+The `cache_nd/` directory holds `.joblib` artifacts for C=4 through C=19. It is not tracked in git — the artifacts are downloaded from Zenodo into `zenodo/` (tracked with Git LFS, see `.gitattributes`) or rebuilt locally:
 
 ```bash
 python scripts/download_zenodo.py --what artifacts
@@ -231,3 +234,7 @@ python scripts/make_artifacts.py --all
 
 Artifact specs (class count, tree count, seed) are declared in `configs/artifacts.yml`.
 Use `scripts/verify_artifacts.py` to check a built artifact against its spec.
+
+## Citation
+
+Metadata is in [`CITATION.cff`](CITATION.cff); GitHub renders it under "Cite this repository". The archived data and artifacts have their own DOI, [10.5281/zenodo.20456943](https://doi.org/10.5281/zenodo.20456943).

{ndscape-0.1.1 → ndscape-0.1.2}/ndscape/api.py

@@ -35,7 +35,7 @@ def _get_model(base):
     if isinstance(base, str):
         if base == "lr":
             return LogisticRegression(
-                penalty="l2", solver="newton-cholesky", C=0.1, max_iter=2000
+                solver="newton-cholesky", C=0.1, max_iter=2000
             )
         if base in ("decisiontree", "dt"):
             return DecisionTreeClassifier(