protspace 3.2.0__tar.gz → 3.3.1__tar.gz

{protspace-3.2.0 → protspace-3.3.1}/.gitignore

@@ -1,6 +1,7 @@
 # ignore for now
 .DS_Store
 /.vscode
+/.claude
 /backup
 /bin
 /docs/publication

{protspace-3.2.0 → protspace-3.3.1}/CHANGELOG.md

@@ -1,6 +1,90 @@
 # CHANGELOG
 
 
+## v3.3.1 (2026-02-27)
+
+### Chores
+
+* chore: remove dead code and redundant logging.basicConfig calls
+
+- Remove 6 redundant logging.basicConfig() calls from library modules
+  (only the CLI entry point setup_logging() should configure logging)
+- Remove duplicate logger assignment in reducers.py
+- Replace bare print(e) with logger.error() in reducers.py
+- Remove commented-out dead code in local_data.py
+
+Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> ([`74f04f4`](https://github.com/tsenoner/protspace/commit/74f04f4aee338db09a3abcea699b3932b51ec01a))
+
+### Documentation
+
+* docs(notebook): overhaul Colab UI with interactive widgets
+
+Replace basic SelectMultiple widgets with a polished interactive UI:
+- Annotation selection: checkboxes in 2-column CSS grids per category
+  (UniProt/InterPro/Taxonomy) with per-group and global preset buttons
+- DR method selection: ToggleButtons with color feedback and tooltips
+- Method parameters: bordered cards in responsive flex-wrap grid with
+  shared params merged (n_neighbors for UMAP/PaCMAP/LocalMAP, etc.)
+- Dynamic show/hide of parameter groups based on selected methods
+- Remove gene_name from selectable annotations (always auto-included)
+- Add /.claude to .gitignore
+
+Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> ([`f671715`](https://github.com/tsenoner/protspace/commit/f671715ebf13eb808e7f44567b3c0407a5accb41))
+
+### Fixes
+
+* fix(reducers): robust float16 upcast, annoy fallback, suppress noisy warnings
+
+- Upcast float16 → float32 at HDF5 load time (local_processor) with a
+  safety-net in base_processor, preventing overflow in all DR methods
+- Add lazy annoy health check: on platforms where annoy segfaults or
+  returns empty results (e.g. macOS ARM64), transparently swap in an
+  sklearn NearestNeighbors drop-in so PaCMAP/LocalMAP keep working
+- Suppress harmless sklearn RuntimeWarnings (randomized SVD overflow),
+  FutureWarnings, and umap UserWarnings during fit_transform
+- Add LocalMAP to Colab notebook (METHODS list, parameter sliders,
+  intro text) — previously missing from the preparation UI
+
+Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> ([`7c8193e`](https://github.com/tsenoner/protspace/commit/7c8193ef0ef4c2b9df3a9031dc142f77b28d9fc6))
+
+### Testing
+
+* test(reducers): add comprehensive tests for all 6 DR methods
+
+51 tests covering PCA, t-SNE, UMAP, PaCMAP, MDS, and LocalMAP:
+- Per-method: output shape (2D/3D), NaN-free, get_params, determinism
+- Cross-cutting (parametrized): finite output, float dtype, no Inf
+- Float16 handling: verify upcast produces correct results
+- Config validation: defaults, custom values, invalid inputs
+- End-to-end: all methods through BaseProcessor.process_reduction
+
+Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> ([`85697c0`](https://github.com/tsenoner/protspace/commit/85697c00bd84885999a96bd4613eb3289bcb03bb))
+
+### Unknown
+
+* Merge pull request #34 from tsenoner/stage
+
+Robust reducers, test suite, and Colab UI overhaul ([`e1e416d`](https://github.com/tsenoner/protspace/commit/e1e416d1b51e7880b65a60c60d4d489f4f58645a))
+
+
+## v3.3.0 (2026-02-17)
+
+### Features
+
+* feat(styling): add pinnedValues, __REST__ marker, and value preprocessing
+
+Add legend ordering support to protspace-annotation-colors:
+- pinnedValues for explicit control over legend order and visible categories
+- __REST__ auto-fill marker to expand top values by frequency
+- zOrderSort to decouple zOrder computation from stored sortMode
+- Value preprocessing (pipe trimming, semicolon splitting) matching the
+  ProtSpace web frontend
+- Auto-assign Kelly's palette colors for pinned values, __NA__ key format
+- Comprehensive docs in docs/styling.md, docs/cli.md, and CLI epilog
+
+Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> ([`530cd3c`](https://github.com/tsenoner/protspace/commit/530cd3c90cb7acbc5ea35f3974eb39d0bb74c34c))
+
+
 ## v3.2.0 (2026-02-17)
 
 ### Code Style

{protspace-3.2.0 → protspace-3.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: protspace
-Version: 3.2.0
+Version: 3.3.1
 Summary: A visualisation tool for protein embeddings from pLMs
 Author-email: Tobias Senoner <tobias.senoner@tum.de>
 License-Expression: GPL-3.0

{protspace-3.2.0 → protspace-3.3.1}/docs/cli.md

@@ -59,7 +59,7 @@ protspace-query -q 'organism_name:"Homo sapiens" AND reviewed:true' -m pca2,umap
 
 ## `protspace-annotation-colors`
 
-Add custom colors, shapes, and display settings to existing ProtSpace files. See [Annotation Styling](styling.md) for the full styles JSON format.
+Add custom colors, shapes, legend ordering, and display settings to existing ProtSpace files. See [Annotation Styling](styling.md) for the full styles JSON format, including legend ordering with `pinnedValues`.
 
 ```bash
 # Generate a styles template (values in frequency order, empty color placeholders)
@@ -71,6 +71,14 @@ protspace-annotation-colors input.parquetbundle output.parquetbundle --annotatio
 # Apply styles from an inline JSON string
 protspace-annotation-colors input.parquetbundle output.parquetbundle --annotation_styles '{"ann": {"colors": {"val": "#FF0000"}}}'
 
+# Pin specific legend entries with N/A at the end
+protspace-annotation-colors input.parquetbundle output.parquetbundle --annotation_styles \
+  '{"ann": {"sortMode": "manual", "zOrderSort": "size-desc", "pinnedValues": ["val1", "val2", ""]}}'
+
+# Auto-fill top values by frequency, N/A at end
+protspace-annotation-colors input.parquetbundle output.parquetbundle --annotation_styles \
+  '{"ann": {"sortMode": "manual", "zOrderSort": "size-desc", "pinnedValues": ["__REST__", ""]}}'
+
 # Inspect stored settings
 protspace-annotation-colors data.parquetbundle --dump-settings
 ```

protspace-3.3.1/docs/styling.md

@@ -0,0 +1,122 @@
+# Annotation Styling
+
+Custom colors, shapes, and legend settings for annotation categories.
+
+**Two approaches:**
+
+- **Web UI** — interactive editing in [ProtSpace Web](https://protspace.app/explore), save to download the updated bundle
+- **CLI** — programmatic via `protspace-annotation-colors` (see [CLI Reference](cli.md#protspace-annotation-colors))
+
+## Workflow
+
+```bash
+# 1. Generate a styles template (values listed in frequency order)
+protspace-annotation-colors data.parquetbundle --generate-template > styles.json
+
+# 2. Edit styles.json — fill in colors, adjust settings
+
+# 3. Apply styles to produce a new bundle
+protspace-annotation-colors data.parquetbundle styled.parquetbundle --annotation_styles styles.json
+
+# 4. Verify stored settings
+protspace-annotation-colors styled.parquetbundle --dump-settings
+```
+
+## Styles JSON format
+
+Top-level keys are annotation names. Each annotation accepts the keys below.
+
+| Key                 | Type     | Default       | Stored | Description                                                                                      |
+| ------------------- | -------- | ------------- | ------ | ------------------------------------------------------------------------------------------------ |
+| `colors`            | `{}`     | —             | yes    | `{value: color}` — hex (`#FF0000`) or rgb (`rgb(255,0,0)`). Empty strings are ignored.           |
+| `shapes`            | `{}`     | —             | yes    | `{value: shape}` — one of `circle`, `square`, `diamond`, `triangle-up`, `triangle-down`, `plus`. |
+| `sortMode`          | string   | `"size-desc"` | yes    | Legend sort: `size-desc`, `size-asc`, `alpha-asc`, `alpha-desc`, `manual`.                       |
+| `maxVisibleValues`  | int      | `10`          | yes    | Legend entries shown before the "Other" bucket.                                                  |
+| `shapeSize`         | int      | `30`          | yes    | Marker size in the scatter plot.                                                                 |
+| `hiddenValues`      | string[] | `[]`          | yes    | Categories hidden from the plot.                                                                 |
+| `selectedPaletteId` | string   | `"kellys"`    | yes    | Color palette for categories without explicit colors.                                            |
+| `pinnedValues`      | string[] | —             | no     | Ordered list of values for legend positions 0..N-1. See [Legend ordering](#legend-ordering).     |
+| `zOrderSort`        | string   | —             | no     | Sort mode for zOrder assignment only (overrides `sortMode` for zOrder computation).              |
+
+**Stored** keys are persisted in the output bundle. **Non-stored** (processing-only) keys are consumed during generation — only their effects (the resulting categories with `zOrder`, `color`, `shape`) are written.
+
+### N/A values
+
+Missing values (`""`, `"<NA>"`, `"NaN"`) are normalized automatically — use any form in the styles file. In the output bundle N/A is stored with the key `__NA__` (the frontend's internal format).
+
+### Example: custom colors and shapes
+
+```json
+{
+  "major_group": {
+    "maxVisibleValues": 6,
+    "colors": {
+      "Short-chain": "#63CBE5",
+      "Long-chain": "#24638F",
+      "<NA>": "#C0C0C0"
+    },
+    "shapes": {
+      "Short-chain": "circle",
+      "Long-chain": "square"
+    }
+  }
+}
+```
+
+## Legend ordering
+
+By default (`sortMode: "size-desc"`), legend items are sorted by frequency with N/A sorted by its count like any other category. To control which values appear and in what order, use `pinnedValues` with `sortMode: "manual"`.
+
+### How `pinnedValues` works
+
+- Each value in the list receives a `zOrder` starting from 0. **Only pinned values** are written into the bundle's categories — the frontend treats everything else as "Other".
+- `sortMode: "manual"` tells the frontend to sort legend items by `zOrder` (i.e., the order you defined).
+- `maxVisibleValues` must match the number of pinned values. Example: 12 families + N/A = 13 entries requires `maxVisibleValues: 13`.
+- Colors are auto-assigned from Kelly's 21 Colors of Maximum Contrast when no explicit `colors` are provided. N/A gets a lighter gray (`#DDDDDD`).
+- Use `""` for N/A in `pinnedValues`.
+
+### `__REST__` auto-fill marker
+
+Instead of listing every value, use `"__REST__"` as a placeholder that expands to the top values by frequency (sorted by `zOrderSort`), filling up to `maxVisibleValues`.
+
+**Top 9 by frequency + N/A at end** (default `maxVisibleValues=10`):
+
+```json
+{
+  "ec": {
+    "sortMode": "manual",
+    "zOrderSort": "size-desc",
+    "pinnedValues": ["__REST__", ""]
+  }
+}
+```
+
+**Pin specific values + auto-fill + N/A:**
+
+```json
+{
+  "protein_families": {
+    "maxVisibleValues": 13,
+    "sortMode": "manual",
+    "zOrderSort": "size-desc",
+    "pinnedValues": ["familyA", "familyB", "__REST__", ""]
+  }
+}
+```
+
+Here `familyA` gets zOrder 0, `familyB` zOrder 1, the next 10 top-frequency values fill slots 2–11, and N/A gets slot 12.
+
+### `zOrderSort`
+
+Decouples zOrder assignment from the stored `sortMode`. Useful pattern: `zOrderSort: "size-desc"` computes frequency-based zOrders, while `sortMode: "manual"` tells the frontend to display in that order. Without `zOrderSort`, zOrder assignment falls back to `sortMode`.
+
+## Value preprocessing
+
+Raw annotation values are preprocessed to match the ProtSpace web frontend **before** settings are applied. **All settings (including `pinnedValues`) must use display names** (after preprocessing).
+
+| Delimiter     | Behavior                                                                                                                                          | Example                                           |
+| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------- |
+| Pipe `\|`     | Part after `\|` is a source tag — trimmed for display. Multiple raw variants with the same display name merge into one entry with combined count. | `"familyA\|IC"` → `"familyA"`                     |
+| Semicolon `;` | Multi-label split — each part becomes a separate entry. The protein counts toward all resulting categories.                                       | `"familyA;familyB"` → `"familyA"` and `"familyB"` |
+
+Combined: `"familyA|IC;familyB|SAM"` → split by `;` → `"familyA|IC"`, `"familyB|SAM"` → trim `|` → `"familyA"`, `"familyB"`.