msfiddle 2.0.1__tar.gz → 2.1.0__tar.gz

{msfiddle-2.0.1 → msfiddle-2.1.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: msfiddle
-Version: 2.0.1
+Version: 2.1.0
 Summary: A package for predicting chemical formulas from tandem mass spectra
 Home-page: https://github.com/JosieHong/msfiddle
 Author: Yuhui Hong
@@ -64,17 +64,13 @@ For the full experimental codebase, see https://github.com/JosieHong/FIDDLE.
 ## Installation
 
 ```bash
-pip install msfiddle
+pip install msfiddle              # base install, no PyTorch
+pip install "msfiddle[inference]" # base install + PyTorch (needed for predictions and the CLI)
 ```
 
-PyTorch is required for inference. Install the optional inference extra, 
-or install PyTorch separately for your platform:
-```bash
-pip install "msfiddle[inference]"
-```
-
-See the official PyTorch installation guide for custom CUDA builds:
-https://pytorch.org/get-started/locally/.
+For a custom PyTorch build (CUDA, ROCm, Apple Silicon, etc.), install torch
+first via the [official guide](https://pytorch.org/get-started/locally/), then
+`pip install msfiddle`.
 
 ## Usage
 
@@ -90,7 +86,7 @@ msfiddle-download-models --destination /path/to/models \
                           --models fiddle_tcn_qtof fiddle_rescore_qtof
 ```
 
-`msfiddle` 2.0.1 reuses the FIDDLE `v2.0.0` checkpoint assets.
+All `msfiddle` 2.x releases reuse the FIDDLE `v2.0.0` checkpoint assets.
 
 Run the packaged demo:
 ```bash
@@ -190,6 +186,44 @@ The Python `predict_from_spectrum()` API returns a list of candidate dictionarie
 `predict_batch()` returns one record per input spectrum with `id`, `candidates`,
 and `metadata`.
 
+### Native/original BUDDY and SIRIUS outputs
+
+Use the same MGF input file to generate native
+[BUDDY/msbuddy](https://msbuddy.readthedocs.io/en/latest/cmdapi.html) and
+[SIRIUS](https://v6.docs.sirius-ms.io/cli/) outputs. `msfiddle` accepts these
+native/original files directly through `--buddy_path` and `--sirius_path`. The
+older msfiddle-normalized CSV formats documented in `docs/formats.rst` are
+**deprecated** and will be removed in `msfiddle` 3.0.0; loading them emits a
+`DeprecationWarning`.
+
+```bash
+msbuddy -mgf /path/to/data.mgf \
+        -output /path/to/buddy_output \
+        -ms orbitrap \
+        -p -n_cpu 12 \
+        -d -hal
+```
+
+`msbuddy` writes `msbuddy_result_summary.tsv`; `-d` also writes detailed
+per-query `formula_results.tsv` files. Pass either the summary file or the
+output directory to `--buddy_path`. The full output directory is preferred
+because it includes per-candidate FDR scores for ranks beyond rank 1.
+
+```bash
+sirius --input /path/to/data.mgf \
+       --project /path/to/sirius_project \
+       formulas --profile orbitrap
+
+sirius --project /path/to/sirius_project \
+       summaries --top-k-summary=5 \
+       --output /path/to/sirius_output
+```
+
+SIRIUS writes a project space and exports summary files such as
+`formula_identifications.tsv`. Pass either a formula-identification summary file
+or the summary output directory to `--sirius_path`.
+SIRIUS 6 may require `sirius login` before formula computation.
+
 ### MGF input
 
 The required MGF fields are `TITLE`, `PRECURSOR_MZ`, `PRECURSOR_TYPE`, and

{msfiddle-2.0.1 → msfiddle-2.1.0}/README.md

@@ -23,17 +23,13 @@ For the full experimental codebase, see https://github.com/JosieHong/FIDDLE.
 ## Installation
 
 ```bash
-pip install msfiddle
+pip install msfiddle              # base install, no PyTorch
+pip install "msfiddle[inference]" # base install + PyTorch (needed for predictions and the CLI)
 ```
 
-PyTorch is required for inference. Install the optional inference extra, 
-or install PyTorch separately for your platform:
-```bash
-pip install "msfiddle[inference]"
-```
-
-See the official PyTorch installation guide for custom CUDA builds:
-https://pytorch.org/get-started/locally/.
+For a custom PyTorch build (CUDA, ROCm, Apple Silicon, etc.), install torch
+first via the [official guide](https://pytorch.org/get-started/locally/), then
+`pip install msfiddle`.
 
 ## Usage
 
@@ -49,7 +45,7 @@ msfiddle-download-models --destination /path/to/models \
                           --models fiddle_tcn_qtof fiddle_rescore_qtof
 ```
 
-`msfiddle` 2.0.1 reuses the FIDDLE `v2.0.0` checkpoint assets.
+All `msfiddle` 2.x releases reuse the FIDDLE `v2.0.0` checkpoint assets.
 
 Run the packaged demo:
 ```bash
@@ -149,6 +145,44 @@ The Python `predict_from_spectrum()` API returns a list of candidate dictionarie
 `predict_batch()` returns one record per input spectrum with `id`, `candidates`,
 and `metadata`.
 
+### Native/original BUDDY and SIRIUS outputs
+
+Use the same MGF input file to generate native
+[BUDDY/msbuddy](https://msbuddy.readthedocs.io/en/latest/cmdapi.html) and
+[SIRIUS](https://v6.docs.sirius-ms.io/cli/) outputs. `msfiddle` accepts these
+native/original files directly through `--buddy_path` and `--sirius_path`. The
+older msfiddle-normalized CSV formats documented in `docs/formats.rst` are
+**deprecated** and will be removed in `msfiddle` 3.0.0; loading them emits a
+`DeprecationWarning`.
+
+```bash
+msbuddy -mgf /path/to/data.mgf \
+        -output /path/to/buddy_output \
+        -ms orbitrap \
+        -p -n_cpu 12 \
+        -d -hal
+```
+
+`msbuddy` writes `msbuddy_result_summary.tsv`; `-d` also writes detailed
+per-query `formula_results.tsv` files. Pass either the summary file or the
+output directory to `--buddy_path`. The full output directory is preferred
+because it includes per-candidate FDR scores for ranks beyond rank 1.
+
+```bash
+sirius --input /path/to/data.mgf \
+       --project /path/to/sirius_project \
+       formulas --profile orbitrap
+
+sirius --project /path/to/sirius_project \
+       summaries --top-k-summary=5 \
+       --output /path/to/sirius_output
+```
+
+SIRIUS writes a project space and exports summary files such as
+`formula_identifications.tsv`. Pass either a formula-identification summary file
+or the summary output directory to `--sirius_path`.
+SIRIUS 6 may require `sirius login` before formula computation.
+
 ### MGF input
 
 The required MGF fields are `TITLE`, `PRECURSOR_MZ`, `PRECURSOR_TYPE`, and

{msfiddle-2.0.1 → msfiddle-2.1.0}/msfiddle/__init__.py

@@ -2,7 +2,12 @@
 msfiddle: A package for predicting chemical formulas from tandem mass spectra
 """
 
-__version__ = "2.0.1"
+from importlib.metadata import PackageNotFoundError, version as _pkg_version
+
+try:
+    __version__ = _pkg_version("msfiddle")
+except PackageNotFoundError:  # running from a source checkout without install
+    __version__ = "0.0.0+unknown"
 
 from .api import (
     MsFiddlePredictor,

{msfiddle-2.0.1 → msfiddle-2.1.0}/msfiddle/api.py

@@ -20,6 +20,7 @@ from pyteomics import mgf
 
 from .download import download_models as download_pretrained_models
 from .download import get_checkpoint_dir
+from .external_results import load_buddy_results, load_sirius_results
 from .utils.mol_utils import (
     ATOMS_INDEX,
     formula_to_dict,
@@ -198,6 +199,10 @@ class MsFiddlePredictor:
     ) -> pd.DataFrame:
         """Predict formulas for spectra in an MGF file.
 
+        ``buddy_path`` and ``sirius_path`` may point to either native/original
+        BUDDY/msbuddy and SIRIUS formula-identification outputs or legacy
+        msfiddle-normalized CSV files.
+
         The returned DataFrame uses the same result columns as the CLI CSV.
         """
 
@@ -212,8 +217,8 @@ class MsFiddlePredictor:
         encoded = self._load_mgf_spectra(test_data_path)
         predictions = self._predict_encoded(encoded, batch_size=batch_size)
 
-        buddy_df = pd.read_csv(buddy_path) if buddy_path else None
-        sirius_df = pd.read_csv(sirius_path) if sirius_path else None
+        buddy_df = load_buddy_results(buddy_path) if buddy_path else None
+        sirius_df = load_sirius_results(sirius_path) if sirius_path else None
 
         refined_formula = {f"Refined Formula ({k})": [] for k in range(top_k)}
         refined_mass = {f"Refined Mass ({k})": [] for k in range(top_k)}
@@ -787,7 +792,8 @@ class MsFiddlePredictor:
                 candidates.extend(
                     formula
                     for formula, score in zip(formulas, scores)
-                    if str(formula) != "nan"
+                    if _is_external_formula(formula)
+                    and pd.notna(score)
                     and score < self.config["post_processing"]["buddy_fdr_thr"]
                 )
         if sirius_df is not None:
@@ -815,7 +821,8 @@ class MsFiddlePredictor:
                 candidates.extend(
                     formula
                     for formula, score in zip(formulas, scores)
-                    if str(formula) != "nan"
+                    if _is_external_formula(formula)
+                    and pd.notna(score)
                     and score > self.config["post_processing"]["sirius_score_thr"]
                 )
         return candidates
@@ -934,6 +941,12 @@ def _normalize_instrument_type(instrument_type: str) -> str:
     return normalized
 
 
+def _is_external_formula(value: object) -> bool:
+    if pd.isna(value):
+        return False
+    return str(value).strip().lower() not in {"", "na", "nan", "none", "null", "<na>"}
+
+
 def _load_config(config_path: Path) -> dict[str, Any]:
     if not config_path.exists():
         raise FileNotFoundError(f"Config file not found: {config_path}")

msfiddle-2.1.0/msfiddle/external_results.py

@@ -0,0 +1,380 @@
+"""Load external formula candidates from BUDDY/msbuddy and SIRIUS outputs."""
+
+from __future__ import annotations
+
+import re
+import warnings
+from pathlib import Path
+from typing import Iterable
+
+import pandas as pd
+
+_NORMALIZED_CSV_DEPRECATION = (
+    "The msfiddle-normalized {tool} CSV format is deprecated and will be "
+    "removed in msfiddle 3.0.0. Pass the native/original {tool} output "
+    "(directory or summary file) instead."
+)
+
+BUDDY_FORMULA_COLUMNS = tuple(f"Pred Formula ({idx})" for idx in range(1, 6))
+BUDDY_SCORE_COLUMNS = tuple(f"BUDDY Score ({idx})" for idx in range(1, 6))
+BUDDY_COLUMNS = ("ID", "Adduct", *BUDDY_FORMULA_COLUMNS, *BUDDY_SCORE_COLUMNS)
+
+SIRIUS_FORMULA_COLUMNS = tuple(f"Pred Formula ({idx})" for idx in range(1, 6))
+SIRIUS_ADDUCT_COLUMNS = tuple(f"Pred Adduct ({idx})" for idx in range(1, 6))
+SIRIUS_SCORE_COLUMNS = tuple(f"SIRIUS Score ({idx})" for idx in range(1, 6))
+SIRIUS_COLUMNS = (
+    "ID",
+    *(
+        value
+        for idx in range(1, 6)
+        for value in (
+            f"Pred Formula ({idx})",
+            f"Pred Adduct ({idx})",
+            f"SIRIUS Score ({idx})",
+        )
+    ),
+)
+
+_BUDDY_NATIVE_COLUMNS = (
+    "identifier",
+    "adduct",
+    "formula_rank_1",
+    "estimated_fdr",
+)
+_SIRIUS_FORMULA_FILES = (
+    "formula_identifications_top-",
+    "formula_identifications_all",
+    "formula_identifications_adducts_top-",
+    "formula_identifications_adducts_all",
+    "formula_identifications",
+)
+_SIRIUS_ID_COLUMNS = (
+    "name",
+    "mappingFeatureId",
+    "alignedFeatureId",
+    "featureId",
+    "compoundId",
+    "id",
+)
+_SIRIUS_FORMULA_COLUMNS = ("molecularFormula", "formula", "Formula")
+_SIRIUS_ADDUCT_COLUMNS = ("adduct", "ionType", "ion", "precursorIonType")
+_SIRIUS_SCORE_COLUMNS = (
+    "SiriusScore",
+    "SIRIUS Score",
+    "rankingScore",
+    "ZodiacScore",
+    "ConfidenceScore",
+    "score",
+)
+_SIRIUS_RANK_COLUMNS = ("rank", "formulaRank")
+
+
+def load_buddy_results(path: str | Path) -> pd.DataFrame:
+    """Load native/original or msfiddle-normalized BUDDY/msbuddy results.
+
+    Native/original msbuddy inputs may be either the
+    ``msbuddy_result_summary.tsv`` file or the output directory that contains
+    it. If detailed ``formula_results.tsv`` files are present, their
+    per-candidate FDR values are used.
+    """
+
+    resolved = _resolve_path(path, "BUDDY")
+    if resolved.is_dir():
+        return _load_msbuddy_directory(resolved)
+
+    df = _read_table(resolved)
+    if _has_columns(df, BUDDY_COLUMNS):
+        warnings.warn(
+            _NORMALIZED_CSV_DEPRECATION.format(tool="BUDDY"),
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        return _normalize_buddy(df)
+    if _has_columns(df, _BUDDY_NATIVE_COLUMNS):
+        return _normalize_msbuddy_summary(df)
+    raise ValueError(
+        "Unsupported BUDDY result format. Expected a native/original msbuddy "
+        "summary or output directory, or an msfiddle-normalized CSV. Native "
+        "summary columns: "
+        f"{', '.join(_BUDDY_NATIVE_COLUMNS)}"
+    )
+
+
+def load_sirius_results(path: str | Path) -> pd.DataFrame:
+    """Load native/original or msfiddle-normalized SIRIUS results."""
+
+    resolved = _resolve_path(path, "SIRIUS")
+    if resolved.is_dir():
+        resolved = _find_sirius_formula_file(resolved)
+
+    df = _read_table(resolved)
+    if _has_columns(df, SIRIUS_COLUMNS):
+        warnings.warn(
+            _NORMALIZED_CSV_DEPRECATION.format(tool="SIRIUS"),
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        return _normalize_sirius(df)
+    return _normalize_sirius_native(df)
+
+
+def _resolve_path(path: str | Path, label: str) -> Path:
+    resolved = Path(path)
+    if not resolved.exists():
+        raise FileNotFoundError(f"{label} result path not found: {resolved}")
+    return resolved
+
+
+def _read_table(path: Path) -> pd.DataFrame:
+    suffix = path.suffix.lower()
+    if suffix in {".xlsx", ".xls"}:
+        return _clean_columns(pd.read_excel(path))
+    if suffix == ".tsv":
+        return _clean_columns(pd.read_csv(path, sep="\t"))
+    if suffix == ".csv":
+        return _clean_columns(pd.read_csv(path))
+    return _clean_columns(pd.read_csv(path, sep=None, engine="python"))
+
+
+def _clean_columns(df: pd.DataFrame) -> pd.DataFrame:
+    df = df.copy()
+    df.columns = [str(column).strip().lstrip("\ufeff") for column in df.columns]
+    return df
+
+
+def _has_columns(df: pd.DataFrame, columns: Iterable[str]) -> bool:
+    df_columns = set(df.columns)
+    return all(column in df_columns for column in columns)
+
+
+def _normalize_buddy(df: pd.DataFrame) -> pd.DataFrame:
+    normalized = df.loc[:, BUDDY_COLUMNS].copy()
+    normalized["ID"] = normalized["ID"].map(_clean_identifier)
+    normalized["Adduct"] = normalized["Adduct"].map(_clean_text)
+    for column in BUDDY_FORMULA_COLUMNS:
+        normalized[column] = normalized[column].map(_clean_formula)
+    for column in BUDDY_SCORE_COLUMNS:
+        normalized[column] = pd.to_numeric(normalized[column], errors="coerce")
+    return normalized.dropna(subset=["ID"]).reset_index(drop=True)
+
+
+def _load_msbuddy_directory(path: Path) -> pd.DataFrame:
+    summary_path = path / "msbuddy_result_summary.tsv"
+    if not summary_path.exists():
+        raise FileNotFoundError(
+            "Native msbuddy output directory must contain "
+            f"msbuddy_result_summary.tsv: {path}"
+        )
+    summary = _read_table(summary_path)
+    details = _load_msbuddy_details(path, summary)
+    return _normalize_msbuddy_summary(summary, details=details)
+
+
+def _normalize_msbuddy_summary(
+    df: pd.DataFrame,
+    *,
+    details: dict[str, list[tuple[str, float]]] | None = None,
+) -> pd.DataFrame:
+    if not _has_columns(df, _BUDDY_NATIVE_COLUMNS):
+        raise ValueError(
+            "Native msbuddy summary is missing required columns: "
+            f"{', '.join(_BUDDY_NATIVE_COLUMNS)}"
+        )
+
+    rows: list[dict[str, object]] = []
+    for _, row in df.iterrows():
+        spec_id = _clean_identifier(row["identifier"])
+        out: dict[str, object] = {
+            "ID": spec_id,
+            "Adduct": _clean_text(row.get("adduct", pd.NA)),
+        }
+
+        detail_candidates = details.get(spec_id, []) if details else []
+        if detail_candidates:
+            for idx, (formula, score) in enumerate(detail_candidates[:5], start=1):
+                out[f"Pred Formula ({idx})"] = _clean_formula(formula)
+                out[f"BUDDY Score ({idx})"] = score
+        else:
+            for idx in range(1, 6):
+                out[f"Pred Formula ({idx})"] = _clean_formula(
+                    row.get(f"formula_rank_{idx}", pd.NA)
+                )
+                # msbuddy summary exposes estimated FDR for rank 1 only.
+                out[f"BUDDY Score ({idx})"] = (
+                    _to_number(row.get("estimated_fdr", pd.NA)) if idx == 1 else pd.NA
+                )
+        rows.append(out)
+    return _normalize_buddy(pd.DataFrame(rows, columns=BUDDY_COLUMNS))
+
+
+def _load_msbuddy_details(
+    path: Path, summary: pd.DataFrame
+) -> dict[str, list[tuple[str, float]]]:
+    # Detail directories follow `{sanitized_id}_mz_{mz}_rt_{rt}`. Index by the
+    # sanitized id so we don't depend on how msbuddy formats mz.
+    by_sanitized_id: dict[str, str] = {}
+    for _, row in summary.iterrows():
+        spec_id = _clean_identifier(row["identifier"])
+        if pd.isna(spec_id):
+            continue
+        by_sanitized_id[_sanitize_msbuddy_identifier(spec_id)] = spec_id
+
+    details: dict[str, list[tuple[str, float]]] = {}
+    for result_path in path.glob("**/formula_results.tsv"):
+        folder = result_path.parent.name
+        sanitized_id, _, _ = folder.partition("_mz_")
+        if not sanitized_id:
+            continue
+        spec_id = by_sanitized_id.get(sanitized_id)
+        if spec_id is None:
+            continue
+        df = _read_table(result_path)
+        if not _has_columns(df, ("rank", "formula", "estimated_fdr")):
+            continue
+        df = df.copy()
+        df["rank"] = pd.to_numeric(df["rank"], errors="coerce")
+        df["estimated_fdr"] = pd.to_numeric(df["estimated_fdr"], errors="coerce")
+        df = df.sort_values("rank", na_position="last")
+        candidates: list[tuple[str, float]] = []
+        for _, row in df.iterrows():
+            formula = _clean_formula(row.get("formula", pd.NA))
+            if pd.isna(formula):
+                continue
+            candidates.append((str(formula), row.get("estimated_fdr", pd.NA)))
+            if len(candidates) == 5:
+                break
+        if candidates:
+            details[spec_id] = candidates
+    return details
+
+
+def _sanitize_msbuddy_identifier(spec_id: str) -> str:
+    return spec_id.replace("/", "").replace(":", "").replace(" ", "").strip()
+
+
+def _find_sirius_formula_file(path: Path) -> Path:
+    candidates = [
+        file_path
+        for file_path in path.glob("**/*")
+        if file_path.is_file()
+        and file_path.suffix.lower() in {".tsv", ".csv", ".xlsx", ".xls"}
+        and file_path.stem.startswith("formula_identifications")
+    ]
+    if not candidates:
+        raise FileNotFoundError(
+            "SIRIUS output directory must contain a formula_identifications "
+            f"summary file: {path}"
+        )
+
+    def priority(file_path: Path) -> tuple[int, str]:
+        stem = file_path.stem
+        for idx, prefix in enumerate(_SIRIUS_FORMULA_FILES):
+            if stem.startswith(prefix):
+                return idx, file_path.name
+        return len(_SIRIUS_FORMULA_FILES), file_path.name
+
+    return sorted(candidates, key=priority)[0]
+
+
+def _normalize_sirius(df: pd.DataFrame) -> pd.DataFrame:
+    normalized = df.loc[:, SIRIUS_COLUMNS].copy()
+    normalized["ID"] = normalized["ID"].map(_clean_identifier)
+    for column in SIRIUS_FORMULA_COLUMNS:
+        normalized[column] = normalized[column].map(_clean_formula)
+    for column in SIRIUS_ADDUCT_COLUMNS:
+        normalized[column] = normalized[column].map(_clean_text)
+    for column in SIRIUS_SCORE_COLUMNS:
+        normalized[column] = pd.to_numeric(normalized[column], errors="coerce")
+    return normalized.dropna(subset=["ID"]).reset_index(drop=True)
+
+
+def _normalize_sirius_native(df: pd.DataFrame) -> pd.DataFrame:
+    id_column = _first_existing(df, _SIRIUS_ID_COLUMNS)
+    formula_column = _first_existing(df, _SIRIUS_FORMULA_COLUMNS)
+    adduct_column = _first_existing(df, _SIRIUS_ADDUCT_COLUMNS)
+    score_column = _first_existing(df, _SIRIUS_SCORE_COLUMNS)
+    rank_column = _first_existing(df, _SIRIUS_RANK_COLUMNS)
+    if not all((id_column, formula_column, adduct_column, score_column)):
+        raise ValueError(
+            "Unsupported SIRIUS result format. Expected a native/original "
+            "formula_identifications summary or output directory, or an "
+            "msfiddle-normalized CSV. Native summaries must include "
+            "identifier, formula, adduct, and score columns."
+        )
+
+    native = df.copy()
+    native["_msfiddle_id"] = native[id_column].map(_clean_identifier)
+    native["_msfiddle_formula"] = native[formula_column].map(_clean_formula)
+    native["_msfiddle_adduct"] = native[adduct_column].map(_clean_text)
+    native["_msfiddle_score"] = pd.to_numeric(native[score_column], errors="coerce")
+    if rank_column is not None:
+        native["_msfiddle_rank"] = pd.to_numeric(native[rank_column], errors="coerce")
+    else:
+        native["_msfiddle_rank"] = pd.NA
+    native = native.dropna(subset=["_msfiddle_id", "_msfiddle_formula"])
+
+    rows: list[dict[str, object]] = []
+    for spec_id, group in native.groupby("_msfiddle_id", sort=False):
+        group = group.sort_values(
+            by=["_msfiddle_rank", "_msfiddle_score"],
+            ascending=[True, False],
+            na_position="last",
+        )
+        out: dict[str, object] = {"ID": spec_id}
+        for idx, (_, row) in enumerate(group.head(5).iterrows(), start=1):
+            out[f"Pred Formula ({idx})"] = row["_msfiddle_formula"]
+            out[f"Pred Adduct ({idx})"] = row["_msfiddle_adduct"]
+            out[f"SIRIUS Score ({idx})"] = row["_msfiddle_score"]
+        rows.append(out)
+    return _normalize_sirius(pd.DataFrame(rows, columns=SIRIUS_COLUMNS))
+
+
+def _first_existing(df: pd.DataFrame, candidates: Iterable[str]) -> str | None:
+    df_columns = list(df.columns)
+    for candidate in candidates:
+        if candidate in df_columns:
+            return candidate
+    canonical_columns = {_canonical_column(column): column for column in df_columns}
+    for candidate in candidates:
+        match = canonical_columns.get(_canonical_column(candidate))
+        if match is not None:
+            return match
+    return None
+
+
+def _canonical_column(column: object) -> str:
+    text = str(column).strip().lstrip("\ufeff")
+    if text.lower().startswith("sirius_"):
+        text = text[7:]
+    return re.sub(r"[^a-z0-9]+", "", text.lower())
+
+
+def _clean_identifier(value: object) -> object:
+    text = _clean_text(value)
+    if pd.isna(text):
+        return pd.NA
+    return str(text)
+
+
+def _clean_formula(value: object) -> object:
+    text = _clean_text(value)
+    if pd.isna(text):
+        return pd.NA
+    # SIRIUS summaries may represent adduct-resolved formulas as
+    # "[C20H17NO6 + H]+". Keep only the neutral molecular formula.
+    match = re.match(r"^\[?([A-Z][A-Za-z0-9]*)", str(text))
+    return match.group(1) if match else text
+
+
+def _clean_text(value: object) -> object:
+    if pd.isna(value):
+        return pd.NA
+    text = str(value).strip()
+    if text.lower() in {"", "na", "nan", "none", "null"}:
+        return pd.NA
+    return text
+
+
+def _to_number(value: object) -> object:
+    number = pd.to_numeric(pd.Series([value]), errors="coerce").iloc[0]
+    return number if pd.notna(number) else pd.NA

{msfiddle-2.0.1 → msfiddle-2.1.0}/msfiddle/main.py

@@ -173,10 +173,25 @@ def main():
 
     # Add optional arguments
     parser.add_argument(
-        "--buddy_path", type=str, default="", help="Path to saved BUDDY's results"
+        "--buddy_path",
+        type=str,
+        default="",
+        help=(
+            "Path to BUDDY results: native/original msbuddy output directory "
+            "or summary TSV. The msfiddle-normalized CSV is also accepted but "
+            "deprecated and will be removed in 3.0.0."
+        ),
     )
     parser.add_argument(
-        "--sirius_path", type=str, default="", help="Path to saved SIRIUS's results"
+        "--sirius_path",
+        type=str,
+        default="",
+        help=(
+            "Path to SIRIUS results: native/original formula_identifications "
+            "summary or SIRIUS summary output directory. The "
+            "msfiddle-normalized CSV is also accepted but deprecated and will "
+            "be removed in 3.0.0."
+        ),
     )
     parser.add_argument(
         "--seed", type=int, default=42, help="Seed for random functions"

{msfiddle-2.0.1 → msfiddle-2.1.0}/msfiddle.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: msfiddle
-Version: 2.0.1
+Version: 2.1.0
 Summary: A package for predicting chemical formulas from tandem mass spectra
 Home-page: https://github.com/JosieHong/msfiddle
 Author: Yuhui Hong
@@ -64,17 +64,13 @@ For the full experimental codebase, see https://github.com/JosieHong/FIDDLE.
 ## Installation
 
 ```bash
-pip install msfiddle
+pip install msfiddle              # base install, no PyTorch
+pip install "msfiddle[inference]" # base install + PyTorch (needed for predictions and the CLI)
 ```
 
-PyTorch is required for inference. Install the optional inference extra, 
-or install PyTorch separately for your platform:
-```bash
-pip install "msfiddle[inference]"
-```
-
-See the official PyTorch installation guide for custom CUDA builds:
-https://pytorch.org/get-started/locally/.
+For a custom PyTorch build (CUDA, ROCm, Apple Silicon, etc.), install torch
+first via the [official guide](https://pytorch.org/get-started/locally/), then
+`pip install msfiddle`.
 
 ## Usage
 
@@ -90,7 +86,7 @@ msfiddle-download-models --destination /path/to/models \
                           --models fiddle_tcn_qtof fiddle_rescore_qtof
 ```
 
-`msfiddle` 2.0.1 reuses the FIDDLE `v2.0.0` checkpoint assets.
+All `msfiddle` 2.x releases reuse the FIDDLE `v2.0.0` checkpoint assets.
 
 Run the packaged demo:
 ```bash
@@ -190,6 +186,44 @@ The Python `predict_from_spectrum()` API returns a list of candidate dictionarie
 `predict_batch()` returns one record per input spectrum with `id`, `candidates`,
 and `metadata`.
 
+### Native/original BUDDY and SIRIUS outputs
+
+Use the same MGF input file to generate native
+[BUDDY/msbuddy](https://msbuddy.readthedocs.io/en/latest/cmdapi.html) and
+[SIRIUS](https://v6.docs.sirius-ms.io/cli/) outputs. `msfiddle` accepts these
+native/original files directly through `--buddy_path` and `--sirius_path`. The
+older msfiddle-normalized CSV formats documented in `docs/formats.rst` are
+**deprecated** and will be removed in `msfiddle` 3.0.0; loading them emits a
+`DeprecationWarning`.
+
+```bash
+msbuddy -mgf /path/to/data.mgf \
+        -output /path/to/buddy_output \
+        -ms orbitrap \
+        -p -n_cpu 12 \
+        -d -hal
+```
+
+`msbuddy` writes `msbuddy_result_summary.tsv`; `-d` also writes detailed
+per-query `formula_results.tsv` files. Pass either the summary file or the
+output directory to `--buddy_path`. The full output directory is preferred
+because it includes per-candidate FDR scores for ranks beyond rank 1.
+
+```bash
+sirius --input /path/to/data.mgf \
+       --project /path/to/sirius_project \
+       formulas --profile orbitrap
+
+sirius --project /path/to/sirius_project \
+       summaries --top-k-summary=5 \
+       --output /path/to/sirius_output
+```
+
+SIRIUS writes a project space and exports summary files such as
+`formula_identifications.tsv`. Pass either a formula-identification summary file
+or the summary output directory to `--sirius_path`.
+SIRIUS 6 may require `sirius login` before formula computation.
+
 ### MGF input
 
 The required MGF fields are `TITLE`, `PRECURSOR_MZ`, `PRECURSOR_TYPE`, and

{msfiddle-2.0.1 → msfiddle-2.1.0}/msfiddle.egg-info/SOURCES.txt

@@ -5,6 +5,7 @@ msfiddle/__init__.py
 msfiddle/api.py
 msfiddle/dataset.py
 msfiddle/download.py
+msfiddle/external_results.py
 msfiddle/main.py
 msfiddle/model_tcn.py
 msfiddle.egg-info/PKG-INFO
@@ -31,6 +32,7 @@ msfiddle/utils/pkl_utils.py
 msfiddle/utils/refine_utils.py
 tests/test_api.py
 tests/test_download.py
+tests/test_external_results.py
 tests/test_main.py
 tests/test_mol_utils.py
 tests/test_msms_utils.py

{msfiddle-2.0.1 → msfiddle-2.1.0}/setup.py

@@ -5,7 +5,7 @@ with open("README.md", "r", encoding="utf-8") as fh:
 
 setup(
     name="msfiddle",
-    version="2.0.1",
+    version="2.1.0",
     author="Yuhui Hong",
     author_email="josieexception@outlook.com",
     description="A package for predicting chemical formulas from tandem mass spectra",

{msfiddle-2.0.1 → msfiddle-2.1.0}/tests/test_api.py

@@ -1,7 +1,9 @@
 import pytest
+import numpy as np
+import pandas as pd
 
 import msfiddle.api as api
-from msfiddle.api import MsFiddlePredictor, _ModelPrediction
+from msfiddle.api import MsFiddlePredictor, _EncodedSpectrum, _ModelPrediction
 
 VALID_MZ = [60.0, 85.0, 100.0, 125.0, 150.0]
 VALID_INTENSITY = [10.0, 50.0, 20.0, 35.0, 15.0]
@@ -160,3 +162,101 @@ def test_predict_batch_preserves_order_ids_and_candidate_shape(monkeypatch, tmp_
     assert result[0]["candidates"][0]["mass"] == pytest.approx(122.073)
     assert result[0]["candidates"][0]["metadata"]["raw_rescore"] == 1.2
     assert result[0]["metadata"]["adduct"] == VALID_ADDUCT
+
+
+def test_predict_mgf_loads_native_external_outputs(monkeypatch, tmp_path):
+    predictor = MsFiddlePredictor(
+        resume_path=tmp_path / "missing_tcn.pt",
+        rescore_resume_path=tmp_path / "missing_rescore.pt",
+    )
+    spectrum = _EncodedSpectrum(
+        id="spec-1",
+        precursor_type="[M-H]-",
+        precursor_mz=100.0,
+        collision_energy="50",
+        spec=np.zeros(10, dtype=np.float32),
+        env=np.zeros(3, dtype=np.float32),
+        neutral_add=np.zeros(13, dtype=np.float32),
+    )
+
+    buddy_dir = tmp_path / "buddy_output"
+    buddy_dir.mkdir()
+    pd.DataFrame(
+        [
+            {
+                "identifier": "spec-1",
+                "mz": 100.0,
+                "rt": "NA",
+                "adduct": "[M-H]-",
+                "formula_rank_1": "C4H6N4O3",
+                "estimated_fdr": 0.2,
+                "formula_rank_2": "C5H7NO2",
+                "formula_rank_3": "NA",
+                "formula_rank_4": "NA",
+                "formula_rank_5": "NA",
+            }
+        ]
+    ).to_csv(buddy_dir / "msbuddy_result_summary.tsv", sep="\t", index=False)
+    detail_dir = buddy_dir / "spec-1_mz_100.0_rt_NA"
+    detail_dir.mkdir()
+    pd.DataFrame(
+        [
+            {"rank": 1, "formula": "C4H6N4O3", "estimated_fdr": 0.2},
+            {"rank": 2, "formula": "C5H7NO2", "estimated_fdr": 0.4},
+        ]
+    ).to_csv(detail_dir / "formula_results.tsv", sep="\t", index=False)
+
+    sirius_path = tmp_path / "formula_identifications_top-5.tsv"
+    pd.DataFrame(
+        [
+            {
+                "name": "spec-1",
+                "rank": 1,
+                "molecularFormula": "C6H12O6",
+                "adduct": "[M - H]-",
+                "SiriusScore": 1.0,
+            }
+        ]
+    ).to_csv(sirius_path, sep="\t", index=False)
+
+    seen_extra_formulas = []
+
+    def fake_load_mgf_spectra(self, test_data_path):
+        return [spectrum]
+
+    def fake_predict_encoded(self, encoded, *, batch_size):
+        return [
+            _ModelPrediction(
+                spectrum=encoded[0],
+                vector_string="1;2;3",
+                pred_formula="C3H7NO2",
+                pred_mass=89.0,
+                pred_atom_count=12.0,
+                pred_hc_count=1.5,
+                prediction_time=0.01,
+            )
+        ]
+
+    def fake_refine_and_rescore(self, prediction, top_k, *, extra_formulas=None):
+        seen_extra_formulas.extend(extra_formulas or [])
+        return (
+            {"formula": ["C4H6N4O3"], "mass": [146.0], "rescore": [0.9]},
+            146.0,
+        )
+
+    monkeypatch.setattr(MsFiddlePredictor, "_load_mgf_spectra", fake_load_mgf_spectra)
+    monkeypatch.setattr(MsFiddlePredictor, "_predict_encoded", fake_predict_encoded)
+    monkeypatch.setattr(
+        MsFiddlePredictor, "_refine_and_rescore", fake_refine_and_rescore
+    )
+
+    result = predictor.predict_mgf(
+        tmp_path / "unused.mgf",
+        buddy_path=buddy_dir,
+        sirius_path=sirius_path,
+        top_k=1,
+    )
+
+    assert seen_extra_formulas == ["C4H6N4O3", "C5H7NO2", "C6H12O6"]
+    assert result.loc[0, "ID"] == "spec-1"
+    assert result.loc[0, "Refined Formula (0)"] == "C4H6N4O3"

msfiddle-2.1.0/tests/test_external_results.py

@@ -0,0 +1,282 @@
+import pandas as pd
+import pytest
+
+from msfiddle.external_results import load_buddy_results, load_sirius_results
+
+
+def test_load_buddy_results_preserves_normalized_csv(tmp_path):
+    path = tmp_path / "buddy.csv"
+    pd.DataFrame(
+        [
+            {
+                "ID": "spec-1",
+                "Adduct": "[M-H]-",
+                "Pred Formula (1)": "C4H6N4O3",
+                "Pred Formula (2)": "NA",
+                "Pred Formula (3)": "NA",
+                "Pred Formula (4)": "NA",
+                "Pred Formula (5)": "NA",
+                "BUDDY Score (1)": "0.1",
+                "BUDDY Score (2)": "NA",
+                "BUDDY Score (3)": "NA",
+                "BUDDY Score (4)": "NA",
+                "BUDDY Score (5)": "NA",
+            }
+        ]
+    ).to_csv(path, index=False)
+
+    with pytest.warns(DeprecationWarning, match="msfiddle-normalized BUDDY"):
+        result = load_buddy_results(path)
+
+    assert result.loc[0, "ID"] == "spec-1"
+    assert result.loc[0, "Pred Formula (1)"] == "C4H6N4O3"
+    assert result.loc[0, "BUDDY Score (1)"] == pytest.approx(0.1)
+    assert pd.isna(result.loc[0, "Pred Formula (2)"])
+
+
+def test_load_buddy_results_converts_msbuddy_summary(tmp_path):
+    path = tmp_path / "msbuddy_result_summary.tsv"
+    pd.DataFrame(
+        [
+            {
+                "identifier": "spec-1",
+                "mz": 100.0,
+                "rt": "NA",
+                "adduct": "[M-H]-",
+                "formula_rank_1": "C4H6N4O3",
+                "estimated_fdr": 0.2,
+                "formula_rank_2": "C5H7NO2",
+                "formula_rank_3": "NA",
+                "formula_rank_4": "NA",
+                "formula_rank_5": "NA",
+            }
+        ]
+    ).to_csv(path, sep="\t", index=False)
+
+    result = load_buddy_results(path)
+
+    assert result.loc[0, "ID"] == "spec-1"
+    assert result.loc[0, "Adduct"] == "[M-H]-"
+    assert result.loc[0, "Pred Formula (1)"] == "C4H6N4O3"
+    assert result.loc[0, "BUDDY Score (1)"] == pytest.approx(0.2)
+    assert result.loc[0, "Pred Formula (2)"] == "C5H7NO2"
+    assert pd.isna(result.loc[0, "BUDDY Score (2)"])
+
+
+def test_load_buddy_results_uses_msbuddy_detail_scores_from_directory(tmp_path):
+    summary = tmp_path / "msbuddy_result_summary.tsv"
+    pd.DataFrame(
+        [
+            {
+                "identifier": "spec-1",
+                "mz": 100.0,
+                "rt": "NA",
+                "adduct": "[M-H]-",
+                "formula_rank_1": "C4H6N4O3",
+                "estimated_fdr": 0.2,
+                "formula_rank_2": "C5H7NO2",
+                "formula_rank_3": "NA",
+                "formula_rank_4": "NA",
+                "formula_rank_5": "NA",
+            }
+        ]
+    ).to_csv(summary, sep="\t", index=False)
+    detail_dir = tmp_path / "spec-1_mz_100.0_rt_NA"
+    detail_dir.mkdir()
+    pd.DataFrame(
+        [
+            {"rank": 2, "formula": "C5H7NO2", "estimated_fdr": 0.4},
+            {"rank": 1, "formula": "C4H6N4O3", "estimated_fdr": 0.2},
+        ]
+    ).to_csv(detail_dir / "formula_results.tsv", sep="\t", index=False)
+
+    result = load_buddy_results(tmp_path)
+
+    assert result.loc[0, "Pred Formula (1)"] == "C4H6N4O3"
+    assert result.loc[0, "BUDDY Score (1)"] == pytest.approx(0.2)
+    assert result.loc[0, "Pred Formula (2)"] == "C5H7NO2"
+    assert result.loc[0, "BUDDY Score (2)"] == pytest.approx(0.4)
+
+
+def test_load_buddy_results_matches_detail_dir_when_summary_mz_format_differs(tmp_path):
+    # Real msbuddy rounds mz in the summary, but a future version (or a
+    # hand-edited summary) could disagree with the directory name. The
+    # directory's identifier prefix alone should be enough to match.
+    summary = tmp_path / "msbuddy_result_summary.tsv"
+    pd.DataFrame(
+        [
+            {
+                "identifier": "spec-1",
+                "mz": 111.020160000,  # padded zeros vs. dir name's "111.0202"
+                "rt": "NA",
+                "adduct": "[M-H]-",
+                "formula_rank_1": "C4H4N2O2",
+                "estimated_fdr": 0.0,
+                "formula_rank_2": "NA",
+                "formula_rank_3": "NA",
+                "formula_rank_4": "NA",
+                "formula_rank_5": "NA",
+            }
+        ]
+    ).to_csv(summary, sep="\t", index=False)
+    detail_dir = tmp_path / "spec-1_mz_111.0202_rt_NA"
+    detail_dir.mkdir()
+    pd.DataFrame(
+        [
+            {"rank": 1, "formula": "C4H4N2O2", "estimated_fdr": 0.0},
+            {"rank": 2, "formula": "C5H8O2", "estimated_fdr": 0.5},
+        ]
+    ).to_csv(detail_dir / "formula_results.tsv", sep="\t", index=False)
+
+    result = load_buddy_results(tmp_path)
+
+    assert result.loc[0, "Pred Formula (1)"] == "C4H4N2O2"
+    assert result.loc[0, "Pred Formula (2)"] == "C5H8O2"
+    assert result.loc[0, "BUDDY Score (2)"] == pytest.approx(0.5)
+
+
+def test_load_buddy_results_drops_rows_with_missing_identifier(tmp_path):
+    path = tmp_path / "msbuddy_result_summary.tsv"
+    pd.DataFrame(
+        [
+            {
+                "identifier": "NA",
+                "mz": 100.0,
+                "rt": "NA",
+                "adduct": "[M-H]-",
+                "formula_rank_1": "C4H6N4O3",
+                "estimated_fdr": 0.2,
+                "formula_rank_2": "NA",
+                "formula_rank_3": "NA",
+                "formula_rank_4": "NA",
+                "formula_rank_5": "NA",
+            },
+            {
+                "identifier": "spec-1",
+                "mz": 100.0,
+                "rt": "NA",
+                "adduct": "[M-H]-",
+                "formula_rank_1": "C5H7NO2",
+                "estimated_fdr": 0.3,
+                "formula_rank_2": "NA",
+                "formula_rank_3": "NA",
+                "formula_rank_4": "NA",
+                "formula_rank_5": "NA",
+            },
+        ]
+    ).to_csv(path, sep="\t", index=False)
+
+    result = load_buddy_results(path)
+
+    assert len(result) == 1
+    assert result.loc[0, "ID"] == "spec-1"
+
+
+def test_load_sirius_results_preserves_normalized_csv(tmp_path):
+    path = tmp_path / "sirius.csv"
+    pd.DataFrame(
+        [
+            {
+                "ID": "spec-1",
+                "Pred Formula (1)": "C3H7NO2",
+                "Pred Adduct (1)": "[M - H]-",
+                "SIRIUS Score (1)": "12.5",
+                "Pred Formula (2)": "NA",
+                "Pred Adduct (2)": "NA",
+                "SIRIUS Score (2)": "NA",
+                "Pred Formula (3)": "NA",
+                "Pred Adduct (3)": "NA",
+                "SIRIUS Score (3)": "NA",
+                "Pred Formula (4)": "NA",
+                "Pred Adduct (4)": "NA",
+                "SIRIUS Score (4)": "NA",
+                "Pred Formula (5)": "NA",
+                "Pred Adduct (5)": "NA",
+                "SIRIUS Score (5)": "NA",
+            }
+        ]
+    ).to_csv(path, index=False)
+
+    with pytest.warns(DeprecationWarning, match="msfiddle-normalized SIRIUS"):
+        result = load_sirius_results(path)
+
+    assert result.loc[0, "ID"] == "spec-1"
+    assert result.loc[0, "Pred Formula (1)"] == "C3H7NO2"
+    assert result.loc[0, "SIRIUS Score (1)"] == pytest.approx(12.5)
+    assert pd.isna(result.loc[0, "Pred Formula (2)"])
+
+
+def test_load_sirius_results_converts_native_top_k_summary(tmp_path):
+    path = tmp_path / "formula_identifications_top-5.tsv"
+    pd.DataFrame(
+        [
+            {
+                "name": "spec-1",
+                "rank": 2,
+                "molecularFormula": "C5H7NO2",
+                "adduct": "[M - H]-",
+                "SiriusScore": 5.0,
+            },
+            {
+                "name": "spec-1",
+                "rank": 1,
+                "molecularFormula": "[C4H6N4O3 - H]-",
+                "adduct": "[M - H]-",
+                "SiriusScore": 7.0,
+            },
+        ]
+    ).to_csv(path, sep="\t", index=False)
+
+    result = load_sirius_results(path)
+
+    assert result.loc[0, "ID"] == "spec-1"
+    assert result.loc[0, "Pred Formula (1)"] == "C4H6N4O3"
+    assert result.loc[0, "Pred Adduct (1)"] == "[M - H]-"
+    assert result.loc[0, "SIRIUS Score (1)"] == pytest.approx(7.0)
+    assert result.loc[0, "Pred Formula (2)"] == "C5H7NO2"
+
+
+def test_load_sirius_results_finds_formula_summary_in_directory(tmp_path):
+    pd.DataFrame(
+        [
+            {
+                "mappingFeatureId": "spec-1",
+                "molecularFormula": "C4H6N4O3",
+                "adduct": "[M - H]-",
+                "rankingScore": 0.8,
+            }
+        ]
+    ).to_csv(tmp_path / "formula_identifications.tsv", sep="\t", index=False)
+
+    result = load_sirius_results(tmp_path)
+
+    assert result.loc[0, "ID"] == "spec-1"
+    assert result.loc[0, "Pred Formula (1)"] == "C4H6N4O3"
+    assert result.loc[0, "SIRIUS Score (1)"] == pytest.approx(0.8)
+
+
+def test_load_sirius_results_accepts_prefixed_column_variants(tmp_path):
+    path = tmp_path / "formula_identifications_top-5.tsv"
+    pd.DataFrame(
+        [
+            {
+                "SIRIUS_mappingFeatureId": "spec-1",
+                "SIRIUS_formula": "C5H7NO2",
+                "SIRIUS_ionType": "[M - H]-",
+                "SIRIUS_rankingScore": 0.4,
+            },
+            {
+                "SIRIUS_mappingFeatureId": "spec-1",
+                "SIRIUS_formula": "C4H6N4O3",
+                "SIRIUS_ionType": "[M - H]-",
+                "SIRIUS_rankingScore": 0.9,
+            },
+        ]
+    ).to_csv(path, sep="\t", index=False)
+
+    result = load_sirius_results(path)
+
+    assert result.loc[0, "ID"] == "spec-1"
+    assert result.loc[0, "Pred Formula (1)"] == "C4H6N4O3"
+    assert result.loc[0, "SIRIUS Score (1)"] == pytest.approx(0.9)
+    assert result.loc[0, "Pred Formula (2)"] == "C5H7NO2"