Oncodrive3D 1.0.5__tar.gz → 1.0.7__tar.gz

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/.gitignore

@@ -192,6 +192,4 @@ run_*
 
 # VSC
 launch.json
-.vscode
-
-test/outputs*
+.vscode

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/Dockerfile

@@ -19,7 +19,7 @@ RUN mkdir -p /root/.cache/uv \
     && uv build
 
 # Second stage: Runtime image
-FROM python:3.10-buster AS runtime-stage
+FROM python:3.10.18-bookworm AS runtime-stage
 
 # Set environment variables
 ENV BGDATA_LOCAL="/bgdatacache" \
@@ -30,7 +30,7 @@ COPY --from=build-stage /oncodrive3d/dist /oncodrive3d/dist
 WORKDIR /oncodrive3d
 
 # Install oncodrive3d
-RUN pip install --no-cache-dir dist/*.tar.gz
+RUN pip install dist/*.tar.gz
 
 # Create required directories
 RUN install -d -m 0755 "$BGDATA_LOCAL" "$BBGLAB_HOME"

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: Oncodrive3D
-Version: 1.0.5
+Version: 1.0.7
 Summary: Oncodrive3D is a method designed to analyse patterns of somatic mutations across tumors to identify three-dimensional (3D) clusters of missense mutations and detect genes that are under positive selection.
 Project-URL: Homepage, https://github.com/bbglab/clustering_3d
 Project-URL: Repository, https://github.com/bbglab/clustering_3d
@@ -37,14 +37,15 @@ Description-Content-Type: text/markdown
 
 **Oncodrive3D** is a fast and accurate computational method designed to analyze patterns of somatic mutation across tumors, with the goal of identifying **three-dimensional (3D) clusters** of missense mutations and detecting genes under **positive selection**. 
 
-The method leverages **AlphaFold 2-predicted protein structures** and Predicted Aligned Error (PAE) to define residue contacts within the protein's 3D space. When available, it integrates **mutational profiles** to build an accurate background model of neutral mutagenesis. By applying a novel **rank-based statistical approach**, Oncodrive3D scores potential 3D clusters and computes empirical p-values."
+The method leverages **AlphaFold 2-predicted protein structures** and Predicted Aligned Error (PAE) to define residue contacts within the protein's 3D space. When available, it integrates **mutational profiles** to build an accurate background model of neutral mutagenesis. By applying a novel **rank-based statistical approach**, Oncodrive3D scores potential 3D clusters and computes empirical p-values.
 
 [![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
 [![docker](https://img.shields.io/docker/v/bbglab/oncodrive3d?logo=docker)](https://hub.docker.com/r/bbglab/oncodrive3d)
 [![PyPI - Version](https://img.shields.io/pypi/v/oncodrive3d?logo=pypi)](https://pypi.org/project/Oncodrive3D/)
 
----
+![Graphical abstract of Oncodrive3D](docs/images/graphical_abstract.png "Oncodrive3D")
 
+---
 
 ## Requirements
 
@@ -118,6 +119,13 @@ Options:
                                   Default: human
   -m, --mane                      Use structures predicted from MANE Select transcripts 
                                   (applicable to Homo sapiens only).
+  -M, --mane_only                 Use only structures predicted from MANE Select transcripts
+                                  (applicable to Homo sapiens only).
+  -C, --custom_mane_pdb_dir       Path to directory containing custom MANE PDB structures.
+                                  Default: None
+  -f, --custom_mane_metadata_path Path to a dataframe (typically a samplesheet.csv) including 
+                                  Ensembl IDs and sequences of the custom pdbs.
+                                  Default: None
   -d, --distance_threshold INT    Distance threshold (Å) for defining residues contacts. 
                                   Default: 10
   -c, --cores INT                 Number of CPU cores for computation. 
@@ -128,6 +136,25 @@ Options:
 
 For more information on the output of this step, please refer to the [Building Datasets Output Documentation](https://github.com/bbglab/oncodrive3d/tree/master/docs/build_output.md).
 
+> [!TIP]
+> ### Increasing MANE Structural Coverage
+> To maximize structural coverage of **MANE Select transcripts**, you can predict missing structures locally and integrate them into Oncodrive3D using:
+>
+> - `tools/preprocessing/prepare_samplesheet.py`: a standalone utility that:
+>   - Retrieve the full MANE entries from NCBI.
+>   - Identifies proteins missing from the AlphaFold MANE dataset.
+>   - Generates:
+>     - A `samplesheet.csv` with Ensembl protein IDs, FASTA paths, and optional sequences.
+>     - Individual FASTA files for each missing protein.
+>
+> - `--custom_mane_pdb_dir`: use this to provide your own predicted PDB structures (e.g., from [nf-core/proteinfold](https://nf-co.re/proteinfold/1.1.1/)).
+>
+> - `--custom_mane_metadata_path`: path to the corresponding `samplesheet.csv`, which must include:
+>   - `sequence`: Ensembl protein ID (required)
+>   - `refseq`: amino acid sequence (used to inject sequence into PDB if missing)
+>
+
+
 
 ## Running 3D clustering Analysis
 
@@ -305,4 +332,4 @@ Oncodrive3D is available to the general public subject to certain conditions des
 
 
 ## Citation 
-If you use Oncodrive3D in your research, please cite the original paper: [Oncodrive3D: Fast and accurate detection of structural clusters of somatic mutations under positive selection](https://www.biorxiv.org/content/10.1101/2025.01.11.632354v2).
+If you use Oncodrive3D in your research, please cite the original paper: [Oncodrive3D: fast and accurate detection of structural clusters of somatic mutations under positive selection](https://academic.oup.com/nar/article/53/15/gkaf776/8234003).

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/README.md

@@ -2,14 +2,15 @@
 
 **Oncodrive3D** is a fast and accurate computational method designed to analyze patterns of somatic mutation across tumors, with the goal of identifying **three-dimensional (3D) clusters** of missense mutations and detecting genes under **positive selection**. 
 
-The method leverages **AlphaFold 2-predicted protein structures** and Predicted Aligned Error (PAE) to define residue contacts within the protein's 3D space. When available, it integrates **mutational profiles** to build an accurate background model of neutral mutagenesis. By applying a novel **rank-based statistical approach**, Oncodrive3D scores potential 3D clusters and computes empirical p-values."
+The method leverages **AlphaFold 2-predicted protein structures** and Predicted Aligned Error (PAE) to define residue contacts within the protein's 3D space. When available, it integrates **mutational profiles** to build an accurate background model of neutral mutagenesis. By applying a novel **rank-based statistical approach**, Oncodrive3D scores potential 3D clusters and computes empirical p-values.
 
 [![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
 [![docker](https://img.shields.io/docker/v/bbglab/oncodrive3d?logo=docker)](https://hub.docker.com/r/bbglab/oncodrive3d)
 [![PyPI - Version](https://img.shields.io/pypi/v/oncodrive3d?logo=pypi)](https://pypi.org/project/Oncodrive3D/)
 
----
+![Graphical abstract of Oncodrive3D](docs/images/graphical_abstract.png "Oncodrive3D")
 
+---
 
 ## Requirements
 
@@ -83,6 +84,13 @@ Options:
                                   Default: human
   -m, --mane                      Use structures predicted from MANE Select transcripts 
                                   (applicable to Homo sapiens only).
+  -M, --mane_only                 Use only structures predicted from MANE Select transcripts
+                                  (applicable to Homo sapiens only).
+  -C, --custom_mane_pdb_dir       Path to directory containing custom MANE PDB structures.
+                                  Default: None
+  -f, --custom_mane_metadata_path Path to a dataframe (typically a samplesheet.csv) including 
+                                  Ensembl IDs and sequences of the custom pdbs.
+                                  Default: None
   -d, --distance_threshold INT    Distance threshold (Å) for defining residues contacts. 
                                   Default: 10
   -c, --cores INT                 Number of CPU cores for computation. 
@@ -93,6 +101,25 @@ Options:
 
 For more information on the output of this step, please refer to the [Building Datasets Output Documentation](https://github.com/bbglab/oncodrive3d/tree/master/docs/build_output.md).
 
+> [!TIP]
+> ### Increasing MANE Structural Coverage
+> To maximize structural coverage of **MANE Select transcripts**, you can predict missing structures locally and integrate them into Oncodrive3D using:
+>
+> - `tools/preprocessing/prepare_samplesheet.py`: a standalone utility that:
+>   - Retrieve the full MANE entries from NCBI.
+>   - Identifies proteins missing from the AlphaFold MANE dataset.
+>   - Generates:
+>     - A `samplesheet.csv` with Ensembl protein IDs, FASTA paths, and optional sequences.
+>     - Individual FASTA files for each missing protein.
+>
+> - `--custom_mane_pdb_dir`: use this to provide your own predicted PDB structures (e.g., from [nf-core/proteinfold](https://nf-co.re/proteinfold/1.1.1/)).
+>
+> - `--custom_mane_metadata_path`: path to the corresponding `samplesheet.csv`, which must include:
+>   - `sequence`: Ensembl protein ID (required)
+>   - `refseq`: amino acid sequence (used to inject sequence into PDB if missing)
+>
+
+
 
 ## Running 3D clustering Analysis
 
@@ -270,4 +297,4 @@ Oncodrive3D is available to the general public subject to certain conditions des
 
 
 ## Citation 
-If you use Oncodrive3D in your research, please cite the original paper: [Oncodrive3D: Fast and accurate detection of structural clusters of somatic mutations under positive selection](https://www.biorxiv.org/content/10.1101/2025.01.11.632354v2).
+If you use Oncodrive3D in your research, please cite the original paper: [Oncodrive3D: fast and accurate detection of structural clusters of somatic mutations under positive selection](https://academic.oup.com/nar/article/53/15/gkaf776/8234003).

oncodrive3d-1.0.7/docs/build_output.md

@@ -0,0 +1,19 @@
+# Output
+
+After successfully completing this step, the build folder must include the following files and directories:
+
+- **pae/**: Directory including the AlphaFold predicted aligned error (PAE) for any protein of the proteome with a length lower than 2700 amino acids
+
+- **prob_cmaps/**: Directory including the contact probability map (pCMAPs) for any protein of the proteome
+
+- **confidence.csv**: CSV file including per-residue predicted local distance difference test (pLDDT) score for any protein of the proteome
+
+- **seq_for_mut_prob.csv**: CSV file including HUGO symbol, Uniprot ID, DNA and protein sequences for any proteine of the proteome
+
+- **pdb_structures/**: Directory containing all PDB structure files—both those downloaded from the AlphaFold database and any custom in-house predicted structures (if provided).
+
+- **log/**: Directory containing log files produced during the build-datasets execution.
+
+- **biomart_metadata.csv**: Metadata file from Ensembl BioMart used to prioritize canonical transcripts when multiple transcripts map to the same protein structure.
+
+...

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/docs/run_input_output.md

@@ -176,7 +176,7 @@ It includes the following fields:
 - **C_pos**: List of protein positions of the gene clusters.
 - **C_label**: List of labels indicating the clump to which each cluster is grouped.
 - **Pos_top_vol**: Position of the most significant cluster.
-- **Score_obs_sim_top_vol**: 3D clustering score of the gene (normalized 3D clustering score of the residue with the lowest p-value; if multiple residues have the same lowest p-value, the maximum normalized 3D clustering score among those residues is used).
+- **Score_obs_sim_top_vol**: 3D clustering score of the gene (rescaled 3D clustering score of the residue with the lowest p-value; if multiple residues have the same lowest p-value, the maximum rescaled 3D clustering score among those residues is used).
 - **Mut_in_gene**: Number of missense mutations in the gene.
 - **Clust_mut**: Number of missense mutations in significant clusters.
 - **Clust_res**: Number of residues detected as significant clusters.
@@ -227,12 +227,12 @@ It includes the following fields:
 - **Mut_in_res**: Number of missense mutations in the residue.
 - **Mut_in_vol**: Number of missense mutations in the volume of the residue.
 - **Score**: 3D clustering score for the residue.
-- **Score_obs_sim**: Normalized 3D clustering score for the residue.
+- **Score_obs_sim**: Rescaled 3D clustering score for the residue.
 - **pval**: The p-value of the residue in the 3D clustering analysis.
 - **C**: Binary label indicating whether the cluster at that residue is significant (1) or not (0). A cluster is marked as significant either because it meets the significance criteria directly or because it has been rescued by contributing mutations to another significant cluster.
 - **C_ext**: Binary label indicating whether the cluster has been rescued by contributing mutations to another significant cluster (1) or if it was significant on its own (0).
 - **Clump**: Identifier for the clump to which the cluster at that residue has been assigned.
-- **Rank**: Rank used to perform the calculation of the normalized 3D clustering score and p-values. 
+- **Rank**: Rank used to perform the calculation of the 3D clustering score and p-values. 
 - **Mut_in_cl_vol**: Number of missense mutations in the clusters of the clump to which the cluster has been assigned.
 - **Res_in_cl**: Positions of the clusters of the clump to which the cluster has been assigned.
 - **PAE_vol**: Weighted average predicted aligned error (PAE) of the residues in the volume of the cluster.

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/oncodrive3d_pipeline/README.md

@@ -29,7 +29,7 @@ Run a test to ensure that everything is set up correctly and functioning as expe
 nextflow run main.nf -profile test,container --data_dir <build_folder>
 ```
 
-Replace `<build_folder>` with the path to the Oncodrive3D datasets built in the [building datasets](#building-datasets) step.
+Replace `<build_folder>` with the path to the Oncodrive3D datasets built in the [building datasets](../README.md#building-datasets) step.
 If you prefer to use Conda, replace `container` in the `-profile` argument with `conda`.
 
 ## Usage

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/scripts/__init__.py

@@ -1,2 +1,2 @@
 __logger_name__ = 'oncodrive3d'
-__version__ = "1.0.5"
+__version__ = "1.0.7"

oncodrive3d-1.0.7/scripts/datasets/build_datasets.py

@@ -0,0 +1,173 @@
+"""
+Module to generate datasets necessary to run Oncodrive3D.
+
+The build is a pipeline that perform the following tasks:
+    - Download the PDB structures of the selected proteome
+      predicted by AlphaFold 2 from AlphaFold DB.
+    - Merge the overlapping structures processed as fragments.
+    - Optionally download the PDB structures corresponding to
+      the MANE Select transcripts.
+    - Optionally copy and parse custom PDB structures.  
+    - Generate a dataframe including Uniprot_ID, HUGO Symbol,
+      protein, DNA sequence, and other gene's information.
+    - Extract AlphaFold model confidence (pLDDT).
+    - Download AlphaFold predicted aligned error (PAE) from
+      AlphaFold DB and convert the files into npy format.
+    - Use the PDB structure and PAE to create maps of
+      probability of contacts (pCMAPs) for any protein of the
+      downloaded proteome with available PAE.
+    - Remove unnecessary temp files.
+"""
+
+
+import os
+import daiquiri
+
+from scripts import __logger_name__
+from scripts.datasets.af_merge import merge_af_fragments
+from scripts.datasets.get_pae import get_pae
+from scripts.datasets.get_structures import get_structures, mv_mane_pdb
+from scripts.datasets.model_confidence import get_confidence
+from scripts.datasets.parse_pae import parse_pae
+from scripts.datasets.prob_contact_maps import get_prob_cmaps_mp
+from scripts.datasets.seq_for_mut_prob import get_seq_df
+from scripts.datasets.custom_pdb import copy_and_parse_custom_pdbs
+from scripts.datasets.utils import get_species
+from scripts.globals import clean_dir, clean_temp_files
+
+logger = daiquiri.getLogger(__logger_name__ + ".build")
+
+
+def build(output_datasets,
+          organism,
+          mane,
+          mane_only,
+          custom_pdb_dir,
+          custom_mane_metadata_path,
+          distance_threshold,
+          num_cores,
+          af_version,
+          mane_version):
+    """
+    Build datasets necessary to run Oncodrive3D.
+    """
+
+    # Empty directory
+    clean_dir(output_datasets, 'd', txt_file=True)
+
+    # Download PDB structures
+    species = get_species(organism)
+    if not mane_only:
+      logger.info("Downloading AlphaFold (AF) predicted structures...")
+      get_structures(
+        path=os.path.join(output_datasets,"pdb_structures"),
+        species=species,
+        af_version=str(af_version),
+        threads=num_cores
+        )
+      logger.info("Download of structures completed!")
+
+      # Merge fragmented structures
+      logger.info("Merging fragmented structures...")
+      merge_af_fragments(input_dir=os.path.join(output_datasets,"pdb_structures"), gzip=True)
+
+    # Download PDB MANE structures
+    if species == "Homo sapiens" and mane:
+        logger.info("Downloading AlphaFold (AF) predicted structures overlap with MANE...")
+        get_structures(
+          path=os.path.join(output_datasets,"pdb_structures_mane"),
+          species=species,
+          mane=True,
+          threads=num_cores
+          )
+        mv_mane_pdb(output_datasets, "pdb_structures", "pdb_structures_mane")
+        logger.info("Download of MANE structures completed!")
+        
+    # Copy custom PDB structures and optinally add SEQRES
+    if custom_pdb_dir is not None:
+      if custom_mane_metadata_path is None:
+        logger.error(
+          "custom_mane_metadata_path must be provided when custom_pdb_dir is specified"
+          )
+        raise ValueError(
+          "Both custom_pdb_dir and custom_mane_metadata_path must be provided together"
+          )
+      
+      logger.info("Copying custom PDB structures...")
+      if os.path.exists(custom_pdb_dir):
+        copy_and_parse_custom_pdbs(
+          src_dir=custom_pdb_dir,
+          dst_dir=os.path.join(output_datasets,"pdb_structures"), 
+          af_version=int(af_version),
+          custom_mane_metadata_path=custom_mane_metadata_path
+          )
+      else:
+          logger.error(f"Custom PDB directory does not exist: {custom_pdb_dir}")
+          raise FileNotFoundError(f"Custom PDB directory not found: {custom_pdb_dir}")
+    
+    # Create df including genes and proteins sequences & Hugo to Uniprot_ID mapping
+    logger.info("Generating dataframe for genes and proteins sequences...")
+    seq_df = get_seq_df(
+      datasets_dir=output_datasets,
+      output_seq_df=os.path.join(output_datasets, "seq_for_mut_prob.tsv"),
+      organism=species,
+      mane=mane,
+      num_cores=num_cores,
+      mane_version=mane_version,
+      custom_mane_metadata_path=custom_mane_metadata_path
+      )
+    logger.info("Generation of sequences dataframe completed!")
+
+    # Get model confidence
+    logger.info("Extracting AF model confidence...")
+    get_confidence(
+      input=os.path.join(output_datasets, "pdb_structures"),
+      output_dir=os.path.join(output_datasets),
+      seq_df=seq_df
+      )
+
+    # Get PAE
+    logger.info("Downloading AF predicted aligned error (PAE)...")
+    get_pae(
+      input_dir=os.path.join(output_datasets,"pdb_structures"),
+      output_dir=os.path.join(output_datasets,"pae"),
+      num_cores=num_cores,
+      af_version=str(af_version),
+      custom_pdb_dir=custom_pdb_dir
+      )
+
+    # Parse PAE
+    logger.info("Parsing PAE...")
+    parse_pae(input=os.path.join(output_datasets, 'pae'))
+    logger.info("Parsing PAE completed!")
+
+    # Get pCAMPs
+    logger.info("Generating contact probability maps (pCMAPs)..")
+    get_prob_cmaps_mp(
+      input_pdb=os.path.join(output_datasets, "pdb_structures"),
+      input_pae=os.path.join(output_datasets, "pae"),
+      output=os.path.join(output_datasets,"prob_cmaps"),
+      distance=distance_threshold,
+      num_cores=num_cores
+                      )
+    logger.info("Generation pCMAPs completed!")
+
+    # Clean datasets
+    logger.info("Cleaning datasets...")
+    clean_temp_files(path=output_datasets)
+    logger.info("Datasets cleaning completed!")
+    logger.info("Datasets have been successfully built and are ready for analysis!")
+
+if __name__ == "__main__":
+    build(
+      output_datasets="/data/bbg/nobackup/scratch/oncodrive3d/mane_missing/oncodrive3d/datasets/datasets-mane_only-mane_custom-250729",
+      organism="Homo sapiens",
+      mane=False,
+      mane_only=True,
+      custom_pdb_dir="/data/bbg/nobackup/scratch/oncodrive3d/mane_missing/data/250724-no_fragments/all_pdbs-pred_and_retrieved/pdbs",
+      custom_mane_metadata_path="/data/bbg/nobackup/scratch/oncodrive3d/mane_missing/data/250724-no_fragments/all_pdbs-pred_and_retrieved/samplesheet.csv",
+      distance_threshold=10,
+      num_cores=8,
+      af_version=4,
+      mane_version=1.4
+      )

oncodrive3d-1.0.7/scripts/datasets/custom_pdb.py

@@ -0,0 +1,144 @@
+import os
+import gzip
+import shutil
+import daiquiri
+import pandas as pd
+from scripts.datasets.utils import get_seq_from_pdb
+
+from scripts import __logger_name__
+
+logger = daiquiri.getLogger(__logger_name__ + ".build.custom_PDB")
+
+
+# Mapping single-letter to three-letter codes
+one_to_three_res_map = {
+    'A':'ALA','C':'CYS','D':'ASP','E':'GLU','F':'PHE','G':'GLY',
+    'H':'HIS','I':'ILE','K':'LYS','L':'LEU','M':'MET','N':'ASN',
+    'P':'PRO','Q':'GLN','R':'ARG','S':'SER','T':'THR','V':'VAL',
+    'W':'TRP','Y':'TYR'
+}
+
+
+def get_pdb_seqres_records(lst_res):
+    """
+    Given a list of three-letter residue names, construct SEQRES records.
+
+    :param lst_res: List of residue names (e.g. ['MET','ALA',...])
+    :return: List of formatted SEQRES lines ending with '\n'
+    """
+    records = []
+    num_res = len(lst_res)
+    for i in range(0, num_res, 13):
+        block = lst_res[i:i+13]
+        rec_num = len(records) + 1
+        # Atom name block: up to 13 residues separated by spaces, padded to 51 chars
+        residues_str = ' '.join(block)
+        records.append(
+            f"SEQRES {rec_num:>3} A {num_res:>4}  {residues_str:<51}\n"
+        )
+    return records
+
+
+def add_seqres_to_pdb(path_pdb: str, residues: list) -> None:
+    """
+    Insert SEQRES records at the very top of a PDB file (supports gzipped and plain).
+
+    :param path_pdb: Path to the PDB file to modify in-place (can be .pdb or .pdb.gz)
+    :param residues: List of three-letter residue codes for the sequence
+    """
+    # Choose appropriate open functions
+    open_in = gzip.open if path_pdb.endswith('.gz') else open
+    mode_in = 'rt' if path_pdb.endswith('.gz') else 'r'
+    open_out = gzip.open if path_pdb.endswith('.gz') else open
+    mode_out = 'wt' if path_pdb.endswith('.gz') else 'w'
+
+    # Read original lines
+    with open_in(path_pdb, mode_in) as fh:
+        lines = fh.readlines()
+
+    # Generate SEQRES lines
+    seqres = get_pdb_seqres_records(residues)
+
+    # Insert at top
+    new_lines = seqres + lines
+
+    # Write back
+    with open_out(path_pdb, mode_out) as fh:
+        fh.writelines(new_lines)
+        
+        
+def copy_and_parse_custom_pdbs(
+    src_dir: str,
+    dst_dir: str,
+    af_version: int = 4,
+    custom_mane_metadata_path: str = None
+    ) -> None:
+    """
+    Copy all .pdb files from src_dir to dst_dir, renaming and gzipping them.
+    Optionally add residues information (SEQRES) from samplesheet. 
+    """
+    
+    # Ensure destination directory exists
+    os.makedirs(dst_dir, exist_ok=True)
+    
+    # Handle metadata
+    if custom_mane_metadata_path and os.path.isfile(custom_mane_metadata_path):
+        samplesheet_df = pd.read_csv(custom_mane_metadata_path)
+    else:
+        if not custom_mane_metadata_path:
+            logger.warning(
+                "No samplesheet path provided; skipping SEQRES insertion for accession %s",
+                accession
+            )
+        else:
+            logger.warning(
+                "samplesheet not found at '%s'; skipping SEQRES insertion for accession %s",
+                custom_mane_metadata_path,
+                accession
+            )
+        samplesheet_df = None
+
+    # Copy and gzip pdb and optionally add REFSEQ
+    for fname in os.listdir(src_dir):
+        if not fname.endswith('.pdb'):
+            continue
+
+        parts = fname.split('.')  # e.g. [ACCESSION, fragment_code, 'alphafold', 'pdb']
+        if len(parts) < 4:
+            logger.warning(f"Skipping unexpected filename format: {fname}")
+            continue
+
+        accession = parts[0]
+        fragment = parts[1] 
+
+        new_name = f'AF-{accession}-F{fragment}-model_v{af_version}.pdb.gz'
+
+        src_path = os.path.join(src_dir, fname)
+        dst_path = os.path.join(dst_dir, new_name)
+
+        with open(src_path, 'rb') as fin, gzip.open(dst_path, 'wb') as fout:
+            shutil.copyfileobj(fin, fout)
+
+        logger.debug(f'Copied and gzipped: {fname} -> {new_name}')
+        
+        # Optionally add SEQRES records
+        if samplesheet_df is not None:
+            if accession not in samplesheet_df["sequence"].values:
+                logger.warning("Accession %s not in samplesheet %s", accession, custom_mane_metadata_path)
+                continue
+            seq = samplesheet_df[samplesheet_df["sequence"] == accession].refseq.values[0]
+            
+            if not pd.isna(seq):
+                seq = [one_to_three_res_map[aa] for aa in seq]
+                add_seqres_to_pdb(path_pdb=dst_path, residues=seq)
+                logger.debug(f"Inserted SEQRES records into: {new_name}")
+            else:
+                try:
+                    seq = "".join(list(get_seq_from_pdb(dst_path)))
+                    if not pd.isna(seq):
+                        logger.debug(f"SEQRES record already present in the structure: {new_name}")
+                    else:
+                        logger.warning(f"SEQRES not found in samplesheet and its extraction from structure failed: {new_name}")
+                except Exception as e:
+                    logger.warning(f"SEQRES not found in samplesheet and its extraction from structure failed: {new_name}")
+                    logger.warning(f"Exception captured: {e}")

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/scripts/datasets/get_pae.py

@@ -3,6 +3,7 @@ import os
 import daiquiri
 import requests
 import time
+from typing import Optional
 import concurrent.futures
 from tqdm import tqdm
 
@@ -11,14 +12,20 @@ from scripts import __logger_name__
 logger = daiquiri.getLogger(__logger_name__ + ".build.PAE")
 
 
-def download_pae(uniprot_id: str, af_version: int, output_dir: str) -> None:
+def download_pae(
+    uniprot_id: str, 
+    af_version: int, 
+    output_dir: str,
+    max_retries: int = 100
+    ) -> None:
     """
     Download Predicted Aligned Error (PAE) file from AlphaFold DB.
 
     Args:
-        uniprot_id (str): Uniprot ID of the structure.
-        af_version (int): AlphaFold 2 version.
-        output_dir (str): Output directory where to download the PAE files.
+        uniprot_id: Uniprot ID of the structure.
+        af_version: AlphaFold 2 version.
+        output_dir: Output directory where to download the PAE files.
+        max_retries: Break the loop if the download fails too many times.
     """
 
     file_path = os.path.join(output_dir, f"{uniprot_id}-F1-predicted_aligned_error.json")
@@ -28,6 +35,9 @@ def download_pae(uniprot_id: str, af_version: int, output_dir: str) -> None:
     status = "INIT"
     while status != "FINISHED":
         i += 1
+        if i > max_retries:
+            logger.warning(f"Reached {max_retries} attempts; proceeding without download.")
+            break
         if status != "INIT":
             time.sleep(30)
         try:
@@ -43,7 +53,13 @@ def download_pae(uniprot_id: str, af_version: int, output_dir: str) -> None:
                 logger.debug(f"Request failed {e}: Retrying")
 
 
-def get_pae(input_dir: str, output_dir: str, num_cores: int, af_version: int = 4) -> None:
+def get_pae(
+    input_dir: str, 
+    output_dir: str, 
+    num_cores: int, 
+    af_version: int = 4,
+    custom_pdb_dir: Optional[str] = None
+    ) -> None:
     """
     Download Predicted Aligned Error (PAE) files for all non-fragmented PDB
     structures in the input directory.
@@ -53,6 +69,7 @@ def get_pae(input_dir: str, output_dir: str, num_cores: int, af_version: int = 4
         output_dir (str): Output directory where to download the PAE files.
         num_cores (int): Number of cores for multithreading download.
         af_version (int): AlphaFold 2 version (default is 4).
+        custom_pdb_dir (str | None): Directory including provided custom PDB structures.
     """
 
     if not os.path.exists(output_dir):
@@ -62,9 +79,14 @@ def get_pae(input_dir: str, output_dir: str, num_cores: int, af_version: int = 4
     if os.path.exists(checkpoint):
         logger.debug("PAE already downloaded: Skipping...")
         return
-
+    
     pdb_files = [file for file in os.listdir(input_dir) if file.startswith("AF-") and file.endswith(f"-model_v{af_version}.pdb.gz")]
     uniprot_ids = [pdb_file.split("-")[1] for pdb_file in pdb_files]
+    
+    # Do not download PAE for custom provided structures
+    if custom_pdb_dir is not None:
+        custom_uniprot_ids = [fname.split('.')[0] for fname in os.listdir(custom_pdb_dir) if fname.endswith('.pdb')]
+        uniprot_ids = [uni_id for uni_id in uniprot_ids if uni_id not in custom_uniprot_ids]
 
     with concurrent.futures.ThreadPoolExecutor(max_workers=num_cores) as executor:
         tasks = [executor.submit(download_pae, uniprot_id, af_version, output_dir) for uniprot_id in uniprot_ids]

{oncodrive3d-1.0.5 → oncodrive3d-1.0.7}/scripts/datasets/get_structures.py

@@ -22,8 +22,8 @@ def mv_mane_pdb(path_datasets, pdb_dir, mane_pdb_dir) -> None:
 
     path_pdb = os.path.join(path_datasets, pdb_dir)
     path_mane_pdb = os.path.join(path_datasets, mane_pdb_dir)
-    if not os.path.exists(path_mane_pdb):
-        os.makedirs(path_mane_pdb)
+    for path in (path_pdb, path_mane_pdb):
+        os.makedirs(path, exist_ok=True)
 
     # Move MANE structures
     for filename in [file for file in os.listdir(path_mane_pdb) if file.endswith(".pdb.gz") or file.endswith(".pdb")]:
@@ -66,6 +66,8 @@ def get_structures(path: str,
     # Select proteome
     if mane:
         if species == "Homo sapiens":
+            if str(af_version) not in {"3", "4"}:
+                raise RuntimeError("AlphaFold MANE overlaps are available only for versions 3 or 4. When using --mane or --mane_only, please set --af_version to 3 or 4.")
             proteome = f"mane_overlap_v{af_version}"
         else:
             raise RuntimeError("Structures with MANE transcripts overlap are available only for 'Homo sapiens'. Exiting...")
@@ -78,7 +80,7 @@ def get_structures(path: str,
             raise RuntimeError(f"Failed to recognize '{species}' as organism. Currently accepted ones are 'Homo sapiens' and 'Mus musculus'. Exiting...")
 
     logger.debug(f"Proteome to download: {proteome}")
-    af_url = f"https://ftp.ebi.ac.uk/pub/databases/alphafold/latest/{proteome}.tar"
+    af_url = f"https://ftp.ebi.ac.uk/pub/databases/alphafold/v{af_version}/{proteome}.tar"
     file_path = os.path.join(path, f"{proteome}.tar")
 
     try: