offtracker 2.7.8__zip → 2.10.0__zip

offtracker-2.10.0/PKG-INFO

@@ -0,0 +1,233 @@
+Metadata-Version: 2.1
+Name: offtracker
+Version: 2.10.0
+Summary: Tracking-seq data analysis
+Home-page: https://github.com/Lan-lab/offtracker
+Author: Runda Xu
+Author-email: xrd18@tsinghua.org.cn
+Requires-Python: >=3.6.0
+Description-Content-Type: text/markdown
+License-File: LICENSE.txt
+
+
+# OFF-TRACKER
+
+OFF-TRACKER is an end to end pipeline of Tracking-seq data analysis for detecting off-target sites of any genome editing tools that generate double-strand breaks (DSBs) or single-strand breaks (SSBs).
+
+## System requirements
+
+* Linux/Unix 
+* Python >= 3.6
+
+## Dependency
+
+```bash
+# We recommend creating a new environment using mamba/conda to avoid compatibility problems
+# If you don't use mamba, just replace the code with conda 
+# Windows systems may not be compatible with pybedtools.
+mamba create -n offtracker -c bioconda blast snakemake pybedtools chromap
+```
+
+
+## Installation 
+
+```bash
+# Activate the environment
+conda activate offtracker
+
+# Direct installation with pip
+pip install offtracker
+
+# (Alternative) Download the offtracker from github
+git clone https://github.com/Lan-lab/offtracker.git 
+cd offtracker
+pip install .
+```
+
+
+## Before analyzing samples
+
+```bash
+# Build blast index (only need once for each genome)
+makeblastdb -input_type fasta -title hg38 -dbtype nucl -parse_seqids \
+-in /Your_Path_To_Reference/hg38_genome.fa \
+-out /Your_Path_To_Reference/hg38_genome.blastdb \
+-logfile /Your_Path_To_Reference/hg38_genome.blastdb.log
+
+# Build chromap index (only need once for each genome)
+chromap -i -r /Your_Path_To_Reference/hg38_genome.fa \
+-o /Your_Path_To_Reference/hg38_genome.chromap.index
+
+# Generate candidate regions by sgRNA sequence (need once for each genome and sgRNA)
+# --name: a user-defined name of the sgRNA, which will be used in the following analysis.
+offtracker_candidates.py -t 8 -g hg38 \
+-r /Your_Path_To_Reference/hg38_genome.fa \
+-b /Your_Path_To_Reference/hg38_genome.blastdb \
+--name 'VEGFA2' --sgrna 'GACCCCCTCCACCCCGCCTC' --pam 'NGG' \
+-o /Your_Path_To_Candidates_Folder
+
+```
+
+
+## Quality control and adapter trimming
+
+```bash
+# Generate snakemake config file for quality control and adapter trimming.
+offtracker_qc.py -t 4 \
+-f /Your_Path_To_Input_Folder \
+--subfolder 0
+
+cd /Your_Path_To_Input_Folder/Trimmed_data
+snakemake -np # dry run to check whether everything is alright
+nohup snakemake --cores 16 1>${outdir}/sm_qc.log 2>&1 &
+
+"""
+Set “--subfolder 0” if the file structure is like: 
+| - Input_Folder
+  | - sample1_R1.fastq.gz
+  | - sample1_R2.fastq.gz
+  | - sample2_R1.fastq.gz
+  | - sample2_R2.fastq.gz
+Set “--subfolder 1” if the file structure is like:
+| - Input_Folder
+  | - Sample1_Folder
+    | - sample1_R1.fastq.gz
+    | - sample1_R2.fastq.gz
+  | - Sample2_Folder
+    | - sample2_R1.fastq.gz
+    | - sample2_R2.fastq.gz
+
+The script “offtracker_qc.py” will create a “Trimmed_data” folder under /Your_Path_To_Input_Folder. 
+If “-o /Your_Path_To_Output” is set, the output will be redirected to /Your_Path_To_Output.
+"""
+```
+
+## Strand-specific mapping of Tracking-seq data 
+
+```bash
+
+# Generate snakemake config file for mapping
+# Results will be generated in /Your_Path_To_Output, if -o is not set, the output will be in the same folder as the fastq files
+offtracker_config.py -t 8 -g hg38 --blacklist hg38 \
+-r /Your_Path_To_Reference/hg38_genome.fa \
+-i /Your_Path_To_Reference/hg38_genome.chromap.index \
+-f /Your_Path_To_Trimmed_Data \
+-o /Your_Path_To_Output \ 
+--subfolder 0 
+
+# Warning: Do not contain "fastq" or "fq" in the folder name, otherwise the program may treat the folder as a fastq file
+# This problem may be fixed in the future
+
+# Run the snakemake program
+cd /Your_Path_To_Fastq
+snakemake -np # dry run
+nohup snakemake --cores 16 1>sm_mapping.log 2>sm_mapping.err &
+
+## about cores
+# --cores of snakemake must be larger than -t of offtracker_config.py
+# parallel number = cores/t
+
+## about output
+# This part will generate "*.fw.scaled.bw" and ".rv.scaled.bw" for IGV visualization
+# "*.fw.bed" and "*.rv.bed" are used in the next part.
+```
+
+
+## Analyzing the genome-wide off-target sites
+
+```bash
+# In this part, multiple samples in the same condition can be analyzed in a single run by pattern recognition of sample names
+
+offtracker_analysis.py -g hg38 --name "VEGFA2" \
+--exp 'Cas9_VEGFA2' \
+--control 'WT' \
+--outname 'Cas9_VEGFA_293' \
+-f /Your_Path_To_Output \
+--seqfolder /Your_Path_To_Candidates
+
+# --name: the same gRNA name you set when running offtracker_candidates.py
+# --exp/--control: add one or multiple patterns of file name in regular expressions
+# If multiple samples meet the pattern, their signals will be averaged. Thus, only samples with the same condition should be included in a single analysis.
+
+# This step will generate Offtracker_result_{outname}.csv
+# Default FDR is 0.05, which can be changed by --fdr. This will empirically make the threshold of Track score around 2.
+# Sites with Track score >=2, which is a empirical threshold, are output regardless of FDR.
+# Intermediate files are saved in ./temp folder, which can be deleted.
+# Keeping the intermediate files can make the analysis faster if involving previously analyzed samples (e.g. using the same control samples for different analyses)
+```
+
+## Off-target sequences visualization
+
+```bash
+# After get the Offtracker_result_{outname}.csv, you can visualize the off-target sites with their genomic sequence with the following command:
+
+offtracker_plot.py --result Your_Offtracker_Result_CSV \
+--sgrna 'GACCCCCTCCACCCCGCCTC' --pam 'NGG'
+
+# The default output is a pdf file with Offtracker_result_{outname}.pdf
+# Assigning a specific output file with another suffix can change the format. e.g., "--output Offtracker_plot.png" will generate a png file.
+# The orange dash line indicates the empirical threshold of Track score = 2
+# Empirically, the off-target sites with Track score < 2 are less likely to be real off-target sites.
+```
+
+
+## Note1, when not using hg38 or mm10
+
+The default setting only includes chr1-chr22, chrX, chrY, and chrM. (only suitable for human and mouse) \
+If you are using reference genomes without "chr" at the beginning, or want to analyze all chromosomes or other species, you can set "--ignore_chr" when running offtracker_config.py to skip chromosome filter.
+
+Currently, this software is only ready-to-use for mm10 and hg38. For any other genome, e.g., hg19, please add a genome size file named "hg19.chrom.sizes" to .\offtracker\utility. Besides, add "--blacklist none" or "--blacklist Your_Blacklist" (e.g., ENCODE blacklist) when running offtracker_config.py, because we only include blacklists for mm10 and hg38.
+
+## Note2
+
+The FDRs in the Tracking-seq result do not reflect the real off-target probability.
+It is strongly recommended to observe the "fw.scaled.bw" and "rv.scaled.bw" using genome browser like IGV to visually inspect each target location from the Tracking-seq result.
+
+
+
+# Example Data
+
+Here are example data that contains reads of chr6 from HEK293T cells edited with Cas9 + sgRNA VEGFA2 and wild type cells:
+
+https://figshare.com/articles/dataset/WT_HEK239T_chr6/25956034
+
+It takes about 5-10 minutes to run the mapping (offtracker_config.py & snakemake) of example data with -t 8 and --cores 16 (2 parallel tasks)
+
+## Signal visualization
+
+After mapping, there will be 4 .bw files in the output folder:
+```bash
+Cas9_VEGFA2_chr6.fw.scaled.bw
+
+Cas9_VEGFA2_chr6.rv.scaled.bw
+
+WT_chr6.fw.scaled.bw
+
+WT_chr6.rv.scaled.bw
+```
+These files can be visualized in genome browser like IGV:
+
+![signal](https://github.com/Lan-lab/offtracker/blob/main/example_output/signals_example.png?raw=true)
+
+The signal (coverage) for each sample is normalized to 1e7/total_reads. As only reads mapping to chr6 were extracted in the example data, the signal range is much higher than that of the whole genome samples. 
+
+## Whole genome off-target analysis
+
+For analyzing the signals (offtracker_analysis.py), it takes about 3-5 minutes and outputs a file named "Offtracker_result_{outname}.csv"
+
+After that, you can visualize the off-target sites with their genomic sequence (offtracker_plot.py) and get an image like this:
+
+![offtarget](https://github.com/Lan-lab/offtracker/blob/main/example_output/sequences_example.png?raw=true)
+
+# Citation
+
+If you use Tracking-seq or OFF-TRACKER in your research, please cite the following paper:
+
+Zhu, M., Xu, R., Yuan, J., Wang, J. et al. Tracking-seq reveals the heterogeneity of off-target effects in CRISPR–Cas9-mediated genome editing. Nat Biotechnol (2024). https://doi.org/10.1038/s41587-024-02307-y
+
+The signal visualization of .bw file here was generated by the Integrative Genomics Viewer (IGV) software. The signal visualization in the Tracking-seq article above was generated by either IGV or pyGenomeTracks:
+
+Robinson, J., Thorvaldsdóttir, H., Winckler, W. et al. Integrative genomics viewer. Nat Biotechnol 29, 24–26 (2011). https://doi.org/10.1038/nbt.1754
+
+Lopez-Delisle L, Rabbani L, Wolff J, Bhardwaj V, Backofen R, Grüning B, Ramírez F, Manke T. pyGenomeTracks: reproducible plots for multivariate genomic data sets. Bioinformatics. 2020 Aug 3:btaa692. doi: 10.1093/bioinformatics/btaa692.
+

offtracker-2.10.0/README.md

@@ -0,0 +1,221 @@
+# Offtracker
+
+Offtracker is an end to end pipeline of Tracking-seq data analysis for detecting off-target sites of any genome editing tools that generate double-strand breaks (DSBs) or single-strand breaks (SSBs).
+
+## System requirements
+
+* Linux/Unix 
+* Python >= 3.6
+
+## Dependency
+
+```bash
+# We recommend creating a new environment using mamba/conda to avoid compatibility problems
+# If you don't use mamba, just replace the code with conda 
+# Windows systems may not be compatible with pybedtools.
+mamba create -n offtracker -c bioconda blast snakemake pybedtools chromap
+```
+
+
+## Installation 
+
+```bash
+# Activate the environment
+conda activate offtracker
+
+# Direct installation with pip
+pip install offtracker
+
+# (Alternative) Download the offtracker from github
+git clone https://github.com/Lan-lab/offtracker.git 
+cd offtracker
+pip install .
+```
+
+
+## Before analyzing samples
+
+```bash
+# Build blast index (only need once for each genome)
+makeblastdb -input_type fasta -title hg38 -dbtype nucl -parse_seqids \
+-in /Your_Path_To_Reference/hg38_genome.fa \
+-out /Your_Path_To_Reference/hg38_genome.blastdb \
+-logfile /Your_Path_To_Reference/hg38_genome.blastdb.log
+
+# Build chromap index (only need once for each genome)
+chromap -i -r /Your_Path_To_Reference/hg38_genome.fa \
+-o /Your_Path_To_Reference/hg38_genome.chromap.index
+
+# Generate candidate regions by sgRNA sequence (need once for each genome and sgRNA)
+# --name: a user-defined name of the sgRNA, which will be used in the following analysis.
+offtracker_candidates.py -t 8 -g hg38 \
+-r /Your_Path_To_Reference/hg38_genome.fa \
+-b /Your_Path_To_Reference/hg38_genome.blastdb \
+--name 'VEGFA2' --sgrna 'GACCCCCTCCACCCCGCCTC' --pam 'NGG' \
+-o /Your_Path_To_Candidates_Folder
+
+```
+
+
+## Quality control and adapter trimming
+
+```bash
+# Generate snakemake config file for quality control and adapter trimming.
+offtracker_qc.py -t 4 \
+-f /Your_Path_To_Input_Folder \
+--subfolder 0
+
+cd /Your_Path_To_Input_Folder/Trimmed_data
+snakemake -np # dry run to check whether everything is alright
+nohup snakemake --cores 16 1>${outdir}/sm_qc.log 2>&1 &
+
+"""
+Set “--subfolder 0” if the file structure is like: 
+| - Input_Folder
+  | - sample1_R1.fastq.gz
+  | - sample1_R2.fastq.gz
+  | - sample2_R1.fastq.gz
+  | - sample2_R2.fastq.gz
+Set “--subfolder 1” if the file structure is like:
+| - Input_Folder
+  | - Sample1_Folder
+    | - sample1_R1.fastq.gz
+    | - sample1_R2.fastq.gz
+  | - Sample2_Folder
+    | - sample2_R1.fastq.gz
+    | - sample2_R2.fastq.gz
+
+The script “offtracker_qc.py” will create a “Trimmed_data” folder under /Your_Path_To_Input_Folder. 
+If “-o /Your_Path_To_Output” is set, the output will be redirected to /Your_Path_To_Output.
+"""
+```
+
+## Strand-specific mapping of Tracking-seq data 
+
+```bash
+
+# Generate snakemake config file for mapping
+# Results will be generated in /Your_Path_To_Output, if -o is not set, the output will be in the same folder as the fastq files
+offtracker_config.py -t 8 -g hg38 --blacklist hg38 \
+-r /Your_Path_To_Reference/hg38_genome.fa \
+-i /Your_Path_To_Reference/hg38_genome.chromap.index \
+-f /Your_Path_To_Trimmed_Data \
+-o /Your_Path_To_Output \ 
+--subfolder 0 
+
+# Warning: Do not contain "fastq" or "fq" in the folder name, otherwise the program may treat the folder as a fastq file
+# This problem may be fixed in the future
+
+# Run the snakemake program
+cd /Your_Path_To_Fastq
+snakemake -np # dry run
+nohup snakemake --cores 16 1>sm_mapping.log 2>sm_mapping.err &
+
+## about cores
+# --cores of snakemake must be larger than -t of offtracker_config.py
+# parallel number = cores/t
+
+## about output
+# This part will generate "*.fw.scaled.bw" and ".rv.scaled.bw" for IGV visualization
+# "*.fw.bed" and "*.rv.bed" are used in the next part.
+```
+
+
+## Analyzing the genome-wide off-target sites
+
+```bash
+# In this part, multiple samples in the same condition can be analyzed in a single run by pattern recognition of sample names
+
+offtracker_analysis.py -g hg38 --name "VEGFA2" \
+--exp 'Cas9_VEGFA2' \
+--control 'WT' \
+--outname 'Cas9_VEGFA_293' \
+-f /Your_Path_To_Output \
+--seqfolder /Your_Path_To_Candidates
+
+# --name: the same gRNA name you set when running offtracker_candidates.py
+# --exp/--control: add one or multiple patterns of file name in regular expressions
+# If multiple samples meet the pattern, their signals will be averaged. Thus, only samples with the same condition should be included in a single analysis.
+
+# This step will generate Offtracker_result_{outname}.csv
+# Default FDR is 0.05, which can be changed by --fdr. This will empirically make the threshold of Track score around 2.
+# Sites with Track score >=2, which is a empirical threshold, are output regardless of FDR.
+# Intermediate files are saved in ./temp folder, which can be deleted.
+# Keeping the intermediate files can make the analysis faster if involving previously analyzed samples (e.g. using the same control samples for different analyses)
+```
+
+## Off-target sequences visualization
+
+```bash
+# After get the Offtracker_result_{outname}.csv, you can visualize the off-target sites with their genomic sequence with the following command:
+
+offtracker_plot.py --result Your_Offtracker_Result_CSV \
+--sgrna 'GACCCCCTCCACCCCGCCTC' --pam 'NGG'
+
+# The default output is a pdf file with Offtracker_result_{outname}.pdf
+# Assigning a specific output file with another suffix can change the format. e.g., "--output Offtracker_plot.png" will generate a png file.
+# The orange dash line indicates the empirical threshold of Track score = 2
+# Empirically, the off-target sites with Track score < 2 are less likely to be real off-target sites.
+```
+
+
+## Note1, when not using hg38 or mm10
+
+The default setting only includes chr1-chr22, chrX, chrY, and chrM. (only suitable for human and mouse) \
+If you are using reference genomes without "chr" at the beginning, or want to analyze all chromosomes or other species, you can set "--ignore_chr" when running offtracker_config.py to skip chromosome filter.
+
+Currently, this software is only ready-to-use for mm10 and hg38. For any other genome, e.g., hg19, please add a genome size file named "hg19.chrom.sizes" to .\offtracker\utility. Besides, add "--blacklist none" or "--blacklist Your_Blacklist" (e.g., ENCODE blacklist) when running offtracker_config.py, because we only include blacklists for mm10 and hg38.
+
+## Note2
+
+The FDRs in the Tracking-seq result do not reflect the real off-target probability.
+It is strongly recommended to observe the "fw.scaled.bw" and "rv.scaled.bw" using genome browser like IGV to visually inspect each target location from the Tracking-seq result.
+
+
+
+# Example Data
+
+Here are example data that contains reads of chr6 from HEK293T cells edited with Cas9 + sgRNA VEGFA2 and wild type cells:
+
+https://figshare.com/articles/dataset/WT_HEK239T_chr6/25956034
+
+It takes about 5-10 minutes to run the mapping (offtracker_config.py & snakemake) of example data with -t 8 and --cores 16 (2 parallel tasks)
+
+## Signal visualization
+
+After mapping, there will be 4 .bw files in the output folder:
+```bash
+Cas9_VEGFA2_chr6.fw.scaled.bw
+
+Cas9_VEGFA2_chr6.rv.scaled.bw
+
+WT_chr6.fw.scaled.bw
+
+WT_chr6.rv.scaled.bw
+```
+These files can be visualized in genome browser like IGV:
+
+![signal](https://github.com/Lan-lab/offtracker/blob/main/example_output/signals_example.png?raw=true)
+
+The signal (coverage) for each sample is normalized to 1e7/total_reads. As only reads mapping to chr6 were extracted in the example data, the signal range is much higher than that of the whole genome samples. 
+
+## Whole genome off-target analysis
+
+For analyzing the signals (offtracker_analysis.py), it takes about 3-5 minutes and outputs a file named "Offtracker_result_{outname}.csv"
+
+After that, you can visualize the off-target sites with their genomic sequence (offtracker_plot.py) and get an image like this:
+
+![offtarget](https://github.com/Lan-lab/offtracker/blob/main/example_output/sequences_example.png?raw=true)
+
+# Citation
+
+If you use Tracking-seq or OFF-TRACKER in your research, please cite the following paper:
+
+Zhu, M., Xu, R., Yuan, J., Wang, J. et al. Tracking-seq reveals the heterogeneity of off-target effects in CRISPR–Cas9-mediated genome editing. Nat Biotechnol (2024). https://doi.org/10.1038/s41587-024-02307-y
+
+The signal visualization of .bw file here was generated by the Integrative Genomics Viewer (IGV) software. The signal visualization in the Tracking-seq article above was generated by either IGV or pyGenomeTracks:
+
+Robinson, J., Thorvaldsdóttir, H., Winckler, W. et al. Integrative genomics viewer. Nat Biotechnol 29, 24–26 (2011). https://doi.org/10.1038/nbt.1754
+
+Lopez-Delisle L, Rabbani L, Wolff J, Bhardwaj V, Backofen R, Grüning B, Ramírez F, Manke T. pyGenomeTracks: reproducible plots for multivariate genomic data sets. Bioinformatics. 2020 Aug 3:btaa692. doi: 10.1093/bioinformatics/btaa692.
+

{offtracker-2.7.8 → offtracker-2.10.0}/offtracker/X_offplot.py

@@ -1,13 +1,23 @@
-import matplotlib.pyplot as plt
-import matplotlib.patches as patches
+
 import pandas as pd
 import numpy as np
+import matplotlib.pyplot as plt
+import matplotlib.patches as patches
+from matplotlib import rcParams
+# 和用 plt.rcParams or matplotlib.rcParams 是一样的
+dict_rc = {
+    'pdf.fonttype': 42,
+    'font.family': ['Arial']
+}
+rcParams.update(dict_rc)
+
+# 2024.06.03. offtable 添加 threshold 分界线，默认为 None，常用的是 2
 
-def offtable(offtargets, target_guide, 
+def offtable(offtargets, target_guide,  length_pam = 3, 
                          col_seq='best_target', col_score='track_score', col_mismatch='mismatch', col_loc='target_location',
                          title=None, font='Arial', font_size=9, 
-                         box_size_x=15, box_size_y=20, box_gap=1,
-                         x_offset=15, y_offset=35, dpi=200, savefig=None):
+                         box_size_x=15, box_size_y=20, box_gap=1, threshold=None,
+                         x_offset=15, y_offset=35, dpi=300, savefig=None):
     # Facecolor
     color_dict = {
         'A': 'lightgreen',
@@ -19,10 +29,15 @@ def offtable(offtargets, target_guide,
         '-': 'orange'
     }
 
+
+
     # If offtargets is a DataFrame, convert to list of dictionaries
     if isinstance(offtargets, pd.DataFrame):
+        if threshold is not None:
+            n_positive = sum(offtargets[col_score]>=threshold)
         offtargets = offtargets.to_dict(orient='records')
 
+
     # Configuration
     # title=None
     # font='Arial'
@@ -95,16 +110,30 @@ def offtable(offtargets, target_guide,
             ax.text(x + box_size_x / 2, y + box_size_y / 2, "." if c == target_guide[i] else c, ha='center', va='center', family=font, fontsize=font_size, weight='bold')
 
         # Annotations for score, mismatches, and location coordinates
-        ax.text(x_offset + (len(target_guide) + 2) * box_size_x, y + box_size_y / 2, round(seq[col_score],2), ha='center', va='center', family=font, fontsize=font_size)
+        # 2025.06.05. 如果有负数的，用红色显示
+        if seq[col_score]>0:
+            text_color = 'black'
+        else:
+            text_color = 'red'
+        ax.text(x_offset + (len(target_guide) + 2) * box_size_x, y + box_size_y / 2, round(seq[col_score],2), ha='center', va='center', family=font, fontsize=font_size, color=text_color)
         #ax.text(x_offset + (len(target_guide) + 7) * box_size_x, y + box_size_y / 2, "Target" if seq[col_mismatch] == 0 else seq[col_mismatch], ha='center', va='center', family=font, fontsize=font_size, color='red' if seq[col_mismatch] == 0 else 'black')
-        ax.text(x_offset + (len(target_guide) + 4) * box_size_x, y + box_size_y / 2, seq[col_loc], ha='left', va='center', family=font, fontsize=font_size)
+        ax.text(x_offset + (len(target_guide) + 4) * box_size_x, y + box_size_y / 2, seq[col_loc], ha='left', va='center', family=font, fontsize=font_size, color=text_color)
 
+        
     # add a vertical line to indicate the PAM
-    x_line = x_offset + (len(target_guide) - 3) * box_size_x
+    x_line = x_offset + (len(target_guide) - length_pam) * box_size_x
     y_start = y_offset # + box_size_y / 2 
     y_end = y_start + (len(offtargets)+1) * (box_size_y + box_gap)
     ax.vlines(x=x_line, ymin=y_start, ymax=y_end, color='indianred', linestyle='--')
 
+    # 2024.06.03. add a horizontal line to indicate the threshold
+    if threshold is not None:
+        thresh_x_start = x_offset
+        thresh_x_end = x_offset + len(target_guide) * box_size_x
+        thresh_y = y_offset + (n_positive+1) * (box_size_y + box_gap) - box_gap*0.5
+        ax.hlines(y=thresh_y, xmin=thresh_x_start, xmax=thresh_x_end, color='orange', linestyle='--')
+
+
     # Styling and save
     ax.set_xlim(0, width*1.1) # location 的文字太长了，所以要加长一点
     ax.set_ylim(height, 0)