discotoolkit 1.1.4__tar.gz → 1.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (29) hide show
  1. discotoolkit-1.2.0/MANIFEST.in +2 -0
  2. discotoolkit-1.2.0/PKG-INFO +140 -0
  3. discotoolkit-1.2.0/README.md +110 -0
  4. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit/CELLiD.py +41 -22
  5. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit/DownloadDiscoData.py +3 -3
  6. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit/GeneSearch.py +4 -2
  7. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit/GetMetadata.py +4 -6
  8. discotoolkit-1.2.0/discotoolkit/GlobalVariable.py +81 -0
  9. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit/__init__.py +2 -1
  10. discotoolkit-1.2.0/discotoolkit.egg-info/PKG-INFO +140 -0
  11. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit.egg-info/SOURCES.txt +6 -1
  12. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit.egg-info/requires.txt +4 -0
  13. discotoolkit-1.2.0/pyproject.toml +3 -0
  14. discotoolkit-1.2.0/requirements.txt +12 -0
  15. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/setup.py +5 -6
  16. discotoolkit-1.2.0/tests/test_server_contract.py +160 -0
  17. discotoolkit-1.2.0/tests/test_settings.py +90 -0
  18. discotoolkit-1.2.0/tests/test_version.py +41 -0
  19. discotoolkit-1.1.4/MANIFEST.in +0 -1
  20. discotoolkit-1.1.4/PKG-INFO +0 -13
  21. discotoolkit-1.1.4/README.md +0 -72
  22. discotoolkit-1.1.4/discotoolkit/GlobalVariable.py +0 -33
  23. discotoolkit-1.1.4/discotoolkit.egg-info/PKG-INFO +0 -13
  24. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit/CellMapper.py +0 -0
  25. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit/DiscoClass.py +0 -0
  26. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit/Utilities.py +0 -0
  27. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit.egg-info/dependency_links.txt +0 -0
  28. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/discotoolkit.egg-info/top_level.txt +0 -0
  29. {discotoolkit-1.1.4 → discotoolkit-1.2.0}/setup.cfg +0 -0
@@ -0,0 +1,2 @@
1
+ include README.md
2
+ include requirements.txt
@@ -0,0 +1,140 @@
1
+ Metadata-Version: 2.4
2
+ Name: discotoolkit
3
+ Version: 1.2.0
4
+ Summary: DISCOtoolkit is an python package that allows users to access data and use the tools provided by the DISCO database.
5
+ Home-page: https://disco.bii.a-star.edu.sg/
6
+ Author: Li Mengwei, Rom Uddamvathanak
7
+ Author-email: uddamvathanak_rom@immunol.a-star.edu.sg
8
+ Requires-Python: >=3.9
9
+ Description-Content-Type: text/markdown
10
+ Requires-Dist: pandas>=1.4.2
11
+ Requires-Dist: numpy>=1.21.6
12
+ Requires-Dist: scanpy>=1.9.3
13
+ Requires-Dist: scipy>=1.10.1
14
+ Requires-Dist: joblib>=1.1.0
15
+ Requires-Dist: pandarallel>=1.6.5
16
+ Requires-Dist: requests>=2.28.1
17
+ Requires-Dist: colorcet>=3.0.1
18
+ Requires-Dist: leidenalg
19
+ Requires-Dist: h5py>=3.8
20
+ Requires-Dist: matplotlib>=3.5
21
+ Requires-Dist: seaborn>=0.12
22
+ Dynamic: author
23
+ Dynamic: author-email
24
+ Dynamic: description
25
+ Dynamic: description-content-type
26
+ Dynamic: home-page
27
+ Dynamic: requires-dist
28
+ Dynamic: requires-python
29
+ Dynamic: summary
30
+
31
+ <!--
32
+ * @Descripttion:
33
+ * @version:
34
+ * @Author: Mengwei Li
35
+ * @Date: 2023-04-16 21:20:42
36
+ * @LastEditors: Mengwei Li
37
+ * @LastEditTime: 2023-04-16 21:22:03
38
+ -->
39
+ [![Documentation Status](https://readthedocs.org/projects/discotoolkit-py/badge/?version=latest)](https://discotoolkit-py.readthedocs.io/en/latest/?badge=latest) [![Downloads](https://static.pepy.tech/personalized-badge/discotoolkit?period=total&units=international_system&left_color=black&right_color=orange&left_text=Downloads)](https://pepy.tech/project/discotoolkit) [![PyPI version](https://img.shields.io/pypi/v/discotoolkit)](https://pypi.org/project/discotoolkit)
40
+
41
+ # DISCOtoolkit 1.2.0
42
+
43
+ DISCOtoolkit is a Python package for accessing the data and tools of the [DISCO database](https://disco.bii.a-star.edu.sg/) (DISCO v1). Read the documentation at [discotoolkit-py.readthedocs.io](https://discotoolkit-py.readthedocs.io/en/latest/).
44
+
45
+ - Filter and download DISCO data based on sample metadata and cell type information
46
+ - Gene search: a gene's expression across cell types and tissues
47
+ - CELLiD: cell type annotation
48
+ - scEnrichment: gene set enrichment using DISCO DEGs
49
+
50
+ > **DISCO v1 and v2.** The toolkit talks to **DISCO v1** by default. DISCO v2 has its own R package ([DISCOtoolkit](https://github.com/JinmiaoChenLab/DISCOtoolkit)); the Python package can still be pointed at a v2 server, see [Choosing a server](#choosing-a-server).
51
+
52
+ ## Quickstart (Google Colab)
53
+
54
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/quickstart_disco_v1.ipynb)
55
+
56
+ Installs, checks the server, filters, downloads and plots in about ten cells, with nothing to set up locally.
57
+
58
+ ## Installation
59
+
60
+ Python 3.9 or newer. In your current environment:
61
+
62
+ ```
63
+ pip install discotoolkit
64
+ ```
65
+
66
+ To try the latest development version straight from GitHub:
67
+
68
+ ```
69
+ pip install "git+https://github.com/JinmiaoChenLab/DISCOtoolkit_py.git"
70
+ ```
71
+
72
+ To work on the code, or to test changes before they are released, install from a local clone. `-e` links the install to the folder, so edits take effect without reinstalling:
73
+
74
+ ```
75
+ git clone https://github.com/JinmiaoChenLab/DISCOtoolkit_py.git
76
+ cd DISCOtoolkit_py
77
+ pip install -e .
78
+ ```
79
+
80
+ `pip install discotoolkit` and a local install give the same code when the versions match. The difference is the source: PyPI delivers a released, packaged copy, while a local install uses the files in your folder.
81
+
82
+ Dependencies (installed automatically): numpy, pandas, scanpy, scipy, joblib, pandarallel, requests, colorcet, leidenalg, h5py, matplotlib, seaborn.
83
+
84
+ We recommend a virtual environment, for example with miniconda:
85
+
86
+ ```
87
+ conda create --name disco python=3.10
88
+ conda activate disco
89
+ conda install ipykernel
90
+ python -m ipykernel install --user --name disco --display-name "disco"
91
+ python -m pip install -U discotoolkit
92
+ ```
93
+
94
+ ## Choosing a server
95
+
96
+ ```python
97
+ import discotoolkit as dt
98
+
99
+ dt.get_server() # 'https://disco.bii.a-star.edu.sg/disco_v3_api/' (DISCO v1, the default)
100
+ dt.set_server("v2") # DISCO v2
101
+ dt.set_server("https://my.server/disco_v3_api/") # any server with the same API
102
+ ```
103
+
104
+ or set the `DISCO_API_URL` environment variable before importing the package.
105
+
106
+ ## Testing
107
+
108
+ Two small test files need nothing beyond `requests`, so they run anywhere, including Colab:
109
+
110
+ ```
111
+ python tests/test_settings.py # offline: server selection logic
112
+ python tests/test_version.py # offline: setup.py and __init__.py agree on the version
113
+ python tests/test_server_contract.py # online: does the server answer everything the toolkit needs?
114
+ python tests/test_server_contract.py https://immunesinglecell.org/disco_v3_api/ # check another server
115
+ ```
116
+
117
+ GitHub Actions runs these, and installs the package on Python 3.9-3.12, on every push. Maintainers: see [RELEASING.md](RELEASING.md) for how a release is published.
118
+
119
+ `test_server_contract.py` checks status codes and the shape of each response (the columns and file types the toolkit reads). It only samples the large reference files, it does not download them.
120
+
121
+ ## Basic Usage
122
+ Example in Jupyter notebook.
123
+
124
+ <em>please select disco as the kernel for running the jupyter notebook</em>
125
+
126
+ ### [Quickstart for DISCO v1](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/quickstart_disco_v1.ipynb)
127
+
128
+ ### [Filter and download DISCO data](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/download_data.ipynb)
129
+
130
+ ### [Cell Type Annotation using CELLiD](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/CELLiD_celltype_annotation.ipynb)
131
+
132
+ ### [scEnrichment](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/scEnrichment.ipynb)
133
+
134
+ ## Citation
135
+ 1. [Li, Mengwei, et al. "DISCO: a database of Deeply Integrated human Single-Cell Omics data." Nucleic acids research 50.D1 (2022): D596-D602.](https://academic.oup.com/nar/article/50/D1/D596/6430491)
136
+ 2. [Mengwei Li, Kok Siong Ang, Brian Teo, Uddamvathanak Rom, Minh N Nguyen, Sebastian Maurer-Stroh, Jinmiao Chen. "Rediscovering publicly available single-cell data with the DISCO platform." Nucleic Acids Research (2024): gkae1108](https://academic.oup.com/nar/advance-article/doi/10.1093/nar/gkae1108/7899529)
137
+
138
+ ## Follow us on our social media!
139
+ - [HSCRM2](https://twitter.com/HSCRM2)
140
+ - [JinmiaoChenLab Github repo](https://github.com/JinmiaoChenLab)
@@ -0,0 +1,110 @@
1
+ <!--
2
+ * @Descripttion:
3
+ * @version:
4
+ * @Author: Mengwei Li
5
+ * @Date: 2023-04-16 21:20:42
6
+ * @LastEditors: Mengwei Li
7
+ * @LastEditTime: 2023-04-16 21:22:03
8
+ -->
9
+ [![Documentation Status](https://readthedocs.org/projects/discotoolkit-py/badge/?version=latest)](https://discotoolkit-py.readthedocs.io/en/latest/?badge=latest) [![Downloads](https://static.pepy.tech/personalized-badge/discotoolkit?period=total&units=international_system&left_color=black&right_color=orange&left_text=Downloads)](https://pepy.tech/project/discotoolkit) [![PyPI version](https://img.shields.io/pypi/v/discotoolkit)](https://pypi.org/project/discotoolkit)
10
+
11
+ # DISCOtoolkit 1.2.0
12
+
13
+ DISCOtoolkit is a Python package for accessing the data and tools of the [DISCO database](https://disco.bii.a-star.edu.sg/) (DISCO v1). Read the documentation at [discotoolkit-py.readthedocs.io](https://discotoolkit-py.readthedocs.io/en/latest/).
14
+
15
+ - Filter and download DISCO data based on sample metadata and cell type information
16
+ - Gene search: a gene's expression across cell types and tissues
17
+ - CELLiD: cell type annotation
18
+ - scEnrichment: gene set enrichment using DISCO DEGs
19
+
20
+ > **DISCO v1 and v2.** The toolkit talks to **DISCO v1** by default. DISCO v2 has its own R package ([DISCOtoolkit](https://github.com/JinmiaoChenLab/DISCOtoolkit)); the Python package can still be pointed at a v2 server, see [Choosing a server](#choosing-a-server).
21
+
22
+ ## Quickstart (Google Colab)
23
+
24
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/quickstart_disco_v1.ipynb)
25
+
26
+ Installs, checks the server, filters, downloads and plots in about ten cells, with nothing to set up locally.
27
+
28
+ ## Installation
29
+
30
+ Python 3.9 or newer. In your current environment:
31
+
32
+ ```
33
+ pip install discotoolkit
34
+ ```
35
+
36
+ To try the latest development version straight from GitHub:
37
+
38
+ ```
39
+ pip install "git+https://github.com/JinmiaoChenLab/DISCOtoolkit_py.git"
40
+ ```
41
+
42
+ To work on the code, or to test changes before they are released, install from a local clone. `-e` links the install to the folder, so edits take effect without reinstalling:
43
+
44
+ ```
45
+ git clone https://github.com/JinmiaoChenLab/DISCOtoolkit_py.git
46
+ cd DISCOtoolkit_py
47
+ pip install -e .
48
+ ```
49
+
50
+ `pip install discotoolkit` and a local install give the same code when the versions match. The difference is the source: PyPI delivers a released, packaged copy, while a local install uses the files in your folder.
51
+
52
+ Dependencies (installed automatically): numpy, pandas, scanpy, scipy, joblib, pandarallel, requests, colorcet, leidenalg, h5py, matplotlib, seaborn.
53
+
54
+ We recommend a virtual environment, for example with miniconda:
55
+
56
+ ```
57
+ conda create --name disco python=3.10
58
+ conda activate disco
59
+ conda install ipykernel
60
+ python -m ipykernel install --user --name disco --display-name "disco"
61
+ python -m pip install -U discotoolkit
62
+ ```
63
+
64
+ ## Choosing a server
65
+
66
+ ```python
67
+ import discotoolkit as dt
68
+
69
+ dt.get_server() # 'https://disco.bii.a-star.edu.sg/disco_v3_api/' (DISCO v1, the default)
70
+ dt.set_server("v2") # DISCO v2
71
+ dt.set_server("https://my.server/disco_v3_api/") # any server with the same API
72
+ ```
73
+
74
+ or set the `DISCO_API_URL` environment variable before importing the package.
75
+
76
+ ## Testing
77
+
78
+ Two small test files need nothing beyond `requests`, so they run anywhere, including Colab:
79
+
80
+ ```
81
+ python tests/test_settings.py # offline: server selection logic
82
+ python tests/test_version.py # offline: setup.py and __init__.py agree on the version
83
+ python tests/test_server_contract.py # online: does the server answer everything the toolkit needs?
84
+ python tests/test_server_contract.py https://immunesinglecell.org/disco_v3_api/ # check another server
85
+ ```
86
+
87
+ GitHub Actions runs these, and installs the package on Python 3.9-3.12, on every push. Maintainers: see [RELEASING.md](RELEASING.md) for how a release is published.
88
+
89
+ `test_server_contract.py` checks status codes and the shape of each response (the columns and file types the toolkit reads). It only samples the large reference files, it does not download them.
90
+
91
+ ## Basic Usage
92
+ Example in Jupyter notebook.
93
+
94
+ <em>please select disco as the kernel for running the jupyter notebook</em>
95
+
96
+ ### [Quickstart for DISCO v1](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/quickstart_disco_v1.ipynb)
97
+
98
+ ### [Filter and download DISCO data](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/download_data.ipynb)
99
+
100
+ ### [Cell Type Annotation using CELLiD](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/CELLiD_celltype_annotation.ipynb)
101
+
102
+ ### [scEnrichment](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/scEnrichment.ipynb)
103
+
104
+ ## Citation
105
+ 1. [Li, Mengwei, et al. "DISCO: a database of Deeply Integrated human Single-Cell Omics data." Nucleic acids research 50.D1 (2022): D596-D602.](https://academic.oup.com/nar/article/50/D1/D596/6430491)
106
+ 2. [Mengwei Li, Kok Siong Ang, Brian Teo, Uddamvathanak Rom, Minh N Nguyen, Sebastian Maurer-Stroh, Jinmiao Chen. "Rediscovering publicly available single-cell data with the DISCO platform." Nucleic Acids Research (2024): gkae1108](https://academic.oup.com/nar/advance-article/doi/10.1093/nar/gkae1108/7899529)
107
+
108
+ ## Follow us on our social media!
109
+ - [HSCRM2](https://twitter.com/HSCRM2)
110
+ - [JinmiaoChenLab Github repo](https://github.com/JinmiaoChenLab)
@@ -18,7 +18,7 @@ from pandarallel import pandarallel # multiprocessing library
18
18
  pd.options.mode.chained_assignment = None
19
19
 
20
20
  # import variable and class from other script
21
- from .GlobalVariable import logging, prefix_disco_url, timeout
21
+ from .GlobalVariable import logging, api_url, timeout
22
22
  from .DiscoClass import FilterData, Filter
23
23
  from .GetMetadata import check_in_list
24
24
 
@@ -77,7 +77,7 @@ def get_atlas(ref_data=None, ref_path=None):
77
77
  if not (os.path.exists(ref_path + "/ref_data.pkl")):
78
78
  logging.info("Downloading reference dataset...")
79
79
  response = requests.get(
80
- url=prefix_disco_url + "toolkit/getRef?type=pkl", timeout=timeout
80
+ url=api_url("toolkit/getRef?type=pkl"), timeout=timeout
81
81
  )
82
82
  open(ref_path + "/ref_data.pkl", "wb").write(response.content)
83
83
  ref_data = pd.read_pickle(
@@ -89,6 +89,42 @@ def get_atlas(ref_data=None, ref_path=None):
89
89
  return list(set(all_atlas))
90
90
 
91
91
 
92
+ def _load_geneset(ref_path: str) -> pd.DataFrame:
93
+ """Get the gene-set reference (columns logfc, gene, atlas, name), downloading it once.
94
+
95
+ DISCO v1 serves it as plain CSV (`toolkit/getGeneSet?format=csv`), which is read with
96
+ pandas.read_csv. Older servers only offer a pickle (`getGeneSetPkl`); that is used as a
97
+ fallback, with a warning, because unpickling data from a network is only safe if the
98
+ server is trusted.
99
+ """
100
+ csv_file = ref_path + "/ref_geneset.csv"
101
+ pkl_file = ref_path + "/ref_geneset.pkl"
102
+
103
+ if os.path.exists(csv_file):
104
+ return pd.read_csv(csv_file)
105
+ if os.path.exists(pkl_file):
106
+ return pd.read_pickle(pkl_file, compression={"method": "gzip", "compresslevel": 6})
107
+
108
+ logging.info("Downloading gene set reference...")
109
+ response = requests.get(
110
+ url=api_url("toolkit/getGeneSet"), params={"format": "csv"}, timeout=timeout
111
+ )
112
+ if response.status_code == 200:
113
+ with open(csv_file, "wb") as handle:
114
+ handle.write(response.content)
115
+ return pd.read_csv(csv_file)
116
+
117
+ logging.warning(
118
+ "This server has no CSV gene set (HTTP %s); falling back to the pickle download.",
119
+ response.status_code,
120
+ )
121
+ response = requests.get(url=api_url("getGeneSetPkl"), timeout=timeout)
122
+ response.raise_for_status()
123
+ with open(pkl_file, "wb") as handle:
124
+ handle.write(response.content)
125
+ return pd.read_pickle(pkl_file, compression={"method": "gzip", "compresslevel": 6})
126
+
127
+
92
128
  def CELLiD_cluster(
93
129
  rna,
94
130
  ref_data: pd.DataFrame = None,
@@ -144,7 +180,7 @@ def CELLiD_cluster(
144
180
  if not (os.path.exists(ref_path + "/ref_data.pkl")):
145
181
  logging.info("Downloading reference dataset...")
146
182
  response = requests.get(
147
- url=prefix_disco_url + "toolkit/getRef?type=pkl", timeout=timeout
183
+ url=api_url("toolkit/getRef?type=pkl"), timeout=timeout
148
184
  )
149
185
  open(ref_path + "/ref_data.pkl", "wb").write(response.content)
150
186
  ref_data = pd.read_pickle(
@@ -157,7 +193,7 @@ def CELLiD_cluster(
157
193
  if not (os.path.exists(ref_path + "/ref_deg.pkl")):
158
194
  logging.info("Downloading deg dataset...")
159
195
  response = requests.get(
160
- url=prefix_disco_url + "toolkit/getRefDeg?type=pkl", timeout=timeout
196
+ url=api_url("toolkit/getRefDeg?type=pkl"), timeout=timeout
161
197
  )
162
198
  open(ref_path + "/ref_deg.pkl", "wb").write(response.content)
163
199
  ref_deg = pd.read_pickle(
@@ -347,24 +383,7 @@ def CELLiD_enrichment(
347
383
  if not os.path.exists(ref_path):
348
384
  os.mkdir(ref_path)
349
385
 
350
- # check if the file does not exist
351
- if not (os.path.exists(ref_path + "/ref_geneset.pkl")):
352
- # downloading the geneset data from disco database
353
- response = requests.get(
354
- url=prefix_disco_url + "/getGeneSetPkl", timeout=timeout
355
- )
356
- open(ref_path + "/ref_geneset.pkl", "wb").write(response.content)
357
- reference = pd.read_pickle(
358
- ref_path + "/ref_geneset.pkl",
359
- compression={"method": "gzip", "compresslevel": 6},
360
- )
361
-
362
- else:
363
- # read the data into pandas dataframe for subsequent analysis
364
- reference = pd.read_pickle(
365
- ref_path + "/ref_geneset.pkl",
366
- compression={"method": "gzip", "compresslevel": 6},
367
- )
386
+ reference = _load_geneset(ref_path)
368
387
 
369
388
  # rename the name data to include the reference atlas
370
389
  reference["name"] = reference["name"] + " in " + reference["atlas"]
@@ -17,7 +17,7 @@ import hashlib
17
17
  import scanpy as sc
18
18
 
19
19
  # import variable and class from other script
20
- from .GlobalVariable import logging, prefix_disco_url
20
+ from .GlobalVariable import logging, api_url
21
21
  from .DiscoClass import FilterData, Filter
22
22
  from .GetMetadata import check_in_list
23
23
 
@@ -52,11 +52,11 @@ def download_disco_data(metadata, output_dir: str = "DISCOtmp"):
52
52
  if os.path.exists(output_file):
53
53
  os.remove(output_file)
54
54
 
55
- url = prefix_disco_url + "download/getRawH5/" + p + "/" + s
55
+ url = api_url("download/getRawH5/" + p + "/" + s)
56
56
  rna = sc.read_10x_h5(filename=output_file, backup_url=url)
57
57
 
58
58
  cell = pd.read_csv(
59
- prefix_disco_url + "toolkit/" + "getCellTypeSample?sampleId=" + s,
59
+ api_url("toolkit/getCellTypeSample?sampleId=" + s),
60
60
  sep="\t",
61
61
  header=0,
62
62
  )
@@ -7,6 +7,8 @@ import numpy as np
7
7
  from typing import Union
8
8
  import colorcet as cc
9
9
 
10
+ from .GlobalVariable import api_url
11
+
10
12
 
11
13
  def gene_search(gene : str, atlas : Union[str, list] = None, figsize : tuple = None, dpi : int = 300):
12
14
 
@@ -27,8 +29,8 @@ def gene_search(gene : str, atlas : Union[str, list] = None, figsize : tuple = N
27
29
  atlas = [atlas]
28
30
 
29
31
  # url to get the API data
30
- url = "https://www.immunesinglecell.org/api/vishuo/geneExp/getRefExp?gene=%s" % (gene)
31
- response = requests.get(url)
32
+ response = requests.get(api_url("geneExp/getRefExp"), params={"gene": gene})
33
+ response.raise_for_status()
32
34
  data = response.json()
33
35
 
34
36
  # Extract relevant information from JSON data
@@ -11,7 +11,7 @@ import numpy as np
11
11
  from typing import Union
12
12
 
13
13
  # import variable and class from other script
14
- from .GlobalVariable import logging, prefix_disco_url
14
+ from .GlobalVariable import logging, api_url
15
15
  from .DiscoClass import FilterData, Filter
16
16
 
17
17
 
@@ -35,7 +35,7 @@ def get_json(url: str, info_msg: str, error_msg: str, prefix: bool = True):
35
35
  if not prefix:
36
36
  response = requests.get(url)
37
37
  else:
38
- response = requests.get(prefix_disco_url + url)
38
+ response = requests.get(api_url(url))
39
39
 
40
40
  # if the loading of JSON is successful, we get the text
41
41
  if response.status_code == 200:
@@ -51,9 +51,7 @@ def get_sample_ct_info():
51
51
  Returns:
52
52
  Pandas DataFrame: return pandas dataframe to the user
53
53
  """
54
- temp = pd.read_csv(
55
- prefix_disco_url + "toolkit/" + "getCellTypeSummary", sep="\t", header=0
56
- )
54
+ temp = pd.read_csv(api_url("toolkit/getCellTypeSummary"), sep="\t", header=0)
57
55
  return temp # return Dataframe of the JSON data
58
56
 
59
57
 
@@ -101,7 +99,7 @@ def get_disco_metadata():
101
99
  """
102
100
 
103
101
  metadata = pd.read_csv(
104
- "https://immunesinglecell.org/disco_v3_api/toolkit/getSampleMetadata",
102
+ api_url("toolkit/getSampleMetadata"),
105
103
  sep="\t",
106
104
  header=0,
107
105
  )
@@ -0,0 +1,81 @@
1
+ """
2
+ Settings shared by the rest of the package: logging, the request timeout and --
3
+ most importantly -- which DISCO server the toolkit talks to.
4
+
5
+ By default the toolkit talks to DISCO v1 (https://disco.bii.a-star.edu.sg). To use
6
+ another server:
7
+
8
+ import discotoolkit as dt
9
+ dt.set_server("v2") # a named preset
10
+ dt.set_server("https://my.server/disco_v3_api/") # any server with the same API
11
+
12
+ or set the DISCO_API_URL environment variable before importing the package.
13
+ """
14
+
15
+ import logging
16
+ import os
17
+
18
+ logging.basicConfig(level=logging.INFO)
19
+
20
+ # seconds to wait for the server; the reference downloads are tens of MB
21
+ timeout = 600
22
+
23
+ # The API roots the toolkit knows by name. Both serve the same `toolkit/...` endpoints.
24
+ SERVERS = {
25
+ "v1": "https://disco.bii.a-star.edu.sg/disco_v3_api/",
26
+ "v2": "https://immunesinglecell.org/disco_v3_api/",
27
+ }
28
+ DEFAULT_SERVER = "v1"
29
+
30
+
31
+ def _normalise(server: str) -> str:
32
+ """Turn a preset name or a URL into an API root that ends with exactly one '/'."""
33
+ if not isinstance(server, str) or not server.strip():
34
+ raise ValueError("server must be a preset name (%s) or a URL" % ", ".join(SERVERS))
35
+ server = server.strip()
36
+ if server.lower() in SERVERS:
37
+ return SERVERS[server.lower()]
38
+ if not server.lower().startswith(("http://", "https://")):
39
+ raise ValueError(
40
+ "unknown server %r: use one of %s or a full http(s):// URL" % (server, ", ".join(SERVERS))
41
+ )
42
+ return server.rstrip("/") + "/"
43
+
44
+
45
+ _server = _normalise(os.environ.get("DISCO_API_URL") or DEFAULT_SERVER)
46
+
47
+
48
+ def set_server(server: str) -> str:
49
+ """Choose which DISCO server the toolkit uses for the rest of the session.
50
+
51
+ Args:
52
+ server (str): "v1", "v2", or the full URL of an API root such as
53
+ "https://disco.bii.a-star.edu.sg/disco_v3_api/".
54
+
55
+ Returns:
56
+ str: the API root now in use.
57
+ """
58
+ global _server
59
+ _server = _normalise(server)
60
+ logging.info("DISCOtoolkit is using %s", _server)
61
+ return _server
62
+
63
+
64
+ def get_server() -> str:
65
+ """The API root the toolkit is currently using."""
66
+ return _server
67
+
68
+
69
+ def api_url(path: str) -> str:
70
+ """Full URL for an API path, e.g. api_url("toolkit/getSampleMetadata").
71
+
72
+ Looked up on every call, so set_server() takes effect immediately everywhere.
73
+ """
74
+ return _server + path.lstrip("/")
75
+
76
+
77
+ def __getattr__(name):
78
+ # `prefix_disco_url` was a plain variable in earlier releases; keep it readable.
79
+ if name == "prefix_disco_url":
80
+ return _server
81
+ raise AttributeError("module %r has no attribute %r" % (__name__, name))
@@ -3,5 +3,6 @@ from .DiscoClass import *
3
3
  from .DownloadDiscoData import *
4
4
  from .CELLiD import *
5
5
  from .GeneSearch import *
6
+ from .GlobalVariable import set_server, get_server
6
7
 
7
- __version__ = "1.1.4"
8
+ __version__ = "1.2.0"
@@ -0,0 +1,140 @@
1
+ Metadata-Version: 2.4
2
+ Name: discotoolkit
3
+ Version: 1.2.0
4
+ Summary: DISCOtoolkit is an python package that allows users to access data and use the tools provided by the DISCO database.
5
+ Home-page: https://disco.bii.a-star.edu.sg/
6
+ Author: Li Mengwei, Rom Uddamvathanak
7
+ Author-email: uddamvathanak_rom@immunol.a-star.edu.sg
8
+ Requires-Python: >=3.9
9
+ Description-Content-Type: text/markdown
10
+ Requires-Dist: pandas>=1.4.2
11
+ Requires-Dist: numpy>=1.21.6
12
+ Requires-Dist: scanpy>=1.9.3
13
+ Requires-Dist: scipy>=1.10.1
14
+ Requires-Dist: joblib>=1.1.0
15
+ Requires-Dist: pandarallel>=1.6.5
16
+ Requires-Dist: requests>=2.28.1
17
+ Requires-Dist: colorcet>=3.0.1
18
+ Requires-Dist: leidenalg
19
+ Requires-Dist: h5py>=3.8
20
+ Requires-Dist: matplotlib>=3.5
21
+ Requires-Dist: seaborn>=0.12
22
+ Dynamic: author
23
+ Dynamic: author-email
24
+ Dynamic: description
25
+ Dynamic: description-content-type
26
+ Dynamic: home-page
27
+ Dynamic: requires-dist
28
+ Dynamic: requires-python
29
+ Dynamic: summary
30
+
31
+ <!--
32
+ * @Descripttion:
33
+ * @version:
34
+ * @Author: Mengwei Li
35
+ * @Date: 2023-04-16 21:20:42
36
+ * @LastEditors: Mengwei Li
37
+ * @LastEditTime: 2023-04-16 21:22:03
38
+ -->
39
+ [![Documentation Status](https://readthedocs.org/projects/discotoolkit-py/badge/?version=latest)](https://discotoolkit-py.readthedocs.io/en/latest/?badge=latest) [![Downloads](https://static.pepy.tech/personalized-badge/discotoolkit?period=total&units=international_system&left_color=black&right_color=orange&left_text=Downloads)](https://pepy.tech/project/discotoolkit) [![PyPI version](https://img.shields.io/pypi/v/discotoolkit)](https://pypi.org/project/discotoolkit)
40
+
41
+ # DISCOtoolkit 1.2.0
42
+
43
+ DISCOtoolkit is a Python package for accessing the data and tools of the [DISCO database](https://disco.bii.a-star.edu.sg/) (DISCO v1). Read the documentation at [discotoolkit-py.readthedocs.io](https://discotoolkit-py.readthedocs.io/en/latest/).
44
+
45
+ - Filter and download DISCO data based on sample metadata and cell type information
46
+ - Gene search: a gene's expression across cell types and tissues
47
+ - CELLiD: cell type annotation
48
+ - scEnrichment: gene set enrichment using DISCO DEGs
49
+
50
+ > **DISCO v1 and v2.** The toolkit talks to **DISCO v1** by default. DISCO v2 has its own R package ([DISCOtoolkit](https://github.com/JinmiaoChenLab/DISCOtoolkit)); the Python package can still be pointed at a v2 server, see [Choosing a server](#choosing-a-server).
51
+
52
+ ## Quickstart (Google Colab)
53
+
54
+ [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/quickstart_disco_v1.ipynb)
55
+
56
+ Installs, checks the server, filters, downloads and plots in about ten cells, with nothing to set up locally.
57
+
58
+ ## Installation
59
+
60
+ Python 3.9 or newer. In your current environment:
61
+
62
+ ```
63
+ pip install discotoolkit
64
+ ```
65
+
66
+ To try the latest development version straight from GitHub:
67
+
68
+ ```
69
+ pip install "git+https://github.com/JinmiaoChenLab/DISCOtoolkit_py.git"
70
+ ```
71
+
72
+ To work on the code, or to test changes before they are released, install from a local clone. `-e` links the install to the folder, so edits take effect without reinstalling:
73
+
74
+ ```
75
+ git clone https://github.com/JinmiaoChenLab/DISCOtoolkit_py.git
76
+ cd DISCOtoolkit_py
77
+ pip install -e .
78
+ ```
79
+
80
+ `pip install discotoolkit` and a local install give the same code when the versions match. The difference is the source: PyPI delivers a released, packaged copy, while a local install uses the files in your folder.
81
+
82
+ Dependencies (installed automatically): numpy, pandas, scanpy, scipy, joblib, pandarallel, requests, colorcet, leidenalg, h5py, matplotlib, seaborn.
83
+
84
+ We recommend a virtual environment, for example with miniconda:
85
+
86
+ ```
87
+ conda create --name disco python=3.10
88
+ conda activate disco
89
+ conda install ipykernel
90
+ python -m ipykernel install --user --name disco --display-name "disco"
91
+ python -m pip install -U discotoolkit
92
+ ```
93
+
94
+ ## Choosing a server
95
+
96
+ ```python
97
+ import discotoolkit as dt
98
+
99
+ dt.get_server() # 'https://disco.bii.a-star.edu.sg/disco_v3_api/' (DISCO v1, the default)
100
+ dt.set_server("v2") # DISCO v2
101
+ dt.set_server("https://my.server/disco_v3_api/") # any server with the same API
102
+ ```
103
+
104
+ or set the `DISCO_API_URL` environment variable before importing the package.
105
+
106
+ ## Testing
107
+
108
+ Two small test files need nothing beyond `requests`, so they run anywhere, including Colab:
109
+
110
+ ```
111
+ python tests/test_settings.py # offline: server selection logic
112
+ python tests/test_version.py # offline: setup.py and __init__.py agree on the version
113
+ python tests/test_server_contract.py # online: does the server answer everything the toolkit needs?
114
+ python tests/test_server_contract.py https://immunesinglecell.org/disco_v3_api/ # check another server
115
+ ```
116
+
117
+ GitHub Actions runs these, and installs the package on Python 3.9-3.12, on every push. Maintainers: see [RELEASING.md](RELEASING.md) for how a release is published.
118
+
119
+ `test_server_contract.py` checks status codes and the shape of each response (the columns and file types the toolkit reads). It only samples the large reference files, it does not download them.
120
+
121
+ ## Basic Usage
122
+ Example in Jupyter notebook.
123
+
124
+ <em>please select disco as the kernel for running the jupyter notebook</em>
125
+
126
+ ### [Quickstart for DISCO v1](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/quickstart_disco_v1.ipynb)
127
+
128
+ ### [Filter and download DISCO data](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/download_data.ipynb)
129
+
130
+ ### [Cell Type Annotation using CELLiD](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/CELLiD_celltype_annotation.ipynb)
131
+
132
+ ### [scEnrichment](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/scEnrichment.ipynb)
133
+
134
+ ## Citation
135
+ 1. [Li, Mengwei, et al. "DISCO: a database of Deeply Integrated human Single-Cell Omics data." Nucleic acids research 50.D1 (2022): D596-D602.](https://academic.oup.com/nar/article/50/D1/D596/6430491)
136
+ 2. [Mengwei Li, Kok Siong Ang, Brian Teo, Uddamvathanak Rom, Minh N Nguyen, Sebastian Maurer-Stroh, Jinmiao Chen. "Rediscovering publicly available single-cell data with the DISCO platform." Nucleic Acids Research (2024): gkae1108](https://academic.oup.com/nar/advance-article/doi/10.1093/nar/gkae1108/7899529)
137
+
138
+ ## Follow us on our social media!
139
+ - [HSCRM2](https://twitter.com/HSCRM2)
140
+ - [JinmiaoChenLab Github repo](https://github.com/JinmiaoChenLab)
@@ -1,5 +1,7 @@
1
1
  MANIFEST.in
2
2
  README.md
3
+ pyproject.toml
4
+ requirements.txt
3
5
  setup.py
4
6
  discotoolkit/CELLiD.py
5
7
  discotoolkit/CellMapper.py
@@ -14,4 +16,7 @@ discotoolkit.egg-info/PKG-INFO
14
16
  discotoolkit.egg-info/SOURCES.txt
15
17
  discotoolkit.egg-info/dependency_links.txt
16
18
  discotoolkit.egg-info/requires.txt
17
- discotoolkit.egg-info/top_level.txt
19
+ discotoolkit.egg-info/top_level.txt
20
+ tests/test_server_contract.py
21
+ tests/test_settings.py
22
+ tests/test_version.py
@@ -6,3 +6,7 @@ joblib>=1.1.0
6
6
  pandarallel>=1.6.5
7
7
  requests>=2.28.1
8
8
  colorcet>=3.0.1
9
+ leidenalg
10
+ h5py>=3.8
11
+ matplotlib>=3.5
12
+ seaborn>=0.12
@@ -0,0 +1,3 @@
1
+ [build-system]
2
+ requires = ["setuptools>=61", "wheel"]
3
+ build-backend = "setuptools.build_meta"
@@ -0,0 +1,12 @@
1
+ pandas>=1.4.2
2
+ numpy>=1.21.6
3
+ scanpy>=1.9.3
4
+ scipy>=1.10.1
5
+ joblib>=1.1.0
6
+ pandarallel>=1.6.5
7
+ requests>=2.28.1
8
+ colorcet>=3.0.1
9
+ leidenalg
10
+ h5py>=3.8
11
+ matplotlib>=3.5
12
+ seaborn>=0.12
@@ -6,16 +6,15 @@ with open("requirements.txt", "r") as f:
6
6
 
7
7
  setup(
8
8
  name='discotoolkit',
9
- version="1.1.4",
10
- url='http://www.immunesinglecell.org/',
9
+ version="1.2.0",
10
+ url='https://disco.bii.a-star.edu.sg/',
11
11
  author='Li Mengwei, Rom Uddamvathanak',
12
12
  author_email='uddamvathanak_rom@immunol.a-star.edu.sg',
13
13
  description='DISCOtoolkit is an python package that allows users to access data and use the tools provided by the DISCO database.',
14
14
  packages=find_packages(include=["discotoolkit", "discotoolkit.*"]),
15
15
  install_requires=requirements,
16
- python_requires = '>=3.8.16',
16
+ python_requires = '>=3.9',
17
17
  include_package_data=True,
18
- long_description="DISCOtoolkit is an python package that allows users to access data and use the tools provided by the DISCO database. \
19
- It provides the following functions\nFilter and download DISCO data based on sample metadata and cell type information\nCELLiD: \
20
- cell type annotation\nscEnrichment: geneset enrichment using DISCO DEGs",
18
+ long_description=open("README.md", encoding="utf-8").read(),
19
+ long_description_content_type="text/markdown",
21
20
  )
@@ -0,0 +1,160 @@
1
+ """
2
+ Does a DISCO server answer everything discotoolkit needs?
3
+
4
+ Needs only `requests`, so it runs anywhere -- including a fresh Colab notebook:
5
+
6
+ !pip install requests
7
+ !python tests/test_server_contract.py # DISCO v1 (default)
8
+ !python tests/test_server_contract.py https://immunesinglecell.org/disco_v3_api/ # another server
9
+
10
+ or with pytest: DISCO_API_URL=<api root> pytest tests/test_server_contract.py
11
+
12
+ It checks status codes and the *shape* of each response (the columns and file types the
13
+ toolkit reads), not the science. Large reference files are only sampled (first bytes), never
14
+ downloaded in full.
15
+ """
16
+ import os
17
+ import sys
18
+
19
+ import requests
20
+
21
+ DEFAULT = "https://disco.bii.a-star.edu.sg/disco_v3_api/"
22
+ BASE = (sys.argv[1] if len(sys.argv) > 1 and sys.argv[1].startswith("http") else os.environ.get("DISCO_API_URL", DEFAULT)).rstrip("/") + "/"
23
+ TIMEOUT = 120
24
+
25
+
26
+ def url(path):
27
+ return BASE + path.lstrip("/")
28
+
29
+
30
+ def first_lines(path, n=3, **params):
31
+ """First n text lines of a (possibly huge) response, without downloading the rest."""
32
+ with requests.get(url(path), params=params, stream=True, timeout=TIMEOUT) as r:
33
+ assert r.status_code == 200, "HTTP %s for %s" % (r.status_code, path)
34
+ # Downloads are served as application/octet-stream, which has no charset, so without
35
+ # this requests would hand back bytes instead of text.
36
+ r.encoding = "utf-8"
37
+ lines = []
38
+ for line in r.iter_lines(decode_unicode=True):
39
+ lines.append(line)
40
+ if len(lines) >= n:
41
+ break
42
+ return lines
43
+
44
+
45
+ def first_bytes(path, n=8, **params):
46
+ with requests.get(url(path), params=params, stream=True, timeout=TIMEOUT) as r:
47
+ assert r.status_code == 200, "HTTP %s for %s" % (r.status_code, path)
48
+ size = r.headers.get("Content-Length")
49
+ return next(r.iter_content(chunk_size=n)), (int(size) if size else None)
50
+
51
+
52
+ def columns(line):
53
+ return line.split("\t")
54
+
55
+
56
+ # ---- metadata the Filter / FilterData classes work on ------------------------------------
57
+
58
+ def test_sample_metadata_has_the_filter_columns():
59
+ header, row = first_lines("toolkit/getSampleMetadata", 2)
60
+ need = {"sample_id", "project_id", "sample_type", "tissue", "disease", "platform", "cell_number"}
61
+ missing = need - set(columns(header))
62
+ assert not missing, "getSampleMetadata lacks columns: %s" % sorted(missing)
63
+ assert len(columns(row)) == len(columns(header)), "rows are not tab-aligned with the header"
64
+
65
+
66
+ def test_cell_type_summary_columns():
67
+ header, row = first_lines("toolkit/getCellTypeSummary", 2)
68
+ assert columns(header) == ["sample_id", "cell_type", "cell_type_score", "cell_number"], columns(header)
69
+ float(columns(row)[2]); int(columns(row)[3])
70
+
71
+
72
+ def test_cell_ontology_is_a_list_of_cell_name_and_parent():
73
+ r = requests.get(url("toolkit/getCellOntology"), timeout=TIMEOUT)
74
+ assert r.status_code == 200
75
+ data = r.json()
76
+ assert isinstance(data, list) and len(data) > 100
77
+ assert {"cell_name", "parent"} <= set(data[0]), data[0]
78
+
79
+
80
+ # ---- per-sample download (download_disco_data) ---------------------------------------------
81
+
82
+ def _a_sample():
83
+ header, row = first_lines("toolkit/getSampleMetadata", 2)
84
+ cols = columns(header)
85
+ values = columns(row)
86
+ return values[cols.index("project_id")], values[cols.index("sample_id")]
87
+
88
+
89
+ def test_cell_type_labels_for_one_sample():
90
+ project, sample = _a_sample()
91
+ header, row = first_lines("toolkit/getCellTypeSample", 2, sampleId=sample)
92
+ cols = columns(header)
93
+ # the toolkit reads columns 0, 2 and 5 by position, so their order matters
94
+ assert cols[0] == "cell_id" and cols[2] == "cell_type" and cols[5] == "cell_type_score", cols
95
+
96
+
97
+ def test_raw_10x_h5_is_downloadable_and_is_hdf5():
98
+ project, sample = _a_sample()
99
+ head, size = first_bytes("download/getRawH5/%s/%s" % (project, sample))
100
+ assert head.startswith(b"\x89HDF"), "not an HDF5 file: %r" % head
101
+ assert size is None or size > 1000
102
+
103
+
104
+ # ---- CELLiD / scEnrichment reference data ---------------------------------------------------
105
+
106
+ def test_cellid_reference_is_a_gzip_pickle():
107
+ head, size = first_bytes("toolkit/getRef", type="pkl")
108
+ assert head[:2] == b"\x1f\x8b", "getRef is not gzip data: %r" % head
109
+ assert size is None or size > 1_000_000
110
+
111
+
112
+ def test_cellid_deg_reference_is_a_gzip_pickle():
113
+ head, size = first_bytes("toolkit/getRefDeg", type="pkl")
114
+ assert head[:2] == b"\x1f\x8b", "getRefDeg is not gzip data: %r" % head
115
+
116
+
117
+ def test_gene_set_reference_is_available_as_plain_csv():
118
+ header, row = first_lines("toolkit/getGeneSet", 2, format="csv")
119
+ cols = [c.strip('"') for c in header.split(",")]
120
+ assert cols == ["logfc", "gene", "atlas", "name"], cols
121
+
122
+
123
+ # ---- gene_search ------------------------------------------------------------------------------
124
+
125
+ def test_gene_reference_expression_rows_have_the_shape_gene_search_reads():
126
+ r = requests.get(url("geneExp/getRefExp"), params={"gene": "CD4"}, timeout=TIMEOUT)
127
+ assert r.status_code == 200
128
+ rows = r.json()
129
+ assert rows, "no reference expression for CD4"
130
+ # entry[0] ';'-separated values, entry[1] cell type, entry[3] atlas / tissue
131
+ assert isinstance(rows[0][0], str) and ";" in rows[0][0]
132
+ assert isinstance(rows[0][1], str) and isinstance(rows[0][3], str)
133
+
134
+
135
+ # ---- housekeeping ---------------------------------------------------------------------------------
136
+
137
+ def test_server_reports_a_toolkit_version_and_api_url():
138
+ v = requests.get(url("getToolkitVersion"), timeout=TIMEOUT).json()
139
+ assert v.get("version"), v
140
+ u = requests.get(url("getToolkitUrl"), timeout=TIMEOUT).json()
141
+ assert u.get("url", "").startswith("http"), u
142
+
143
+
144
+ def _run_all():
145
+ tests = [(n, f) for n, f in sorted(globals().items()) if n.startswith("test_") and callable(f)]
146
+ print("DISCO toolkit contract test against %s\n" % BASE)
147
+ failed = 0
148
+ for name, fn in tests:
149
+ try:
150
+ fn()
151
+ print(" PASS %s" % name)
152
+ except Exception as error: # report every failure, do not stop at the first
153
+ failed += 1
154
+ print(" FAIL %s\n %s: %s" % (name, type(error).__name__, error))
155
+ print("\n%d/%d passed" % (len(tests) - failed, len(tests)))
156
+ return 1 if failed else 0
157
+
158
+
159
+ if __name__ == "__main__":
160
+ sys.exit(_run_all())
@@ -0,0 +1,90 @@
1
+ """Offline tests for choosing a DISCO server. No network, and no pandas/scanpy needed:
2
+ GlobalVariable is loaded straight from its file, so this runs on any Python >= 3.9.
3
+
4
+ python tests/test_settings.py or pytest tests/test_settings.py
5
+ """
6
+ import importlib.util
7
+ import os
8
+ import sys
9
+
10
+ HERE = os.path.dirname(os.path.abspath(__file__))
11
+ PATH = os.path.join(HERE, "..", "discotoolkit", "GlobalVariable.py")
12
+
13
+
14
+ def load(env=None):
15
+ """A fresh copy of GlobalVariable, as if the package had just been imported."""
16
+ saved = os.environ.get("DISCO_API_URL")
17
+ if env is None:
18
+ os.environ.pop("DISCO_API_URL", None)
19
+ else:
20
+ os.environ["DISCO_API_URL"] = env
21
+ try:
22
+ spec = importlib.util.spec_from_file_location("GlobalVariable_under_test", PATH)
23
+ module = importlib.util.module_from_spec(spec)
24
+ spec.loader.exec_module(module)
25
+ return module
26
+ finally:
27
+ if saved is None:
28
+ os.environ.pop("DISCO_API_URL", None)
29
+ else:
30
+ os.environ["DISCO_API_URL"] = saved
31
+
32
+
33
+ def test_default_server_is_disco_v1():
34
+ g = load()
35
+ assert g.get_server() == "https://disco.bii.a-star.edu.sg/disco_v3_api/"
36
+
37
+
38
+ def test_api_url_joins_paths_with_exactly_one_slash():
39
+ g = load()
40
+ assert g.api_url("toolkit/getSampleMetadata") == g.get_server() + "toolkit/getSampleMetadata"
41
+ assert g.api_url("/toolkit/getSampleMetadata") == g.get_server() + "toolkit/getSampleMetadata"
42
+
43
+
44
+ def test_named_presets_and_urls():
45
+ g = load()
46
+ assert g.set_server("v2") == "https://immunesinglecell.org/disco_v3_api/"
47
+ assert g.api_url("x") == "https://immunesinglecell.org/disco_v3_api/x" # takes effect at once
48
+ assert g.set_server("V1") == g.SERVERS["v1"] # preset names are case-insensitive
49
+ assert g.set_server("https://example.org/api") == "https://example.org/api/" # trailing slash added
50
+ assert g.set_server("http://localhost:8889/disco_v3_api///") == "http://localhost:8889/disco_v3_api/"
51
+
52
+
53
+ def test_bad_servers_are_rejected_without_changing_the_current_one():
54
+ g = load()
55
+ before = g.get_server()
56
+ for bad in ["", " ", "v3", "ftp://x", "disco.bii.a-star.edu.sg", None, 5]:
57
+ try:
58
+ g.set_server(bad)
59
+ except ValueError:
60
+ pass
61
+ else:
62
+ raise AssertionError("accepted %r" % (bad,))
63
+ assert g.get_server() == before
64
+
65
+
66
+ def test_environment_variable_sets_the_starting_server():
67
+ assert load("v2").get_server() == "https://immunesinglecell.org/disco_v3_api/"
68
+ assert load("https://example.org/disco_v3_api").get_server() == "https://example.org/disco_v3_api/"
69
+
70
+
71
+ def test_old_prefix_variable_still_readable():
72
+ g = load()
73
+ g.set_server("v2")
74
+ assert g.prefix_disco_url == "https://immunesinglecell.org/disco_v3_api/"
75
+
76
+
77
+ def _run_all():
78
+ tests = [(n, f) for n, f in sorted(globals().items()) if n.startswith("test_") and callable(f)]
79
+ failed = 0
80
+ for name, fn in tests:
81
+ try:
82
+ fn(); print(" PASS %s" % name)
83
+ except Exception as error:
84
+ failed += 1; print(" FAIL %s\n %s: %s" % (name, type(error).__name__, error))
85
+ print("\n%d/%d passed" % (len(tests) - failed, len(tests)))
86
+ return 1 if failed else 0
87
+
88
+
89
+ if __name__ == "__main__":
90
+ sys.exit(_run_all())
@@ -0,0 +1,41 @@
1
+ """The version must be the same in setup.py and discotoolkit/__init__.py (and in the release tag).
2
+
3
+ Pure text checks, so they need no dependencies:
4
+
5
+ python tests/test_version.py # setup.py == __init__.py
6
+ python tests/test_version.py v1.2.0 # ... and equals the tag
7
+ """
8
+ import os
9
+ import re
10
+ import sys
11
+
12
+ ROOT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..")
13
+
14
+
15
+ def find(path, pattern):
16
+ with open(os.path.join(ROOT, path)) as handle:
17
+ match = re.search(pattern, handle.read())
18
+ assert match, "no version found in %s" % path
19
+ return match.group(1)
20
+
21
+
22
+ def test_setup_and_init_agree():
23
+ setup_version = find("setup.py", r'version\s*=\s*"([^"]+)"')
24
+ init_version = find("discotoolkit/__init__.py", r'__version__\s*=\s*"([^"]+)"')
25
+ assert setup_version == init_version, "setup.py says %s, __init__.py says %s" % (setup_version, init_version)
26
+
27
+
28
+ def main():
29
+ test_setup_and_init_agree()
30
+ version = find("setup.py", r'version\s*=\s*"([^"]+)"')
31
+ if len(sys.argv) > 1:
32
+ tag = sys.argv[1]
33
+ assert tag == "v" + version, "tag %s does not match package version %s (expected v%s)" % (tag, version, version)
34
+ print("version %s OK" % version)
35
+
36
+
37
+ if __name__ == "__main__":
38
+ try:
39
+ main()
40
+ except AssertionError as error:
41
+ sys.exit("FAIL: %s" % error)
@@ -1 +0,0 @@
1
- include README.md
@@ -1,13 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: discotoolkit
3
- Version: 1.1.4
4
- Summary: DISCOtoolkit is an python package that allows users to access data and use the tools provided by the DISCO database.
5
- Home-page: http://www.immunesinglecell.org/
6
- Author: Li Mengwei, Rom Uddamvathanak
7
- Author-email: uddamvathanak_rom@immunol.a-star.edu.sg
8
- Requires-Python: >=3.8.16
9
-
10
- DISCOtoolkit is an python package that allows users to access data and use the tools provided by the DISCO database. It provides the following functions
11
- Filter and download DISCO data based on sample metadata and cell type information
12
- CELLiD: cell type annotation
13
- scEnrichment: geneset enrichment using DISCO DEGs
@@ -1,72 +0,0 @@
1
- <!--
2
- * @Descripttion:
3
- * @version:
4
- * @Author: Mengwei Li
5
- * @Date: 2023-04-16 21:20:42
6
- * @LastEditors: Mengwei Li
7
- * @LastEditTime: 2023-04-16 21:22:03
8
- -->
9
- [![Documentation Status](https://readthedocs.org/projects/discotoolkit-py/badge/?version=latest)](https://discotoolkit-py.readthedocs.io/en/latest/?badge=latest) [![Downloads](https://static.pepy.tech/personalized-badge/discotoolkit?period=total&units=international_system&left_color=black&right_color=orange&left_text=Downloads)](https://pepy.tech/project/discotoolkit) [![PyPI version](https://img.shields.io/pypi/v/discotoolkit)](https://pypi.org/project/discotoolkit)
10
-
11
- # DISCOtoolkit 1.1.4
12
-
13
- DISCOtoolkit is an python package that allows users to access data and use the tools provided by the [DISCO database](https://www.immunesinglecell.org/). Read the documentation [DISCOtoolkit](https://discotoolkit-py.readthedocs.io/en/latest/). It provides the following functions:
14
-
15
- - Filter and download DISCO data based on sample metadata and cell type information
16
- - CELLiD: cell type annotation
17
- - scEnrichment: geneset enrichment using DISCO DEGs
18
-
19
- Dependency Requirements:
20
-
21
- - Numpy >= 1.21.6
22
- - Pandas >= 1.4.2
23
- - Scanpy >= 1.9.3
24
- - Scipy >= 1.8.0
25
- - joblib >= 1.1.0
26
- - pandarallel >= 1.6.5
27
-
28
- ## Minimal installation:
29
-
30
- The DISCOtoolkit can be easily installed in the current Python environment using `pip`:
31
-
32
- ```
33
- pip install discotoolkit
34
- ```
35
-
36
- ## Installation guide:
37
-
38
- we recommend to install miniconda first and install discotoolkit in virtual env
39
-
40
- ```
41
- conda create --name disco python=3.8
42
- ```
43
- ```
44
- conda activate disco
45
- ```
46
- ```
47
- conda install ipykernel
48
- ```
49
- ```
50
- python -m ipykernel install --user --name disco --display-name "disco"
51
- ```
52
- ```
53
- python -m pip install discotoolkit # adding -U for installing the latest version
54
- ```
55
-
56
- ## Basic Usage
57
- Example in Jupyter notebook.
58
-
59
- <em>please select disco as the kernel for running the jupyter notebook</em>
60
-
61
- ### [Filter and download DISCO data](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/download_data.ipynb)
62
-
63
- ### [Cell Type Annotation using CELLiD](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/CELLiD_celltype_annotation.ipynb)
64
-
65
- ### [scEnrichment](https://github.com/JinmiaoChenLab/DISCOtoolkit_py/blob/main/docs/scEnrichment.ipynb)
66
-
67
- ## Citation
68
- 1. [Li, Mengwei, et al. "DISCO: a database of Deeply Integrated human Single-Cell Omics data." Nucleic acids research 50.D1 (2022): D596-D602.](https://academic.oup.com/nar/article/50/D1/D596/6430491)
69
-
70
- ## Follow us on our social media!
71
- - [HSCRM2](https://twitter.com/HSCRM2)
72
- - [JinmiaoChenLab Github repo](https://github.com/JinmiaoChenLab)
@@ -1,33 +0,0 @@
1
- '''
2
- Descripttion:
3
- version:
4
- Author: Mengwei Li
5
- Date: 2023-07-06 16:59:59
6
- LastEditors: Mengwei Li
7
- LastEditTime: 2024-10-23 15:52:29
8
- '''
9
- """
10
- Global variable file to import for the subsequent script.
11
-
12
- Main purpose is for the logging to be imported to other script and getting the url
13
- """
14
-
15
- import requests
16
- import json
17
- import logging
18
- # import random
19
-
20
- # setting the logging level
21
- # logger = logging.StreamHandler()
22
- # logger.setLevel(0)
23
- logging.basicConfig(level = logging.INFO)
24
-
25
- timeout = 600
26
-
27
- # Define package-level variable
28
- # response = requests.get("http://www.immunesinglecell.org/api/vishuo/getToolkitUrl")
29
-
30
- # if response.status_code == 200:
31
- # prefix_disco_url = json.loads(response.text)["url"]
32
- # else:
33
- prefix_disco_url = "https://immunesinglecell.org/disco_v3_api/"
@@ -1,13 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: discotoolkit
3
- Version: 1.1.4
4
- Summary: DISCOtoolkit is an python package that allows users to access data and use the tools provided by the DISCO database.
5
- Home-page: http://www.immunesinglecell.org/
6
- Author: Li Mengwei, Rom Uddamvathanak
7
- Author-email: uddamvathanak_rom@immunol.a-star.edu.sg
8
- Requires-Python: >=3.8.16
9
-
10
- DISCOtoolkit is an python package that allows users to access data and use the tools provided by the DISCO database. It provides the following functions
11
- Filter and download DISCO data based on sample metadata and cell type information
12
- CELLiD: cell type annotation
13
- scEnrichment: geneset enrichment using DISCO DEGs
File without changes