impc-api 1.0.6__tar.gz → 1.0.7__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,41 +1,43 @@
1
- Metadata-Version: 2.1
1
+ Metadata-Version: 2.4
2
2
  Name: impc_api
3
- Version: 1.0.6
3
+ Version: 1.0.7
4
4
  Summary: A package to facilitate making API requests to the IMPC Solr API
5
5
  Author: MPI2, Marina Kan, Diego Pava
6
- Project-URL: Homepage, https://github.com/mpi2/impc-data-api-workshop
6
+ Project-URL: Homepage, https://github.com/mpi2/impc-api
7
7
  Requires-Python: >=3.10
8
8
  Description-Content-Type: text/markdown
9
9
  License-File: LICENSE
10
+ Requires-Dist: ipykernel>=6.29.5
11
+ Requires-Dist: notebook>=7.2.2
10
12
  Requires-Dist: pandas>=2.2.0
11
13
  Requires-Dist: requests>=2.31.0
12
14
  Requires-Dist: tqdm>=4.66.4
13
15
  Requires-Dist: pydantic>=2.9
14
16
  Provides-Extra: dev
15
17
  Requires-Dist: pytest>=8.2.2; extra == "dev"
18
+ Dynamic: license-file
16
19
 
17
20
  # IMPC_API
18
21
 
19
- `impc_api` is a Python package which provides several helper functions that wrap around the IMPC SOLR API.
20
-
21
- The functions in this package are intended for use in a Jupyter Notebook.
22
+ `impc_api` is a Python package which provides several helper functions that wrap around the IMPC SOLR API. The functions in this package are intended for use in a Jupyter Notebook.
22
23
 
23
24
  ## Installation Instructions
24
25
 
26
+ 0. Ensure that Python is installed on your system. The minimum required version is 3.10.
27
+
25
28
  1. **Create a virtual environment (optional but recommended)**:
26
- On Mac:
29
+ On Mac or Linux:
27
30
  ```bash
28
31
  python3 -m venv .venv
29
32
  source .venv/bin/activate
30
33
  ```
31
34
 
32
35
  2. **Install the package**: `pip install impc_api`
33
- 3. **Install Jupyter**: `pip install jupyter`
34
- 4. **Run the Jupyter Notebook**: `jupyter notebook`
36
+ 3. **Run the Jupyter Notebook**: `jupyter notebook`
35
37
 
36
38
  After executing the command, the Jupyter interface should open in your browser. If it does not, follow the instructions provided in the terminal.
37
39
 
38
- 5. **Try it out**:
40
+ 4. **Try it out**:
39
41
 
40
42
  Create a [Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/latest/) and try some of the examples below:
41
43
 
@@ -96,7 +98,7 @@ num_found, df = solr_request(
96
98
  validate=True
97
99
  )
98
100
 
99
- > InvalidCoreWarning: Invalid core: "genotype-phenotyp", select from the available cores:
101
+ > InvalidCoreWarning: Invalid core: "invalid_core", select from the available cores:
100
102
  > dict_keys(['experiment', 'genotype-phenotype', 'impc_images', 'phenodigm', 'statistical-result'])
101
103
  ```
102
104
 
@@ -116,13 +118,33 @@ num_found, df = solr_request(
116
118
  > To see expected fields check the documentation at: https://www.ebi.ac.uk/mi/impc/solrdoc/
117
119
  ```
118
120
 
121
+ ## c. URL only
122
+
123
+ Users might want help producing the URL to fetch the data without the need of a DataFrame. Use the flag `url_only=True` to print or return the URL for your query.
124
+
125
+ ```python
126
+ url, _ = solr_request(
127
+ core='genotype-phenotype',
128
+ params={
129
+ 'q': '*:*',
130
+ 'rows': 10,
131
+ 'fl': 'marker_symbol,allele_symbol'
132
+ },
133
+ url_only=True
134
+ )
135
+ > "https://www.ebi.ac.uk/mi/impc/solr/genotype-phenotype/select?q=%2A%3A%2A&rows=10&fl=marker_symbol%2Callele_symbol"
136
+
137
+ print(url)
138
+ > "https://www.ebi.ac.uk/mi/impc/solr/genotype-phenotype/select?q=%2A%3A%2A&rows=10&fl=marker_symbol%2Callele_symbol"
139
+ ```
140
+
119
141
  # 2. Batch Solr Request
120
142
 
121
143
  `batch_solr_request` is available for large queries. This solves issues where a request is too large to fit into memory or where it puts a lot of strain on the API.
122
144
 
123
145
  Use `batch_solr_request` for:
124
146
 
125
- - Large queries (>1,000,000)
147
+ - Large queries (>100,000 rows)
126
148
  - Querying multiple items in a list
127
149
  - Downloading data in `json` or `csv` format.
128
150
 
@@ -1,41 +1,24 @@
1
- Metadata-Version: 2.1
2
- Name: impc_api
3
- Version: 1.0.6
4
- Summary: A package to facilitate making API requests to the IMPC Solr API
5
- Author: MPI2, Marina Kan, Diego Pava
6
- Project-URL: Homepage, https://github.com/mpi2/impc-data-api-workshop
7
- Requires-Python: >=3.10
8
- Description-Content-Type: text/markdown
9
- License-File: LICENSE
10
- Requires-Dist: pandas>=2.2.0
11
- Requires-Dist: requests>=2.31.0
12
- Requires-Dist: tqdm>=4.66.4
13
- Requires-Dist: pydantic>=2.9
14
- Provides-Extra: dev
15
- Requires-Dist: pytest>=8.2.2; extra == "dev"
16
-
17
1
  # IMPC_API
18
2
 
19
- `impc_api` is a Python package which provides several helper functions that wrap around the IMPC SOLR API.
20
-
21
- The functions in this package are intended for use in a Jupyter Notebook.
3
+ `impc_api` is a Python package which provides several helper functions that wrap around the IMPC SOLR API. The functions in this package are intended for use in a Jupyter Notebook.
22
4
 
23
5
  ## Installation Instructions
24
6
 
7
+ 0. Ensure that Python is installed on your system. The minimum required version is 3.10.
8
+
25
9
  1. **Create a virtual environment (optional but recommended)**:
26
- On Mac:
10
+ On Mac or Linux:
27
11
  ```bash
28
12
  python3 -m venv .venv
29
13
  source .venv/bin/activate
30
14
  ```
31
15
 
32
16
  2. **Install the package**: `pip install impc_api`
33
- 3. **Install Jupyter**: `pip install jupyter`
34
- 4. **Run the Jupyter Notebook**: `jupyter notebook`
17
+ 3. **Run the Jupyter Notebook**: `jupyter notebook`
35
18
 
36
19
  After executing the command, the Jupyter interface should open in your browser. If it does not, follow the instructions provided in the terminal.
37
20
 
38
- 5. **Try it out**:
21
+ 4. **Try it out**:
39
22
 
40
23
  Create a [Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/latest/) and try some of the examples below:
41
24
 
@@ -96,7 +79,7 @@ num_found, df = solr_request(
96
79
  validate=True
97
80
  )
98
81
 
99
- > InvalidCoreWarning: Invalid core: "genotype-phenotyp", select from the available cores:
82
+ > InvalidCoreWarning: Invalid core: "invalid_core", select from the available cores:
100
83
  > dict_keys(['experiment', 'genotype-phenotype', 'impc_images', 'phenodigm', 'statistical-result'])
101
84
  ```
102
85
 
@@ -116,13 +99,33 @@ num_found, df = solr_request(
116
99
  > To see expected fields check the documentation at: https://www.ebi.ac.uk/mi/impc/solrdoc/
117
100
  ```
118
101
 
102
+ ## c. URL only
103
+
104
+ Users might want help producing the URL to fetch the data without the need of a DataFrame. Use the flag `url_only=True` to print or return the URL for your query.
105
+
106
+ ```python
107
+ url, _ = solr_request(
108
+ core='genotype-phenotype',
109
+ params={
110
+ 'q': '*:*',
111
+ 'rows': 10,
112
+ 'fl': 'marker_symbol,allele_symbol'
113
+ },
114
+ url_only=True
115
+ )
116
+ > "https://www.ebi.ac.uk/mi/impc/solr/genotype-phenotype/select?q=%2A%3A%2A&rows=10&fl=marker_symbol%2Callele_symbol"
117
+
118
+ print(url)
119
+ > "https://www.ebi.ac.uk/mi/impc/solr/genotype-phenotype/select?q=%2A%3A%2A&rows=10&fl=marker_symbol%2Callele_symbol"
120
+ ```
121
+
119
122
  # 2. Batch Solr Request
120
123
 
121
124
  `batch_solr_request` is available for large queries. This solves issues where a request is too large to fit into memory or where it puts a lot of strain on the API.
122
125
 
123
126
  Use `batch_solr_request` for:
124
127
 
125
- - Large queries (>1,000,000)
128
+ - Large queries (>100,000 rows)
126
129
  - Querying multiple items in a list
127
130
  - Downloading data in `json` or `csv` format.
128
131
 
@@ -10,7 +10,7 @@ pd.set_option("display.max_rows", 15)
10
10
  pd.set_option("display.max_columns", None)
11
11
 
12
12
  # Create helper function
13
- def solr_request(core, params, silent=False, validate=False):
13
+ def solr_request(core, params, silent=False, validate=False, url_only=False):
14
14
  """Performs a single Solr request to the IMPC Solr API.
15
15
 
16
16
  Args:
@@ -22,11 +22,22 @@ def solr_request(core, params, silent=False, validate=False):
22
22
  validate (bool, optional): default False
23
23
  If True, validates the parameters against the core schema and raises warnings
24
24
  if any parameter seems invalid.
25
+ url_only (bool, optional): default False
26
+ If true, returns the request URL but no data.
25
27
 
26
28
 
27
29
  Returns:
28
- num_found: Number of docs found.
29
- pandas.DataFrame: Pandas.DataFrame object with the information requested.
30
+ Union[Tuple[int, pandas.DataFrame], Tuple[str, None]]:
31
+ - If url_only is False:
32
+ A tuple containing:
33
+ num_found (int): Number of documents found.
34
+ df (pandas.DataFrame): DataFrame object with the information requested.
35
+
36
+ - If url_only is True:
37
+ A tuple containing:
38
+ url (str): The URL of the request if status_code is 200, otherwise None.
39
+ None: Placeholder for consistency.
40
+
30
41
 
31
42
  Example query:
32
43
  num_found, df = solr_request(
@@ -70,6 +81,16 @@ def solr_request(core, params, silent=False, validate=False):
70
81
  solr_url = base_url + core + "/select"
71
82
 
72
83
  response = requests.get(solr_url, params=params)
84
+
85
+ if url_only:
86
+ if response.status_code == 200:
87
+ print(response.request.url)
88
+ return response.request.url, None
89
+ else:
90
+ print(f"Error: {response.status_code}\n{response.text}")
91
+ return None, None
92
+
93
+
73
94
  if not silent:
74
95
  print(f"\nYour request:\n{response.request.url}\n")
75
96
 
@@ -0,0 +1,463 @@
1
+ {
2
+ "experiment": [
3
+ "id",
4
+ "observation_id",
5
+ "specimen_id",
6
+ "phenotyping_center_id",
7
+ "phenotyping_center",
8
+ "production_center_id",
9
+ "production_center",
10
+ "specimen_project_id",
11
+ "specimen_project_name",
12
+ "gene_accession_id",
13
+ "gene_symbol",
14
+ "allele_accession_id",
15
+ "allele_symbol",
16
+ "zygosity",
17
+ "sex",
18
+ "biological_model_id",
19
+ "biological_sample_id",
20
+ "biological_sample_group",
21
+ "strain_accession_id",
22
+ "strain_name",
23
+ "genetic_background",
24
+ "allelic_composition",
25
+ "colony_id",
26
+ "litter_id",
27
+ "date_of_birth",
28
+ "external_sample_id",
29
+ "life_stage_name",
30
+ "life_stage_acc",
31
+ "datasource_id",
32
+ "datasource_name",
33
+ "project_id",
34
+ "project_name",
35
+ "pipeline_id",
36
+ "pipeline_name",
37
+ "pipeline_stable_id",
38
+ "procedure_id",
39
+ "procedure_name",
40
+ "procedure_stable_id",
41
+ "procedure_group",
42
+ "parameter_id",
43
+ "parameter_name",
44
+ "parameter_stable_id",
45
+ "procedure_sequence_id",
46
+ "experiment_id",
47
+ "observation_type",
48
+ "data_type",
49
+ "experiment_source_id",
50
+ "date_of_experiment",
51
+ "weight_parameter_stable_id",
52
+ "weight_date",
53
+ "weight_days_old",
54
+ "weight",
55
+ "data_point",
56
+ "order_index",
57
+ "dimension",
58
+ "time_point",
59
+ "discrete_point",
60
+ "category",
61
+ "raw_category",
62
+ "metadata",
63
+ "metadata_group",
64
+ "anatomy_id",
65
+ "anatomy_term",
66
+ "anatomy_id_term",
67
+ "anatomy_term_synonym",
68
+ "top_level_anatomy_id",
69
+ "top_level_anatomy_term",
70
+ "top_level_anatomy_term_synonym",
71
+ "selected_top_level_anatomy_id",
72
+ "selected_top_level_anatomy_term",
73
+ "selected_top_level_anatomy_term_synonym",
74
+ "intermediate_anatomy_id",
75
+ "intermediate_anatomy_term",
76
+ "intermediate_anatomy_term_synonym",
77
+ "parent_anatomy_id",
78
+ "parent_anatomy_term",
79
+ "parent_anatomy_term_synonym",
80
+ "child_anatomy_id",
81
+ "child_anatomy_term",
82
+ "child_anatomy_term_synonym",
83
+ "download_file_path",
84
+ "image_link",
85
+ "file_type",
86
+ "increment_value",
87
+ "parameter_association_stable_id",
88
+ "parameter_association_sequence_id",
89
+ "parameter_association_dim_id",
90
+ "parameter_association_name",
91
+ "parameter_association_value",
92
+ "developmental_stage_acc",
93
+ "developmental_stage_name",
94
+ "text_value",
95
+ "sub_term_id",
96
+ "sub_term_name",
97
+ "sub_term_description",
98
+ "age_in_days",
99
+ "age_in_weeks"
100
+ ],
101
+ "genotype-phenotype": [
102
+ "doc_id",
103
+ "ontology_db_id",
104
+ "assertion_type",
105
+ "assertion_type_id",
106
+ "mpath_term_id",
107
+ "mpath_term_name",
108
+ "anatomy_term_id",
109
+ "anatomy_term_name",
110
+ "intermediate_anatomy_term_id",
111
+ "intermediate_anatomy_term_name",
112
+ "top_level_anatomy_term_id",
113
+ "top_level_anatomy_term_name",
114
+ "mp_term_id",
115
+ "mp_term_name",
116
+ "alt_mp_term_id",
117
+ "top_level_mp_term_id",
118
+ "top_level_mp_term_name",
119
+ "intermediate_mp_term_id",
120
+ "intermediate_mp_term_name",
121
+ "marker_symbol",
122
+ "marker_accession_id",
123
+ "colony_id",
124
+ "allele_name",
125
+ "allele_symbol",
126
+ "allele_accession_id",
127
+ "strain_name",
128
+ "strain_accession_id",
129
+ "phenotyping_center",
130
+ "project_external_id",
131
+ "project_name",
132
+ "project_fullname",
133
+ "resource_name",
134
+ "resource_fullname",
135
+ "sex",
136
+ "zygosity",
137
+ "pipeline_name",
138
+ "pipeline_stable_id",
139
+ "pipeline_stable_key",
140
+ "procedure_name",
141
+ "procedure_stable_id",
142
+ "procedure_stable_key",
143
+ "parameter_name",
144
+ "parameter_stable_id",
145
+ "parameter_stable_key",
146
+ "statistical_method",
147
+ "percentage_change",
148
+ "p_value",
149
+ "effect_size",
150
+ "external_id",
151
+ "life_stage_acc",
152
+ "life_stage_name"
153
+ ],
154
+ "impc_images": [
155
+ "id",
156
+ "observation_id",
157
+ "specimen_id",
158
+ "phenotyping_center_id",
159
+ "phenotyping_center",
160
+ "production_center_id",
161
+ "production_center",
162
+ "specimen_project_id",
163
+ "specimen_project_name",
164
+ "gene_accession_id",
165
+ "gene_symbol",
166
+ "allele_accession_id",
167
+ "allele_symbol",
168
+ "zygosity",
169
+ "sex",
170
+ "biological_model_id",
171
+ "biological_sample_id",
172
+ "biological_sample_group",
173
+ "strain_accession_id",
174
+ "strain_name",
175
+ "genetic_background",
176
+ "allelic_composition",
177
+ "colony_id",
178
+ "litter_id",
179
+ "date_of_birth",
180
+ "external_sample_id",
181
+ "life_stage_name",
182
+ "life_stage_acc",
183
+ "datasource_id",
184
+ "datasource_name",
185
+ "project_id",
186
+ "project_name",
187
+ "pipeline_id",
188
+ "pipeline_name",
189
+ "pipeline_stable_id",
190
+ "procedure_id",
191
+ "procedure_name",
192
+ "procedure_stable_id",
193
+ "procedure_group",
194
+ "parameter_id",
195
+ "parameter_name",
196
+ "parameter_stable_id",
197
+ "procedure_sequence_id",
198
+ "experiment_id",
199
+ "observation_type",
200
+ "data_type",
201
+ "experiment_source_id",
202
+ "date_of_experiment",
203
+ "weight_parameter_stable_id",
204
+ "weight_date",
205
+ "weight_days_old",
206
+ "weight",
207
+ "data_point",
208
+ "order_index",
209
+ "dimension",
210
+ "time_point",
211
+ "discrete_point",
212
+ "category",
213
+ "raw_category",
214
+ "metadata",
215
+ "metadata_group",
216
+ "mp_id",
217
+ "mp_term",
218
+ "top_level_mp_id",
219
+ "top_level_mp_term",
220
+ "intermediate_mp_id",
221
+ "intermediate_mp_term",
222
+ "anatomy_id",
223
+ "anatomy_term",
224
+ "anatomy_id_term",
225
+ "anatomy_term_synonym",
226
+ "top_level_anatomy_id",
227
+ "top_level_anatomy_term",
228
+ "top_level_anatomy_term_synonym",
229
+ "selected_top_level_anatomy_id",
230
+ "selected_top_level_anatomy_term",
231
+ "selected_top_level_anatomy_term_synonym",
232
+ "intermediate_anatomy_id",
233
+ "intermediate_anatomy_term",
234
+ "intermediate_anatomy_term_synonym",
235
+ "parent_anatomy_id",
236
+ "parent_anatomy_term",
237
+ "parent_anatomy_term_synonym",
238
+ "child_anatomy_id",
239
+ "child_anatomy_term",
240
+ "child_anatomy_term_synonym",
241
+ "download_file_path",
242
+ "image_link",
243
+ "file_type",
244
+ "parameter_association_stable_id",
245
+ "parameter_association_sequence_id",
246
+ "parameter_association_dim_id",
247
+ "parameter_association_name",
248
+ "parameter_association_value",
249
+ "developmental_stage_acc",
250
+ "developmental_stage_name",
251
+ "text_value",
252
+ "sub_term_id",
253
+ "sub_term_name",
254
+ "sub_term_description",
255
+ "sequence_id",
256
+ "age_in_days",
257
+ "age_in_weeks",
258
+ "download_url",
259
+ "jpeg_url",
260
+ "thumbnail_url",
261
+ "omero_id"
262
+ ],
263
+ "phenodigm": [
264
+ "type",
265
+ "disease_id",
266
+ "disease_source",
267
+ "disease_term",
268
+ "disease_alts",
269
+ "disease_locus",
270
+ "disease_classes",
271
+ "disease_phenotypes",
272
+ "gene_id",
273
+ "gene_symbol",
274
+ "gene_symbols_withdrawn",
275
+ "gene_locus",
276
+ "hgnc_gene_id",
277
+ "hgnc_gene_symbol",
278
+ "hgnc_gene_symbols_withdrawn",
279
+ "hgnc_gene_locus",
280
+ "mouse_model",
281
+ "impc_model",
282
+ "model_id",
283
+ "model_source",
284
+ "model_description",
285
+ "model_genetic_background",
286
+ "marker_id",
287
+ "marker_symbol",
288
+ "marker_locus",
289
+ "marker_num_models",
290
+ "model_phenotypes",
291
+ "ontology",
292
+ "phenotype_id",
293
+ "phenotype_term",
294
+ "phenotype_synonym",
295
+ "hp_id",
296
+ "hp_term",
297
+ "mp_id",
298
+ "mp_term",
299
+ "association_curated",
300
+ "association_ortholog",
301
+ "marker_symbols_withdrawn",
302
+ "disease_matched_phenotypes",
303
+ "model_matched_phenotypes",
304
+ "disease_model_avg_raw",
305
+ "disease_model_avg_norm",
306
+ "disease_model_max_raw",
307
+ "disease_model_max_norm",
308
+ "search_qf",
309
+ "human_curated_gene",
310
+ "impc_model_with_curated_gene",
311
+ "mgi_model_with_curated_gene",
312
+ "impc_model_with_computed_association",
313
+ "mgi_model_with_computed_association"
314
+ ],
315
+ "statistical-result": [
316
+ "additional_information",
317
+ "allele_accession_id",
318
+ "allele_name",
319
+ "allele_symbol",
320
+ "anatomy_term_id",
321
+ "anatomy_term_name",
322
+ "batch_significant",
323
+ "blups_test",
324
+ "categorical_effect_size",
325
+ "categorical_p_value",
326
+ "categories",
327
+ "classification_tag",
328
+ "colony_id",
329
+ "control_biological_model_id",
330
+ "control_selection_method",
331
+ "data_frame",
332
+ "data_type",
333
+ "db_id",
334
+ "dependent_variable",
335
+ "doc_id",
336
+ "effect_size",
337
+ "external_db_id",
338
+ "female_control_count",
339
+ "female_control_mean",
340
+ "female_effect_size",
341
+ "female_effect_size_low_normal_vs_high",
342
+ "female_effect_size_low_vs_normal_high",
343
+ "female_intermediate_mp_term_id",
344
+ "female_intermediate_mp_term_name",
345
+ "female_ko_effect_p_value",
346
+ "female_ko_effect_stderr_estimate",
347
+ "female_ko_parameter_estimate",
348
+ "female_mp_term_id",
349
+ "female_mp_term_name",
350
+ "female_mutant_count",
351
+ "female_mutant_mean",
352
+ "female_percentage_change",
353
+ "female_pvalue_low_normal_vs_high",
354
+ "female_pvalue_low_vs_normal_high",
355
+ "female_top_level_mp_term_id",
356
+ "female_top_level_mp_term_name",
357
+ "genetic_background",
358
+ "genotype_effect_p_value",
359
+ "genotype_effect_parameter_estimate",
360
+ "genotype_effect_size_low_normal_vs_high",
361
+ "genotype_effect_size_low_vs_normal_high",
362
+ "genotype_effect_stderr_estimate",
363
+ "genotype_pvalue_low_normal_vs_high",
364
+ "genotype_pvalue_low_vs_normal_high",
365
+ "group_1_genotype",
366
+ "group_1_residuals_normality_test",
367
+ "group_2_genotype",
368
+ "group_2_residuals_normality_test",
369
+ "id",
370
+ "interaction_effect_p_value",
371
+ "interaction_significant",
372
+ "intercept_estimate",
373
+ "intercept_estimate_stderr_estimate",
374
+ "intermediate_anatomy_term_id",
375
+ "intermediate_anatomy_term_name",
376
+ "intermediate_mp_term_id",
377
+ "intermediate_mp_term_name",
378
+ "life_stage_acc",
379
+ "life_stage_name",
380
+ "male_control_count",
381
+ "male_control_mean",
382
+ "male_effect_size",
383
+ "male_effect_size_low_normal_vs_high",
384
+ "male_effect_size_low_vs_normal_high",
385
+ "male_intermediate_mp_term_id",
386
+ "male_intermediate_mp_term_name",
387
+ "male_ko_effect_p_value",
388
+ "male_ko_effect_stderr_estimate",
389
+ "male_ko_parameter_estimate",
390
+ "male_mp_term_id",
391
+ "male_mp_term_name",
392
+ "male_mutant_count",
393
+ "male_mutant_mean",
394
+ "male_percentage_change",
395
+ "male_pvalue_low_normal_vs_high",
396
+ "male_pvalue_low_vs_normal_high",
397
+ "male_top_level_mp_term_id",
398
+ "male_top_level_mp_term_name",
399
+ "marker_accession_id",
400
+ "marker_symbol",
401
+ "metadata",
402
+ "metadata_group",
403
+ "mp_term_id",
404
+ "mp_term_id_options",
405
+ "mp_term_name",
406
+ "mp_term_name_options",
407
+ "mutant_biological_model_id",
408
+ "null_test_p_value",
409
+ "organisation_id",
410
+ "p_value",
411
+ "parameter_id",
412
+ "parameter_name",
413
+ "parameter_stable_id",
414
+ "parameter_stable_key",
415
+ "phenotype_sex",
416
+ "phenotyping_center",
417
+ "phenotyping_center_id",
418
+ "pipeline_id",
419
+ "pipeline_name",
420
+ "pipeline_stable_id",
421
+ "pipeline_stable_key",
422
+ "procedure_id",
423
+ "procedure_name",
424
+ "procedure_stable_id",
425
+ "procedure_stable_key",
426
+ "production_center",
427
+ "project_id",
428
+ "project_name",
429
+ "raw_output",
430
+ "resource_fullname",
431
+ "resource_id",
432
+ "resource_name",
433
+ "rotated_residuals_test",
434
+ "sex",
435
+ "sex_effect_p_value",
436
+ "sex_effect_parameter_estimate",
437
+ "sex_effect_stderr_estimate",
438
+ "significant",
439
+ "soft_windowing_bandwidth",
440
+ "soft_windowing_doe_note",
441
+ "soft_windowing_min_obs_required",
442
+ "soft_windowing_number_of_doe",
443
+ "soft_windowing_peaks",
444
+ "soft_windowing_shape",
445
+ "soft_windowing_threshold",
446
+ "soft_windowing_total_obs_or_weight",
447
+ "statistical_method",
448
+ "status",
449
+ "strain_accession_id",
450
+ "strain_name",
451
+ "text",
452
+ "top_level_anatomy_term_id",
453
+ "top_level_anatomy_term_name",
454
+ "top_level_mp_term_id",
455
+ "top_level_mp_term_name",
456
+ "variance_significant",
457
+ "weight_effect_p_value",
458
+ "weight_effect_parameter_estimate",
459
+ "weight_effect_stderr_estimate",
460
+ "workflow",
461
+ "zygosity"
462
+ ]
463
+ }
@@ -93,7 +93,7 @@ class CoreParamsValidator(BaseModel):
93
93
  # Validate each field in params
94
94
  if invalid_core is not True:
95
95
  for fl in field_list:
96
- if fl not in jv.valid_fields(core):
96
+ if fl.strip() not in jv.valid_fields(core):
97
97
  warnings.warn(
98
98
  message=f"""Unexpected field name: "{fl}". Check the spelling of fields.\nTo see expected fields check the documentation at: https://www.ebi.ac.uk/mi/impc/solrdoc/""",
99
99
  category=InvalidFieldWarning,
@@ -1,25 +1,43 @@
1
+ Metadata-Version: 2.4
2
+ Name: impc_api
3
+ Version: 1.0.7
4
+ Summary: A package to facilitate making API requests to the IMPC Solr API
5
+ Author: MPI2, Marina Kan, Diego Pava
6
+ Project-URL: Homepage, https://github.com/mpi2/impc-api
7
+ Requires-Python: >=3.10
8
+ Description-Content-Type: text/markdown
9
+ License-File: LICENSE
10
+ Requires-Dist: ipykernel>=6.29.5
11
+ Requires-Dist: notebook>=7.2.2
12
+ Requires-Dist: pandas>=2.2.0
13
+ Requires-Dist: requests>=2.31.0
14
+ Requires-Dist: tqdm>=4.66.4
15
+ Requires-Dist: pydantic>=2.9
16
+ Provides-Extra: dev
17
+ Requires-Dist: pytest>=8.2.2; extra == "dev"
18
+ Dynamic: license-file
19
+
1
20
  # IMPC_API
2
21
 
3
- `impc_api` is a Python package which provides several helper functions that wrap around the IMPC SOLR API.
4
-
5
- The functions in this package are intended for use in a Jupyter Notebook.
22
+ `impc_api` is a Python package which provides several helper functions that wrap around the IMPC SOLR API. The functions in this package are intended for use in a Jupyter Notebook.
6
23
 
7
24
  ## Installation Instructions
8
25
 
26
+ 0. Ensure that Python is installed on your system. The minimum required version is 3.10.
27
+
9
28
  1. **Create a virtual environment (optional but recommended)**:
10
- On Mac:
29
+ On Mac or Linux:
11
30
  ```bash
12
31
  python3 -m venv .venv
13
32
  source .venv/bin/activate
14
33
  ```
15
34
 
16
35
  2. **Install the package**: `pip install impc_api`
17
- 3. **Install Jupyter**: `pip install jupyter`
18
- 4. **Run the Jupyter Notebook**: `jupyter notebook`
36
+ 3. **Run the Jupyter Notebook**: `jupyter notebook`
19
37
 
20
38
  After executing the command, the Jupyter interface should open in your browser. If it does not, follow the instructions provided in the terminal.
21
39
 
22
- 5. **Try it out**:
40
+ 4. **Try it out**:
23
41
 
24
42
  Create a [Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/latest/) and try some of the examples below:
25
43
 
@@ -80,7 +98,7 @@ num_found, df = solr_request(
80
98
  validate=True
81
99
  )
82
100
 
83
- > InvalidCoreWarning: Invalid core: "genotype-phenotyp", select from the available cores:
101
+ > InvalidCoreWarning: Invalid core: "invalid_core", select from the available cores:
84
102
  > dict_keys(['experiment', 'genotype-phenotype', 'impc_images', 'phenodigm', 'statistical-result'])
85
103
  ```
86
104
 
@@ -100,13 +118,33 @@ num_found, df = solr_request(
100
118
  > To see expected fields check the documentation at: https://www.ebi.ac.uk/mi/impc/solrdoc/
101
119
  ```
102
120
 
121
+ ## c. URL only
122
+
123
+ Users might want help producing the URL to fetch the data without the need of a DataFrame. Use the flag `url_only=True` to print or return the URL for your query.
124
+
125
+ ```python
126
+ url, _ = solr_request(
127
+ core='genotype-phenotype',
128
+ params={
129
+ 'q': '*:*',
130
+ 'rows': 10,
131
+ 'fl': 'marker_symbol,allele_symbol'
132
+ },
133
+ url_only=True
134
+ )
135
+ > "https://www.ebi.ac.uk/mi/impc/solr/genotype-phenotype/select?q=%2A%3A%2A&rows=10&fl=marker_symbol%2Callele_symbol"
136
+
137
+ print(url)
138
+ > "https://www.ebi.ac.uk/mi/impc/solr/genotype-phenotype/select?q=%2A%3A%2A&rows=10&fl=marker_symbol%2Callele_symbol"
139
+ ```
140
+
103
141
  # 2. Batch Solr Request
104
142
 
105
143
  `batch_solr_request` is available for large queries. This solves issues where a request is too large to fit into memory or where it puts a lot of strain on the API.
106
144
 
107
145
  Use `batch_solr_request` for:
108
146
 
109
- - Large queries (>1,000,000)
147
+ - Large queries (>100,000 rows)
110
148
  - Querying multiple items in a list
111
149
  - Downloading data in `json` or `csv` format.
112
150
 
@@ -1,3 +1,5 @@
1
+ ipykernel>=6.29.5
2
+ notebook>=7.2.2
1
3
  pandas>=2.2.0
2
4
  requests>=2.31.0
3
5
  tqdm>=4.66.4
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "impc_api"
7
- version = "1.0.6"
7
+ version = "1.0.7"
8
8
  description = "A package to facilitate making API requests to the IMPC Solr API"
9
9
  authors = [
10
10
  { name = "MPI2" },
@@ -12,6 +12,8 @@ authors = [
12
12
  { name = "Diego Pava" }
13
13
  ]
14
14
  dependencies = [
15
+ "ipykernel>=6.29.5",
16
+ "notebook>=7.2.2",
15
17
  "pandas>=2.2.0",
16
18
  "requests>=2.31.0",
17
19
  "tqdm>=4.66.4",
@@ -30,5 +32,5 @@ dev = [
30
32
  include = ["impc_api", "impc_api.*"]
31
33
 
32
34
  [project.urls]
33
- "Homepage" = "https://github.com/mpi2/impc-data-api-workshop"
35
+ "Homepage" = "https://github.com/mpi2/impc-api"
34
36
 
@@ -4,6 +4,7 @@ from impc_api.utils.warnings import InvalidCoreWarning, InvalidFieldWarning
4
4
  from solr_request import solr_request, _process_faceting
5
5
  from .test_helpers import check_url_status_code_and_params
6
6
 
7
+
7
8
  class TestSolrRequest:
8
9
  """Test class for the Solr Request function
9
10
 
@@ -316,3 +317,93 @@ class TestSolrRequest:
316
317
  params={"q": "*:*", "fl": "invalid_field,another_invalid_field"},
317
318
  validate=True,
318
319
  )
320
+
321
+ @pytest.mark.parametrize(
322
+ "mock_response", [_validation_response()], indirect=["mock_response"]
323
+ )
324
+ def test_solr_request_fields_validation_spacing(self, common_params, mock_response):
325
+ try:
326
+ _ = solr_request(
327
+ core="experiment",
328
+ params={
329
+ **common_params,
330
+ "fl": "experiment_id, parameter_stable_id , parameter_name",
331
+ },
332
+ validate=True,
333
+ )
334
+ except InvalidFieldWarning:
335
+ pytest.fail("InvalidFieldWarning raised when it shouldn't have been")
336
+
337
+
338
+ @pytest.fixture
339
+ def test_url(self):
340
+ return "http://test.url/url_only"
341
+
342
+ @pytest.fixture
343
+ def mock_response_url(self, request, test_url):
344
+ """
345
+ Fixture to mock the response from the Solr API
346
+
347
+ Args:
348
+ request (FixtureRequest): Request object provided by pytest containing the parameterized data
349
+
350
+ Yields:
351
+ MagicMock: A mock object "representing" the requests.get function used in solr_request.py.
352
+ Its returned value is configured according to the parameterized data.
353
+ """
354
+ with patch("solr_request.requests.get") as mock_get:
355
+ mock_response = mock_get.return_value
356
+ mock_response.status_code = request.param.get("status_code")
357
+ mock_response.request.url = test_url
358
+ yield mock_get
359
+
360
+ @pytest.mark.parametrize(
361
+ "mock_response_url",
362
+ [
363
+ (
364
+ {
365
+ "status_code": 200,
366
+ }
367
+ )
368
+ ],
369
+ indirect=True
370
+ )
371
+ def test_solr_request_url_only_success(self, core, common_params, capsys, mock_response_url, test_url):
372
+ # Assert when the params are status_code 200, the URL is printed and returned
373
+ url, _ = solr_request(
374
+ core=core,
375
+ params=common_params,
376
+ url_only=True
377
+ )
378
+ # Assert url is returned
379
+ assert url == test_url, f"Expected URL {test_url}, got {url}"
380
+ captured = capsys.readouterr()
381
+
382
+ # Assert test url is printed to console
383
+ assert test_url in captured.out, "Expected test URL to be printed on console output"
384
+
385
+ @pytest.mark.parametrize(
386
+ "mock_response_url",
387
+ [
388
+ (
389
+ {
390
+ "status_code": 404,
391
+ }
392
+ )
393
+ ],
394
+ indirect=True
395
+ )
396
+ def test_solr_request_url_only_fail(self, core, common_params, capsys, mock_response_url, test_url):
397
+ # Assert that when status code is not 200, it returns None and the error message
398
+ url, _ = solr_request(
399
+ core=core,
400
+ params=common_params,
401
+ url_only=True
402
+ )
403
+ # Assert url is returned
404
+ assert url is None, "Expected no URL to be returned"
405
+ captured = capsys.readouterr()
406
+
407
+ # Assert test url is printed to console
408
+ assert "Error" in captured.out, "Expected error message to be printed in console output"
409
+
@@ -1,15 +0,0 @@
1
- {
2
- "experiment": [
3
- "id", "observation_id", "specimen_id", "phenotyping_center_id", "phenotyping_center", "production_center_id", "production_center", "specimen_project_id", "specimen_project_name", "gene_accession_id", "gene_symbol", "allele_accession_id", "allele_symbol", "zygosity", "sex", "biological_model_id", "biological_sample_id", "biological_sample_group", "strain_accession_id", "strain_name", "genetic_background", "allelic_composition", "colony_id", "litter_id", "date_of_birth", "external_sample_id", "life_stage_name", "life_stage_acc", "datasource_id", "datasource_name", "project_id", "project_name", "pipeline_id", "pipeline_name", "pipeline_stable_id", "procedure_id", "procedure_name", "procedure_stable_id", "procedure_group", "parameter_id", "parameter_name", "parameter_stable_id", "procedure_sequence_id", "experiment_id", "observation_type", "data_type", "experiment_source_id", "date_of_experiment", "weight_parameter_stable_id", "weight_date", "weight_days_old", "weight", "data_point", "order_index", "dimension", "time_point", "discrete_point", "category", "raw_category", "metadata", "metadata_group", "anatomy_id", "anatomy_term", "anatomy_id_term", "anatomy_term_synonym", "top_level_anatomy_id", "top_level_anatomy_term", "top_level_anatomy_term_synonym", "selected_top_level_anatomy_id", "selected_top_level_anatomy_term", "selected_top_level_anatomy_term_synonym", "intermediate_anatomy_id", "intermediate_anatomy_term", "intermediate_anatomy_term_synonym", "parent_anatomy_id", "parent_anatomy_term", "parent_anatomy_term_synonym", "child_anatomy_id", "child_anatomy_term", "child_anatomy_term_synonym", "download_file_path", "image_link", "file_type", "increment_value", "parameter_association_stable_id", "parameter_association_sequence_id", "parameter_association_dim_id", "parameter_association_name", "parameter_association_value", "developmental_stage_acc", "developmental_stage_name", "text_value", "sub_term_id", "sub_term_name", "sub_term_description", "age_in_days", "age_in_weeks"
4
- ],
5
- "genotype-phenotype": [
6
- "doc_id", "ontology_db_id", "assertion_type", "assertion_type_id", "mpath_term_id", "mpath_term_name", "anatomy_term_id", "anatomy_term_name", "intermediate_anatomy_term_id", "intermediate_anatomy_term_name", "top_level_anatomy_term_id", "top_level_anatomy_term_name", "mp_term_id", "mp_term_name", "alt_mp_term_id", "top_level_mp_term_id", "top_level_mp_term_name", "intermediate_mp_term_id", "intermediate_mp_term_name", "marker_symbol", "marker_accession_id", "colony_id", "allele_name", "allele_symbol", "allele_accession_id", "strain_name", "strain_accession_id", "phenotyping_center", "project_external_id", "project_name", "project_fullname", "resource_name", "resource_fullname", "sex", "zygosity", "pipeline_name", "pipeline_stable_id", "pipeline_stable_key", "procedure_name", "procedure_stable_id", "procedure_stable_key", "parameter_name", "parameter_stable_id", "parameter_stable_key", "statistical_method", "percentage_change", "p_value", "effect_size", "external_id", "life_stage_acc", "life_stage_name"
7
- ],
8
- "impc_images": [
9
- "id", "observation_id", "specimen_id", "phenotyping_center_id", "phenotyping_center", "production_center_id", "production_center", "specimen_project_id", "specimen_project_name", "gene_accession_id", "gene_symbol", "allele_accession_id", "allele_symbol", "zygosity", "sex", "biological_model_id", "biological_sample_id", "biological_sample_group", "strain_accession_id", "strain_name", "genetic_background", "allelic_composition", "colony_id", "litter_id", "date_of_birth", "external_sample_id", "life_stage_name", "life_stage_acc", "datasource_id", "datasource_name", "project_id", "project_name", "pipeline_id", "pipeline_name", "pipeline_stable_id", "procedure_id", "procedure_name", "procedure_stable_id", "procedure_group", "parameter_id", "parameter_name", "parameter_stable_id", "procedure_sequence_id", "experiment_id", "observation_type", "data_type", "experiment_source_id", "date_of_experiment", "weight_parameter_stable_id", "weight_date", "weight_days_old", "weight", "data_point", "order_index", "dimension", "time_point", "discrete_point", "category", "raw_category", "metadata", "metadata_group", "mp_id", "mp_term", "top_level_mp_id", "top_level_mp_term", "intermediate_mp_id", "intermediate_mp_term", "anatomy_id", "anatomy_term", "anatomy_id_term", "anatomy_term_synonym", "top_level_anatomy_id", "top_level_anatomy_term", "top_level_anatomy_term_synonym", "selected_top_level_anatomy_id", "selected_top_level_anatomy_term", "selected_top_level_anatomy_term_synonym", "intermediate_anatomy_id", "intermediate_anatomy_term", "intermediate_anatomy_term_synonym", "parent_anatomy_id", "parent_anatomy_term", "parent_anatomy_term_synonym", "child_anatomy_id", "child_anatomy_term", "child_anatomy_term_synonym", "download_file_path", "image_link", "file_type", "parameter_association_stable_id", "parameter_association_sequence_id", "parameter_association_dim_id", "parameter_association_name", "parameter_association_value", "developmental_stage_acc", "developmental_stage_name", "text_value", "sub_term_id", "sub_term_name", "sub_term_description", "sequence_id", "age_in_days", "age_in_weeks", "download_url", "jpeg_url", "thumbnail_url", "omero_id"
10
- ],
11
- "phenodigm": [
12
- "type", "disease_id", "disease_source", "disease_term", "disease_alts", "disease_locus", "disease_classes", "disease_phenotypes", "gene_id", "gene_symbol", "gene_symbols_withdrawn", "gene_locus", "hgnc_gene_id", "hgnc_gene_symbol", "hgnc_gene_symbols_withdrawn", "hgnc_gene_locus", "mouse_model", "impc_model", "model_id", "model_source", "model_description", "model_genetic_background", "marker_id", "marker_symbol", "marker_locus", "marker_num_models", "model_phenotypes", "ontology", "phenotype_id", "phenotype_term", "phenotype_synonym", "hp_id", "hp_term", "mp_id", "mp_term", "association_curated", "association_ortholog", "marker_symbols_withdrawn", "disease_matched_phenotypes", "model_matched_phenotypes", "disease_model_avg_raw", "disease_model_avg_norm", "disease_model_max_raw", "disease_model_max_norm", "search_qf", "human_curated_gene", "impc_model_with_curated_gene", "mgi_model_with_curated_gene", "impc_model_with_computed_association", "mgi_model_with_computed_association"
13
- ],
14
- "statistical-result": ["doc_id", "db_id", "data_type", "anatomy_term_id", "anatomy_term_name", "intermediate_anatomy_term_id", "intermediate_anatomy_term_name", "top_level_anatomy_term_id", "top_level_anatomy_term_name", "mp_term_id_options", "mp_term_name_options", "mp_term_id", "mp_term_name", "top_level_mp_term_id", "top_level_mp_term_name", "intermediate_mp_term_id", "intermediate_mp_term_name", "male_mp_term_id", "male_mp_term_name", "male_top_level_mp_term_id", "male_top_level_mp_term_name", "male_intermediate_mp_term_id", "male_intermediate_mp_term_name", "female_mp_term_id", "female_mp_term_name", "female_top_level_mp_term_id", "female_top_level_mp_term_name", "female_intermediate_mp_term_id", "female_intermediate_mp_term_name", "resource_name", "resource_fullname", "resource_id", "project_name", "phenotyping_center", "pipeline_stable_id", "pipeline_stable_key", "pipeline_name", "pipeline_id", "procedure_stable_id", "procedure_stable_key", "procedure_name", "procedure_id", "parameter_stable_id", "parameter_stable_key", "parameter_name", "parameter_id", "colony_id", "marker_symbol", "marker_accession_id", "allele_symbol", "allele_name", "allele_accession_id", "strain_name", "strain_accession_id", "sex", "zygosity", "control_selection_method", "dependent_variable", "metadata_group", "data_frame", "genetic_background", "production_center", "external_db_id", "id", "organisation_id", "phenotyping_center_id", "project_id", "male_control_mean", "male_mutant_mean", "female_control_mean", "female_mutant_mean", "genotype_p_value_low_vs_normal_high", "genotype_p_value_low_normal_vs_high", "genotype_effect_size_low_vs_normal_high", "genotype_effect_size_low_normal_vs_high", "female_p_value_low_vs_normal_high", "female_p_value_low_normal_vs_high", "female_effect_size_low_vs_normal_high", "female_effect_size_low_normal_vs_high", "male_p_value_low_vs_normal_high", "male_p_value_low_normal_vs_high", "male_effect_size_low_vs_normal_high", "male_effect_size_low_normal_vs_high", "categories", "categorical_p_value", "categorical_effect_size", "batch_significant", "variance_significant", "null_test_p_value", "genotype_effect_p_value", "genotype_effect_stderr_estimate", "genotype_effect_parameter_estimate", "male_percentage_change", "female_percentage_change", "sex_effect_p_value", "sex_effect_stderr_estimate", "sex_effect_parameter_estimate", "weight_effect_p_value", "weight_effect_stderr_estimate", "weight_effect_parameter_estimate", "group1_genotype", "group1_residuals_normality_test", "group2_genotype", "group2_residuals_normality_test", "blups_test", "rotated_residuals_test", "intercept_estimate", "intercept_estimate_stderr_estimate", "interaction_significant", "interaction_effect_p_value", "female_ko_effect_p_value", "female_ko_effect_stderr_estimate", "female_ko_parameter_estimate", "female_effect_size", "male_ko_effect_p_value", "male_ko_effect_stderr_estimate", "male_ko_parameter_estimate", "male_effect_size", "classification_tag", "phenotype_sex", "life_stage_acc", "life_stage_name", "significant", "soft_windowing_bandwidth", "soft_windowing_shape", "soft_windowing_peaks", "soft_windowing_min_obs_required", "soft_windowing_total_obs_or_weight", "soft_windowing_threshold", "soft_windowing_number_of_doe", "soft_windowing_doe_note", "metadata"]
15
- }
File without changes
File without changes
File without changes
File without changes
File without changes