bacdive 0.3.1__tar.gz → 2.0.0__tar.gz

{bacdive-0.3.1 → bacdive-2.0.0}/PKG-INFO

@@ -1,108 +1,146 @@
-Metadata-Version: 2.1
-Name: bacdive
-Version: 0.3.1
-Summary: BacDive-API - Programmatic Access to the BacDive Database
-Home-page: https://bacdive.dsmz.de/
-Author: Julia Koblitz
-Author-email: julia.koblitz@dsmz.de
-Keywords: microbiology bacteria strains phenotypes
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Intended Audience :: Science/Research
-Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
-Requires-Python: >=3.6
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: python-keycloak
-Requires-Dist: requests>=2.25.1
-Requires-Dist: urllib3>=1.26.5
-
-# BacDive API
-
-Using the BacDive API requires registration. Registration is free but the usage of BacDive data is only permitted when in compliance with the BacDive terms of use. See [About BacDive](https://bacdive.dsmz.de/about) for details.
-
-Please register [here](https://api.bacdive.dsmz.de/login).
-
-The Python package can be initialized using your login credentials:
-
-
-```python
-import bacdive
-
-client = bacdive.BacdiveClient('name@mail.example', 'password')
-
-# the search method fetches all BacDive-IDs matching your query
-# and returns the number of IDs found
-count = client.search(taxonomy='Bacillus subtilis subtilis')
-print(count, 'strains found.')
-
-# the retrieve method lets you iterate over all strains
-# and returns the full entry as dict
-# Entries can be further filtered using a list of keys (e.g. ['keywords'])
-for strain in client.retrieve():
-    print(strain)
-```
-
-## Example queries:
-
-```python
-# Search by BacDive-IDs (either semicolon separated or as list):
-query = {"id": 24493}
-query = {"id": "24493;12;132485"}
-query = {"id": [24493, 12, 132485]}
-
-# Search by culture collection number
-query = {"culturecolno": "DSM 26640"}
-
-# Search by taxonomy (either as full name or as list):
-# With genus name, species epithet (optional), and subspecies (optional).
-query = {"taxonomy": "Bacillus subtilis subsp. subtilis"}
-query = {"taxonomy": ("Escherichia", "coli")}
-
-# Search by sequence accession numbers:
-query = {"16s": "AF000162"} # 16S sequence
-query = {"genome": "GCA_006094295"} # genome sequence
-
-# run query
-client.search(**query)
-```
-
-## Filtering
-
-Results from the `retrieve` Method of both clients can be further filtered. The result contains a list of matched keyword dicts:
-
-```python
-filter=['keywords', 'culture collection no.']
-result = client.retrieve(filter)
-print({k:v for x in result for k,v in x.items()})
-```
-
-The printed result will look like this:
-
-```python
-{'1161': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4393, pC194, SB202'}],
- '1162': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4514, ATCC 37015, BD170, NCIB 11624, '
-                                     'pUB110'}],
- '1163': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4554, ATCC 37128, BGSC 1E18, pE194'}],
- '1164': [{'keywords': 'Bacteria'},
-          {'culture collection no.': 'DSM 4750, 1E7, BGSC 1E7, pE194-cop6'}],
-...
-```
-
-## New in v0.3
-
-We added AI-based predictions to the Bac*Dive* database. Predicted traits are excluded by default. To include them, you have to call the method `includePredictions()`:
-
-```python
-client.includePredictions()
-```
-
-You can exclude predictions again by calling: 
-
-```python
-client.excludePredictions()
-```
-
+Metadata-Version: 2.4
+Name: bacdive
+Version: 2.0.0
+Summary: BacDive-API - Programmatic Access to the BacDive Database
+Home-page: https://bacdive.dsmz.de/
+Author: Julia Koblitz
+Author-email: julia.koblitz@dsmz.de
+Keywords: microbiology bacteria strains phenotypes
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25.1
+Requires-Dist: urllib3>=1.26.5
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# BacDive API v2
+
+Using the BacDive API does not require registration anymore, but the usage of BacDive data is only permitted when in compliance with the BacDive terms of use. See [About BacDive](https://bacdive.dsmz.de/about) for details.
+
+```python
+import bacdive
+
+client = bacdive.BacdiveClient()
+
+# [optional] You may define the search type as one of the following: 
+# 'exact' (default), 'contains', 'startswith', 'endswith'
+client.setSearchType('exact')
+
+# The search method fetches all BacDive-IDs matching your query
+# and returns the number of IDs found
+count = client.search(taxonomy='Bacillus subtilis subtilis')
+print(count, 'strains found.')
+
+# the retrieve method lets you iterate over all strains
+# and returns the full entry as dict
+# Entries can be further filtered using a list of keys (e.g. ['keywords'])
+for strain in client.retrieve():
+    print(strain)
+```
+
+## Example queries:
+
+```python
+# Search by BacDive-IDs (either semicolon separated or as list):
+query = {"id": 24493}
+query = {"id": "24493;12;132485"}
+query = {"id": [24493, 12, 132485]}
+
+# Search by culture collection number
+query = {"culturecolno": "DSM 26640"}
+# New in v1.0: Search by culture collection number with multiple numbers:
+query = {"culturecolno": ["DSM 26640", "DSM 26646"]}
+query = {"culturecolno": "DSM 26640;DSM 26646"} # semicolon may be used as separator
+
+# Search by culture collection number with search type 'startswith':
+client.setSearchType('startswith')
+query = {"culturecolno": "DSM"}
+
+# Search by taxonomy (either as full name or as list):
+# With genus name, species epithet (optional), and subspecies (optional).
+query = {"taxonomy": "Bacillus subtilis subsp. subtilis"}
+query = {"taxonomy": ("Escherichia", "coli")}
+
+# Search by 16S sequence accession numbers:
+query = {"16s": "AF000162"}
+# New in v1.0: Search by 16S sequence with multiple sequence accession numbers:
+query = {"16s": ["AB681963", "JN566021", "AY027686"]}
+# New in v1.0: Search by 16S sequence with search type 'startswith':
+client.setSearchType('startswith')
+query = {"16s": "AB"}
+
+# Search by genome sequence accession numbers:
+query = {"genome": "GCA_006094295"}
+# New in v1.0: Search by genome sequence with multiple sequence accession numbers:
+query = {"genome": ["GCA_003332855", "GCA_024623325", "GCA_017377855"]}
+# New in v1.0: Search by genome sequence with search type 'startswith':
+client.setSearchType('startswith')
+query = {"genome": "DSM"}
+
+
+# run query
+client.search(**query)
+```
+
+## Filtering
+
+Results from the `retrieve` Method of both clients can be further filtered. The result contains a list of matched keyword dicts:
+
+```python
+filter=['keywords', 'culture collection no.']
+result = client.retrieve(filter)
+print({k:v for x in result for k,v in x.items()})
+```
+
+The printed result will look like this:
+
+```python
+{'1161': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4393, pC194, SB202'}],
+ '1162': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4514, ATCC 37015, BD170, NCIB 11624, '
+                                     'pUB110'}],
+ '1163': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4554, ATCC 37128, BGSC 1E18, pE194'}],
+ '1164': [{'keywords': 'Bacteria'},
+          {'culture collection no.': 'DSM 4750, 1E7, BGSC 1E7, pE194-cop6'}],
+...
+```
+
+## Hints for more advanced queries
+If you have more advanced queries that are currently not covered by the API, we recommend you to use the [Bac*Dive* Advanced Search](https://bacdive.dsmz.de/advsearch), which is very flexible and powerful. You can then download the resulting table as CSV (button at the top right), import the CSV into your Python script, and use the BacDive-IDs to download all relevant information via the API.
+
+
+## New in v0.3
+
+We added AI-based predictions to the Bac*Dive* database. Predicted traits are excluded by default. To include them, you have to call the method `includePredictions()`:
+
+```python
+client.includePredictions()
+```
+
+You can exclude predictions again by calling: 
+
+```python
+client.excludePredictions()
+```
+
+## New in v1.0
+
+Thanks to [phenolophthaleinum](https://github.com/phenolophthaleinum) for improving the error handling and Joaquim Sardá for improving the BacDive-API and adding new search possibilities.
+
+Examples for search type definitions and array requests are included in the examples above.

bacdive-2.0.0/README.md

@@ -0,0 +1,117 @@
+# BacDive API v2
+
+Using the BacDive API does not require registration anymore, but the usage of BacDive data is only permitted when in compliance with the BacDive terms of use. See [About BacDive](https://bacdive.dsmz.de/about) for details.
+
+```python
+import bacdive
+
+client = bacdive.BacdiveClient()
+
+# [optional] You may define the search type as one of the following: 
+# 'exact' (default), 'contains', 'startswith', 'endswith'
+client.setSearchType('exact')
+
+# The search method fetches all BacDive-IDs matching your query
+# and returns the number of IDs found
+count = client.search(taxonomy='Bacillus subtilis subtilis')
+print(count, 'strains found.')
+
+# the retrieve method lets you iterate over all strains
+# and returns the full entry as dict
+# Entries can be further filtered using a list of keys (e.g. ['keywords'])
+for strain in client.retrieve():
+    print(strain)
+```
+
+## Example queries:
+
+```python
+# Search by BacDive-IDs (either semicolon separated or as list):
+query = {"id": 24493}
+query = {"id": "24493;12;132485"}
+query = {"id": [24493, 12, 132485]}
+
+# Search by culture collection number
+query = {"culturecolno": "DSM 26640"}
+# New in v1.0: Search by culture collection number with multiple numbers:
+query = {"culturecolno": ["DSM 26640", "DSM 26646"]}
+query = {"culturecolno": "DSM 26640;DSM 26646"} # semicolon may be used as separator
+
+# Search by culture collection number with search type 'startswith':
+client.setSearchType('startswith')
+query = {"culturecolno": "DSM"}
+
+# Search by taxonomy (either as full name or as list):
+# With genus name, species epithet (optional), and subspecies (optional).
+query = {"taxonomy": "Bacillus subtilis subsp. subtilis"}
+query = {"taxonomy": ("Escherichia", "coli")}
+
+# Search by 16S sequence accession numbers:
+query = {"16s": "AF000162"}
+# New in v1.0: Search by 16S sequence with multiple sequence accession numbers:
+query = {"16s": ["AB681963", "JN566021", "AY027686"]}
+# New in v1.0: Search by 16S sequence with search type 'startswith':
+client.setSearchType('startswith')
+query = {"16s": "AB"}
+
+# Search by genome sequence accession numbers:
+query = {"genome": "GCA_006094295"}
+# New in v1.0: Search by genome sequence with multiple sequence accession numbers:
+query = {"genome": ["GCA_003332855", "GCA_024623325", "GCA_017377855"]}
+# New in v1.0: Search by genome sequence with search type 'startswith':
+client.setSearchType('startswith')
+query = {"genome": "DSM"}
+
+
+# run query
+client.search(**query)
+```
+
+## Filtering
+
+Results from the `retrieve` Method of both clients can be further filtered. The result contains a list of matched keyword dicts:
+
+```python
+filter=['keywords', 'culture collection no.']
+result = client.retrieve(filter)
+print({k:v for x in result for k,v in x.items()})
+```
+
+The printed result will look like this:
+
+```python
+{'1161': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4393, pC194, SB202'}],
+ '1162': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4514, ATCC 37015, BD170, NCIB 11624, '
+                                     'pUB110'}],
+ '1163': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4554, ATCC 37128, BGSC 1E18, pE194'}],
+ '1164': [{'keywords': 'Bacteria'},
+          {'culture collection no.': 'DSM 4750, 1E7, BGSC 1E7, pE194-cop6'}],
+...
+```
+
+## Hints for more advanced queries
+If you have more advanced queries that are currently not covered by the API, we recommend you to use the [Bac*Dive* Advanced Search](https://bacdive.dsmz.de/advsearch), which is very flexible and powerful. You can then download the resulting table as CSV (button at the top right), import the CSV into your Python script, and use the BacDive-IDs to download all relevant information via the API.
+
+
+## New in v0.3
+
+We added AI-based predictions to the Bac*Dive* database. Predicted traits are excluded by default. To include them, you have to call the method `includePredictions()`:
+
+```python
+client.includePredictions()
+```
+
+You can exclude predictions again by calling: 
+
+```python
+client.excludePredictions()
+```
+
+## New in v1.0
+
+Thanks to [phenolophthaleinum](https://github.com/phenolophthaleinum) for improving the error handling and Joaquim Sardá for improving the BacDive-API and adding new search possibilities.
+
+Examples for search type definitions and array requests are included in the examples above.

{bacdive-0.3.1 → bacdive-2.0.0}/bacdive/client.py

@@ -1,232 +1,267 @@
-'''
-Using the BacDive API requires registration. Registrations is free but the 
-usage of BacDive data is only permitted when in compliance with the BacDive 
-terms of use. See https://bacdive.dsmz.de/about for details.
-
-Please register at https://api.bacdive.dsmz.de/login.
-'''
-
-from keycloak.exceptions import KeycloakAuthenticationError
-from keycloak import KeycloakOpenID
-import requests
-import json
-
-
-class BacdiveClient():
-    def __init__(self, user, password, public=True):
-        ''' Initialize client and authenticate on the server '''
-        self.result = {}
-        self.public = public
-        
-        self.predictions = False
-
-        client_id = "api.bacdive.public"
-        if self.public:
-            server_url = "https://sso.dsmz.de/auth/"
-        else:
-            server_url = "https://sso.dmz.dsmz.de/auth/"
-        try:
-            self.keycloak_openid = KeycloakOpenID(
-                server_url=server_url,
-                client_id=client_id,
-                realm_name="dsmz")
-
-            # Get tokens
-            token = self.keycloak_openid.token(user, password)
-            self.access_token = token['access_token']
-            self.refresh_token = token['refresh_token']
-            print("-- Authentication successful --")
-        except KeycloakAuthenticationError as e:
-            print("ERROR - Authentication failed:", e)
-
-    def includePredictions(self):
-        self.predictions = True
-
-    def excludePredictions(self):
-        self.predictions = False
-
-    def do_api_call(self, url):
-        ''' Initialize API call on given URL and returns result as json '''
-        if self.public:
-            baseurl = "https://api.bacdive.dsmz.de/"
-        else:
-            baseurl = "http://api.bacdive-dev.dsmz.local/"
-        
-        if not url.startswith("http"):
-            # if base is missing add default:
-            url = baseurl + url
-        resp = self.do_request(url)
-
-        if resp.status_code == 500 or resp.status_code == 400:
-            return json.loads(resp.content)
-        elif (resp.status_code == 401):
-            msg = json.loads(resp.content)
-
-            if msg['message'] == "Expired token":
-                # Access token might have expired (15 minutes life time).
-                # Get new tokens using refresh token and try again.
-                token = self.keycloak_openid.refresh_token(self.refresh_token)
-                self.access_token = token['access_token']
-                self.refresh_token = token['refresh_token']
-                return self.do_api_call(url)
-
-            return msg
-        else:
-            return json.loads(resp.content)
-
-    def do_request(self, url):
-        ''' Perform request with authentication '''
-        headers = {
-            "Accept": "application/json",
-            "Authorization": "Bearer {token}".format(token=self.access_token)
-        }
-
-        if self.predictions:
-            if "?" in url:
-                url += "&predictions=1"
-            else:
-                url += "?predictions=1"
-        resp = requests.get(url, headers=headers)
-        return resp
-
-
-    def filterResult(self, d, keys):
-        ''' Helper function to filter nested dict by keys '''
-        if not isinstance(d, dict):
-            yield None
-        for k, v in d.items():
-            if k in keys:
-                yield {k: v}
-            if isinstance(v, dict):
-                yield from self.filterResult(v, keys)
-            elif isinstance(v, list):
-                for i in v:
-                    if isinstance(i, dict):
-                        yield from self.filterResult(i, keys)
-
-    def retrieve(self, filter=None):
-        ''' Yields all the received entries and does next call if result is incomplete '''
-        ids = ";".join([str(i) for i in self.result['results']])
-        entries = self.do_api_call('fetch/'+ids)['results']
-        for el in entries:
-            if isinstance(el, dict):
-                entry = el
-                el = entry.get("id")
-            else:
-                entry = entries[el]
-            if filter:
-                entry = {el: [i for i in self.filterResult(entry, filter)]}
-            yield entry
-        if self.result['next']:
-            self.result = self.do_api_call(self.result['next'])
-            yield from self.retrieve(filter)
-
-    def getIDByCultureno(self, culturecolnumber):
-        ''' Initialize search by culture collection number '''
-        item = culturecolnumber.strip()
-        result = self.do_api_call('culturecollectionno/'+str(item))
-        return result
-
-    def getIDsByTaxonomy(self, genus, species_epithet=None, subspecies_epithet=None):
-        ''' Initialize search by taxonomic names '''
-        item = genus.strip()
-        if species_epithet:
-            item += "/" + species_epithet
-            if subspecies_epithet:
-                item += "/" + subspecies_epithet
-        result = self.do_api_call("taxon/"+item)
-        return result
-
-    def getIDsBy16S(self, seq_acc_num):
-        ''' Initialize search by 16S sequence accession '''
-        item = seq_acc_num.strip()
-        result = self.do_api_call('sequence_16s/'+str(item))
-        return result
-
-    def getIDsByGenome(self, seq_acc_num):
-        ''' Initialize search by genome sequence accession '''
-        item = seq_acc_num.strip()
-        result = self.do_api_call('sequence_genome/'+str(item))
-        return result
-
-    def search(self, **params):
-        ''' Initialize search with *one* of the following parameters:
-        
-        id -- BacDive-IDs either as a semicolon seperated string or list
-        taxonomy -- Taxonomic names either as string or list
-        sequence -- Sequence accession number of unknown type
-        genome -- Genome sequence accession number
-        16s -- 16S sequence accession number
-        culturecolno -- Culture collection number (mind the space!)
-        '''
-        params = list(params.items())
-        allowed = ['id', 'taxonomy', 'sequence',
-                   'genome', '16s', 'culturecolno']
-        if len(params) != 1:
-            print(
-                "ERROR: Exacly one parameter is required. Please choose one of the following:")
-            print(", ".join(allowed))
-            return 0
-        querytype, query = params[0]
-        querytype = querytype.lower()
-        if querytype not in allowed:
-            print(
-                "ERROR: The given query type is not allowed. Please choose one of the following:")
-            print(", ".join(allowed))
-            return 0
-        if querytype == 'id':
-            if type(query) == type(1):
-                query = str(query)
-            if type(query) == type(""):
-                query = query.split(';')
-            self.result = {'count': len(query), 'next': None,
-                           'previous': None, 'results': query}
-        elif querytype == 'taxonomy':
-            if type(query) == type(""):
-                query = [i for i in query.split(" ") if i != "subsp."]
-            if len(query) > 3:
-                print("Your query contains more than three taxonomical units.")
-                print(
-                    "This query supports only genus, species epithet (optional), and subspecies (optional).")
-                print("They can be defined as list, tuple or string (space separated).")
-                return 0
-            self.result = self.getIDsByTaxonomy(*query)
-        elif querytype == 'sequence':
-            self.result = self.getIDsByGenome(query)
-            if self.result['count'] == 0:
-                self.result = self.getIDsBy16S(query)
-        elif querytype == 'genome':
-            self.result = self.getIDsByGenome(query)
-        elif querytype == '16s':
-            self.result = self.getIDsBy16S(query)
-        elif querytype == 'culturecolno':
-            self.result = self.getIDByCultureno(query)
-
-        if not self.result:
-            print("ERROR: Something went wrong. Please check your query and try again")
-            return 0
-        if not 'count' in self.result:
-            print("ERROR:", self.result.get("title"))
-            print(self.result.get("message"))
-            return 0
-        if self.result['count'] == 0:
-            print("Your search did not receive any results.")
-            return 0
-        return self.result['count']
-
-
-
-
-if __name__ == "__main__":
-    client = BacdiveClient('mail.address@server.example', 'password')
-
-    # the prepare method fetches all BacDive-IDs matching your query
-    # and returns the number of IDs found
-    count = client.search(taxonomy='Bacillus subtilis subtilis')
-    print(count, 'entries found.')
-
-    # The retrieve method lets you iterate over all entries
-    # and returns the full entry as dict
-    # Entries can be further filtered using a list of keys (e.g. ['keywords'])
-    for entry in client.retrieve():
-        print(entry)
+'''
+This package is for using the BacDive API V2 (2026-02).
+Registration is not required anymore.
+'''
+
+from requests.adapters import HTTPAdapter
+from urllib3.util.retry import Retry
+import requests
+import json
+import time
+
+
+class ReportRetry(Retry):
+    ''' Wrapper for retry strategy to report retries'''
+    def __init__(self, url=None, *args, **kwargs):
+        self.url = url
+        self.retry_count = 0
+        super().__init__(*args, **kwargs)
+
+    def increment(self, *args, **kwargs):
+        self.retry_count += 1
+        print(f"Retrying API request for {self.url}. Attempt number {self.retry_count}.")
+        return super().increment(*args, **kwargs)
+
+
+class BacdiveClient():
+    def __init__(self, user=None, password=None, public=True, max_retries=10, retry_delay=50, request_timeout=300):
+        ''' Initialize client and authenticate on the server '''
+        self.result = {}
+        self.public = public
+        self.max_retries = max_retries
+        self.retry_delay = retry_delay # in seconds
+        self.request_timeout = request_timeout # in seconds
+        
+        self.predictions = False
+        self.search_type = False
+
+        client_id = "api.bacdive.public"
+        if self.public:
+            server_url = "https://sso.dsmz.de/auth/"
+        else:
+            server_url = "https://sso.dmz.dsmz.de/auth/"
+
+    def includePredictions(self):
+        self.predictions = True
+
+    def excludePredictions(self):
+        self.predictions = False
+
+    def setSearchType(self, search_type):
+        if search_type:
+            allowed = ['exact', 'contains', 'startswith', 'endswith']
+            if search_type not in allowed:
+                print("WARNING - Search Type is not allowed.")
+                self.search_type = "exact"
+            else:
+                self.search_type = search_type
+        else:
+            self.search_type = False
+
+    def do_api_call(self, url):
+        ''' Initialize API call on given URL and returns result as json '''
+        if self.public:
+            baseurl = "https://api.bacdive.dsmz.de/v2/"
+        else:
+            baseurl = "http://api.bacdive-dev.dsmz.local/v2/"
+        
+        if not url.startswith("http"):
+            # if base is missing add default:
+            url = baseurl + url
+        resp = self.do_request(url)
+
+        if resp.status_code == 500 or resp.status_code == 400 or resp.status_code == 503:
+            print(f"Error {resp.status_code}: {resp.content}")
+            return json.loads(resp.content)
+        elif (resp.status_code == 401):
+            msg = json.loads(resp.content)
+
+            return msg
+        else:
+            return json.loads(resp.content)
+
+    def do_request(self, url):
+        ''' Perform request'''
+        headers = {
+            "Accept": "application/json",
+        }
+
+        if self.predictions:
+            if "?" in url:
+                url += "&predictions=1"
+            else:
+                url += "?predictions=1"
+        # session with retry strategy
+        retry_strategy = ReportRetry(
+            url=url,
+            total=self.max_retries,
+            backoff_factor=1, # how much to increase delay between each try
+            status_forcelist=[429, 500, 502, 503, 504] # retry on
+        )
+        adapter = HTTPAdapter(max_retries=retry_strategy)
+        http = requests.Session()
+        http.mount("https://", adapter)
+        http.mount("http://", adapter)
+
+        resp = http.get(url, headers=headers, timeout=self.request_timeout) # timeout in seconds
+        return resp
+
+    def filterResult(self, d, keys):
+        ''' Helper function to filter nested dict by keys '''
+        if not isinstance(d, dict):
+            yield None
+        for k, v in d.items():
+            if k in keys:
+                yield {k: v}
+            if isinstance(v, dict):
+                yield from self.filterResult(v, keys)
+            elif isinstance(v, list):
+                for i in v:
+                    if isinstance(i, dict):
+                        yield from self.filterResult(i, keys)
+
+    def retrieve(self, filter=None):
+        ''' Yields all the received entries and does next call if result is incomplete '''
+        ids = ";".join([str(i) for i in self.result['results']])
+        entries = self.do_api_call('fetch/'+ids)['results'] if ids else []
+        for el in entries:
+            if isinstance(el, dict):
+                entry = el
+                el = entry.get("id")
+            else:
+                entry = entries[el]
+            if filter:
+                entry = {el: [i for i in self.filterResult(entry, filter)]}
+            yield entry
+        if self.result['next']:
+            self.result = self.do_api_call(self.result['next'])
+            yield from self.retrieve(filter)
+
+    def getIDByCultureno(self, culturecolnumber):
+        ''' Initialize search by culture collection number '''
+        item = culturecolnumber.strip()
+        result = self.do_api_call('culturecollectionno/'+str(item))
+        return result
+
+    def getIDsByTaxonomy(self, genus, species_epithet=None, subspecies_epithet=None):
+        ''' Initialize search by taxonomic names '''
+        item = genus.strip()
+        if species_epithet:
+            item += "/" + species_epithet
+            if subspecies_epithet:
+                item += "/" + subspecies_epithet
+        result = self.do_api_call("taxon/"+item)
+        return result
+
+    def getIDsBy16S(self, seq_acc_num):
+        ''' Initialize search by 16S sequence accession '''
+        item = seq_acc_num.strip()
+        result = self.do_api_call('sequence_16s/'+str(item))
+        return result
+
+    def getIDsByGenome(self, seq_acc_num):
+        ''' Initialize search by genome sequence accession '''
+        item = seq_acc_num.strip()
+        result = self.do_api_call('sequence_genome/'+str(item))
+        return result
+
+    def search(self, **params):
+        ''' Initialize search with *one* of the following parameters:
+        
+        id -- BacDive-IDs either as a semicolon seperated string or list
+        taxonomy -- Taxonomic names either as string or list
+        sequence -- Sequence accession number of unknown type
+        genome -- Genome sequence accession number
+        16s -- 16S sequence accession number
+        culturecolno -- Culture collection number (mind the space!)
+        '''
+        params = list(params.items())
+        allowed = ['id', 'taxonomy', 'sequence',
+                   'genome', '16s', 'culturecolno']
+        if len(params) != 1:
+            print(
+                "ERROR: Exacly one parameter is required. Please choose one of the following:")
+            print(", ".join(allowed))
+            return 0
+        querytype, query = params[0]
+        querytype = querytype.lower()
+        if querytype not in allowed:
+            print(
+                "ERROR: The given query type is not allowed. Please choose one of the following:")
+            print(", ".join(allowed))
+            return 0
+        if querytype == 'id':
+            if type(query) == type(1):
+                query = str(query)
+            if type(query) == type(""):
+                query = query.split(';')
+            self.result = {'count': len(query), 'next': None,
+                           'previous': None, 'results': query}
+        elif querytype == 'taxonomy':
+            if type(query) == type(""):
+                query = [i for i in query.split(" ") if i != "subsp."]
+            if len(query) > 3:
+                print("Your query contains more than three taxonomical units.")
+                print(
+                    "This query supports only genus, species epithet (optional), and subspecies (optional).")
+                print("They can be defined as list, tuple or string (space separated).")
+                return 0
+            self.result = self.getIDsByTaxonomy(*query)
+        elif querytype == 'sequence':
+            query = self.parseSearchTypeQuery(query)
+            self.result = self.getIDsByGenome(query)
+            if self.result['count'] == 0:
+                self.result = self.getIDsBy16S(query)
+        elif querytype == 'genome':
+            query = self.parseSearchTypeQuery(query)
+            self.result = self.getIDsByGenome(query)
+        elif querytype == '16s':
+            query = self.parseSearchTypeQuery(query)
+            self.result = self.getIDsBy16S(query)
+        elif querytype == 'culturecolno':
+            query = self.parseSearchTypeQuery(query)
+            self.result = self.getIDByCultureno(query)
+
+        if not self.result:
+            print("ERROR: Something went wrong. Please check your query and try again")
+            return 0
+        if not 'count' in self.result:
+            print("ERROR:", self.result.get("title"))
+            print(self.result.get("message"))
+            return 0
+        if self.result['count'] == 0:
+            print("Your search did not receive any results.")
+            return 0
+        return self.result['count']
+
+    def parseSearchTypeQuery(self, query):
+        if type(query) == type(1):
+            query = str(query)
+        if type(query) == type(""):
+            query = query.split(';')
+
+        item = ";".join([ str(i).strip() for i in query ])
+
+        if self.search_type:
+            if "?" in item:
+                item += "&search_type=" + self.search_type
+            else:
+                item += "?search_type=" + self.search_type
+
+        return item
+
+
+
+if __name__ == "__main__":
+    client = BacdiveClient('mail.address@server.example', 'password')
+
+    # the prepare method fetches all BacDive-IDs matching your query
+    # and returns the number of IDs found
+    count = client.search(taxonomy='Bacillus subtilis subtilis')
+    print(count, 'entries found.')
+
+    # The retrieve method lets you iterate over all entries
+    # and returns the full entry as dict
+    # Entries can be further filtered using a list of keys (e.g. ['keywords'])
+    for entry in client.retrieve():
+        print(entry)

{bacdive-0.3.1 → bacdive-2.0.0}/bacdive.egg-info/PKG-INFO

@@ -1,108 +1,146 @@
-Metadata-Version: 2.1
-Name: bacdive
-Version: 0.3.1
-Summary: BacDive-API - Programmatic Access to the BacDive Database
-Home-page: https://bacdive.dsmz.de/
-Author: Julia Koblitz
-Author-email: julia.koblitz@dsmz.de
-Keywords: microbiology bacteria strains phenotypes
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Intended Audience :: Science/Research
-Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
-Requires-Python: >=3.6
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: python-keycloak
-Requires-Dist: requests>=2.25.1
-Requires-Dist: urllib3>=1.26.5
-
-# BacDive API
-
-Using the BacDive API requires registration. Registration is free but the usage of BacDive data is only permitted when in compliance with the BacDive terms of use. See [About BacDive](https://bacdive.dsmz.de/about) for details.
-
-Please register [here](https://api.bacdive.dsmz.de/login).
-
-The Python package can be initialized using your login credentials:
-
-
-```python
-import bacdive
-
-client = bacdive.BacdiveClient('name@mail.example', 'password')
-
-# the search method fetches all BacDive-IDs matching your query
-# and returns the number of IDs found
-count = client.search(taxonomy='Bacillus subtilis subtilis')
-print(count, 'strains found.')
-
-# the retrieve method lets you iterate over all strains
-# and returns the full entry as dict
-# Entries can be further filtered using a list of keys (e.g. ['keywords'])
-for strain in client.retrieve():
-    print(strain)
-```
-
-## Example queries:
-
-```python
-# Search by BacDive-IDs (either semicolon separated or as list):
-query = {"id": 24493}
-query = {"id": "24493;12;132485"}
-query = {"id": [24493, 12, 132485]}
-
-# Search by culture collection number
-query = {"culturecolno": "DSM 26640"}
-
-# Search by taxonomy (either as full name or as list):
-# With genus name, species epithet (optional), and subspecies (optional).
-query = {"taxonomy": "Bacillus subtilis subsp. subtilis"}
-query = {"taxonomy": ("Escherichia", "coli")}
-
-# Search by sequence accession numbers:
-query = {"16s": "AF000162"} # 16S sequence
-query = {"genome": "GCA_006094295"} # genome sequence
-
-# run query
-client.search(**query)
-```
-
-## Filtering
-
-Results from the `retrieve` Method of both clients can be further filtered. The result contains a list of matched keyword dicts:
-
-```python
-filter=['keywords', 'culture collection no.']
-result = client.retrieve(filter)
-print({k:v for x in result for k,v in x.items()})
-```
-
-The printed result will look like this:
-
-```python
-{'1161': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4393, pC194, SB202'}],
- '1162': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4514, ATCC 37015, BD170, NCIB 11624, '
-                                     'pUB110'}],
- '1163': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4554, ATCC 37128, BGSC 1E18, pE194'}],
- '1164': [{'keywords': 'Bacteria'},
-          {'culture collection no.': 'DSM 4750, 1E7, BGSC 1E7, pE194-cop6'}],
-...
-```
-
-## New in v0.3
-
-We added AI-based predictions to the Bac*Dive* database. Predicted traits are excluded by default. To include them, you have to call the method `includePredictions()`:
-
-```python
-client.includePredictions()
-```
-
-You can exclude predictions again by calling: 
-
-```python
-client.excludePredictions()
-```
-
+Metadata-Version: 2.4
+Name: bacdive
+Version: 2.0.0
+Summary: BacDive-API - Programmatic Access to the BacDive Database
+Home-page: https://bacdive.dsmz.de/
+Author: Julia Koblitz
+Author-email: julia.koblitz@dsmz.de
+Keywords: microbiology bacteria strains phenotypes
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.25.1
+Requires-Dist: urllib3>=1.26.5
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: keywords
+Dynamic: license-file
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
+# BacDive API v2
+
+Using the BacDive API does not require registration anymore, but the usage of BacDive data is only permitted when in compliance with the BacDive terms of use. See [About BacDive](https://bacdive.dsmz.de/about) for details.
+
+```python
+import bacdive
+
+client = bacdive.BacdiveClient()
+
+# [optional] You may define the search type as one of the following: 
+# 'exact' (default), 'contains', 'startswith', 'endswith'
+client.setSearchType('exact')
+
+# The search method fetches all BacDive-IDs matching your query
+# and returns the number of IDs found
+count = client.search(taxonomy='Bacillus subtilis subtilis')
+print(count, 'strains found.')
+
+# the retrieve method lets you iterate over all strains
+# and returns the full entry as dict
+# Entries can be further filtered using a list of keys (e.g. ['keywords'])
+for strain in client.retrieve():
+    print(strain)
+```
+
+## Example queries:
+
+```python
+# Search by BacDive-IDs (either semicolon separated or as list):
+query = {"id": 24493}
+query = {"id": "24493;12;132485"}
+query = {"id": [24493, 12, 132485]}
+
+# Search by culture collection number
+query = {"culturecolno": "DSM 26640"}
+# New in v1.0: Search by culture collection number with multiple numbers:
+query = {"culturecolno": ["DSM 26640", "DSM 26646"]}
+query = {"culturecolno": "DSM 26640;DSM 26646"} # semicolon may be used as separator
+
+# Search by culture collection number with search type 'startswith':
+client.setSearchType('startswith')
+query = {"culturecolno": "DSM"}
+
+# Search by taxonomy (either as full name or as list):
+# With genus name, species epithet (optional), and subspecies (optional).
+query = {"taxonomy": "Bacillus subtilis subsp. subtilis"}
+query = {"taxonomy": ("Escherichia", "coli")}
+
+# Search by 16S sequence accession numbers:
+query = {"16s": "AF000162"}
+# New in v1.0: Search by 16S sequence with multiple sequence accession numbers:
+query = {"16s": ["AB681963", "JN566021", "AY027686"]}
+# New in v1.0: Search by 16S sequence with search type 'startswith':
+client.setSearchType('startswith')
+query = {"16s": "AB"}
+
+# Search by genome sequence accession numbers:
+query = {"genome": "GCA_006094295"}
+# New in v1.0: Search by genome sequence with multiple sequence accession numbers:
+query = {"genome": ["GCA_003332855", "GCA_024623325", "GCA_017377855"]}
+# New in v1.0: Search by genome sequence with search type 'startswith':
+client.setSearchType('startswith')
+query = {"genome": "DSM"}
+
+
+# run query
+client.search(**query)
+```
+
+## Filtering
+
+Results from the `retrieve` Method of both clients can be further filtered. The result contains a list of matched keyword dicts:
+
+```python
+filter=['keywords', 'culture collection no.']
+result = client.retrieve(filter)
+print({k:v for x in result for k,v in x.items()})
+```
+
+The printed result will look like this:
+
+```python
+{'1161': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4393, pC194, SB202'}],
+ '1162': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4514, ATCC 37015, BD170, NCIB 11624, '
+                                     'pUB110'}],
+ '1163': [{'keywords': ['human pathogen', 'Bacteria']},
+          {'culture collection no.': 'DSM 4554, ATCC 37128, BGSC 1E18, pE194'}],
+ '1164': [{'keywords': 'Bacteria'},
+          {'culture collection no.': 'DSM 4750, 1E7, BGSC 1E7, pE194-cop6'}],
+...
+```
+
+## Hints for more advanced queries
+If you have more advanced queries that are currently not covered by the API, we recommend you to use the [Bac*Dive* Advanced Search](https://bacdive.dsmz.de/advsearch), which is very flexible and powerful. You can then download the resulting table as CSV (button at the top right), import the CSV into your Python script, and use the BacDive-IDs to download all relevant information via the API.
+
+
+## New in v0.3
+
+We added AI-based predictions to the Bac*Dive* database. Predicted traits are excluded by default. To include them, you have to call the method `includePredictions()`:
+
+```python
+client.includePredictions()
+```
+
+You can exclude predictions again by calling: 
+
+```python
+client.excludePredictions()
+```
+
+## New in v1.0
+
+Thanks to [phenolophthaleinum](https://github.com/phenolophthaleinum) for improving the error handling and Joaquim Sardá for improving the BacDive-API and adding new search possibilities.
+
+Examples for search type definitions and array requests are included in the examples above.

{bacdive-0.3.1 → bacdive-2.0.0}/bacdive.egg-info/requires.txt

@@ -1,3 +1,2 @@
-python-keycloak
 requests>=2.25.1
 urllib3>=1.26.5

{bacdive-0.3.1 → bacdive-2.0.0}/setup.cfg

@@ -1,4 +1,4 @@
-[egg_info]
-tag_build = 
-tag_date = 0
-
+[egg_info]
+tag_build = 
+tag_date = 0
+

{bacdive-0.3.1 → bacdive-2.0.0}/setup.py

@@ -5,7 +5,7 @@ with open("README.md", "r", encoding="utf-8") as fh:
 
 setuptools.setup(
     name="bacdive",
-    version="0.3.1",
+    version="2.0.0",
     description="BacDive-API - Programmatic Access to the BacDive Database",
     long_description=long_description,
     long_description_content_type="text/markdown",
@@ -22,7 +22,6 @@ setuptools.setup(
     ],
     keywords="microbiology bacteria strains phenotypes",
     install_requires=[
-        "python-keycloak",
         "requests>=2.25.1",
         "urllib3>=1.26.5"
     ]

bacdive-0.3.1/README.md

@@ -1,89 +0,0 @@
-# BacDive API
-
-Using the BacDive API requires registration. Registration is free but the usage of BacDive data is only permitted when in compliance with the BacDive terms of use. See [About BacDive](https://bacdive.dsmz.de/about) for details.
-
-Please register [here](https://api.bacdive.dsmz.de/login).
-
-The Python package can be initialized using your login credentials:
-
-
-```python
-import bacdive
-
-client = bacdive.BacdiveClient('name@mail.example', 'password')
-
-# the search method fetches all BacDive-IDs matching your query
-# and returns the number of IDs found
-count = client.search(taxonomy='Bacillus subtilis subtilis')
-print(count, 'strains found.')
-
-# the retrieve method lets you iterate over all strains
-# and returns the full entry as dict
-# Entries can be further filtered using a list of keys (e.g. ['keywords'])
-for strain in client.retrieve():
-    print(strain)
-```
-
-## Example queries:
-
-```python
-# Search by BacDive-IDs (either semicolon separated or as list):
-query = {"id": 24493}
-query = {"id": "24493;12;132485"}
-query = {"id": [24493, 12, 132485]}
-
-# Search by culture collection number
-query = {"culturecolno": "DSM 26640"}
-
-# Search by taxonomy (either as full name or as list):
-# With genus name, species epithet (optional), and subspecies (optional).
-query = {"taxonomy": "Bacillus subtilis subsp. subtilis"}
-query = {"taxonomy": ("Escherichia", "coli")}
-
-# Search by sequence accession numbers:
-query = {"16s": "AF000162"} # 16S sequence
-query = {"genome": "GCA_006094295"} # genome sequence
-
-# run query
-client.search(**query)
-```
-
-## Filtering
-
-Results from the `retrieve` Method of both clients can be further filtered. The result contains a list of matched keyword dicts:
-
-```python
-filter=['keywords', 'culture collection no.']
-result = client.retrieve(filter)
-print({k:v for x in result for k,v in x.items()})
-```
-
-The printed result will look like this:
-
-```python
-{'1161': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4393, pC194, SB202'}],
- '1162': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4514, ATCC 37015, BD170, NCIB 11624, '
-                                     'pUB110'}],
- '1163': [{'keywords': ['human pathogen', 'Bacteria']},
-          {'culture collection no.': 'DSM 4554, ATCC 37128, BGSC 1E18, pE194'}],
- '1164': [{'keywords': 'Bacteria'},
-          {'culture collection no.': 'DSM 4750, 1E7, BGSC 1E7, pE194-cop6'}],
-...
-```
-
-## New in v0.3
-
-We added AI-based predictions to the Bac*Dive* database. Predicted traits are excluded by default. To include them, you have to call the method `includePredictions()`:
-
-```python
-client.includePredictions()
-```
-
-You can exclude predictions again by calling: 
-
-```python
-client.excludePredictions()
-```
-