pytolino 1.7__tar.gz → 2.0__tar.gz

{pytolino-1.7/src/pytolino.egg-info → pytolino-2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pytolino
-Version: 1.7
+Version: 2.0
 Summary: client for tolino cloud
 Author: Imam Usmani
 Project-URL: Source Code, https://github.com/ImamAzim/pytolino
@@ -15,6 +15,8 @@ Description-Content-Type: text/x-rst
 License-File: LICENSE
 Requires-Dist: requests
 Requires-Dist: mechanize
+Requires-Dist: curl_cffi
+Requires-Dist: varboxes
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: flake8; extra == "dev"
@@ -25,10 +27,16 @@ Requires-Dist: twine; extra == "dev"
 Requires-Dist: sphinx-rtd-theme; extra == "dev"
 Dynamic: license-file
 
+UPDATE
+===========
+
+because of heavy anti-bot protection, it is no longer possible to make a fully automatic login. One can however reuse authorization token after a manual login. The token can then be refreshed automatically, for example with a cronjob (at least once per hour.)
+
+
 pytolino
 ===================
 
-A client to interact (login, upload, delete ebooks, etc..) with the tolino cloud with python. Most of the code is forked from https://github.com/darkphoenix/tolino-calibre-sync
+A client to interact (login, upload, delete ebooks, etc..) with the tolino cloud with python. thanks to https://github.com/darkphoenix/tolino-calibre-sync for the inspiration.
 
 One difference is that I aim to create a python package from it and to put it on pypi, so that one can use this python module in other projects.
 
@@ -42,48 +50,50 @@ Installation
 Usage
 =====
 
+First, login manually, and use an inspector tool in the browser to inspect the requests. After connecting to the digital libray of tolino, there is POST request (named token). From the request response, copy the value of the refresh token (and the expiration time in seconds). Then, in a PATH request, in the request header, find the device_id number.
 
-Before being able to send requests, you need to register your computer on which you will run the code:
+You can then store the token:
 
 .. code-block:: python
 
-    from pytolino.tolino_cloud import Client, PytolinoException
-    client = Client()
-    client.login(USERNAME, PASSWORD)
-    client.register() # do this only once!
-    client.logout()
+    from pytolino.tolino_cloud import Client
+    partner = 'orellfuessli'
+    account_name = 'any name for reference'
+    client = Client(partner)
+    print('login on your browser and get the token.')
+    refresh_token = input('refresh token:\n')
+    expires_in = int(input('expires_in:\n'))
+    hardware_id = input('hardware id:\n')
+    Client.store_token(
+    account_name, refresh_token, expires_in, hardware_id)
 
-You can then upload, add to a collection or delete ebook on your cloud:
+Then, get a new access token. It will expires in 1 hours, so you might want to create a crontab job to do it regularely:
 
 .. code-block:: python
 
-    from pytolino.tolino_cloud import Client, PytolinoException
-    client = Client()
-    client.login(USERNAME, PASSWORD)
+    from pytolino.tolino_cloud import Client
+    partner = 'orellfuessli'
+    account_name = 'any name for reference'
+    client = Client(partner)
+    client.get_new_token(account_name)
+
+After this, instead of login, you only need to retrieve the access token that is stored on disk and upload, delete books, etc...
+
+.. code-block:: python
+
+    from pytolino.tolino_cloud import Client
+    partner = 'orellfuessli'
+    account_name = 'any name for reference'
+    client = Client(partner)
+    client.retrieve_token(account_name)
+
     ebook_id = client.upload(EPUB_FILE_PATH) # return a unique id that can be used for reference
     client.add_collection(epub_id, 'science fiction') # add the previous book to the collection science-fiction
     client.add_cover(epub_id, cover_path) # to upload a cover on the book.
     client.delete_ebook(epub_id) # delete the previousely uploaded ebook
     inventory = client.get_inventory() # get a list of all the books on the cloud and their metadata
     client.upload_metadata(epub_id, title='my title', author='someone') # you can upload various kind of metadata
-    client.logout()
-
 
-if you want to unregister your computer:
-
-.. code-block:: python
-
-    from pytolino.tolino_cloud import Client, PytolinoException
-    client = Client()
-    client.login(USERNAME, PASSWORD)
-    client.register() # now you will not be able to upload books from this computer
-    client.logout()
-
-By default, it will connect to 'www.buecher.de'. In principle you could change the partner with:
-
-.. code-block:: python
-
-   client = Client(server_name='www.orelfuessli') # for example if you have an account at orel fuessli.
 
 To get a list of the supported partners:
 
@@ -92,22 +102,17 @@ To get a list of the supported partners:
    from pytolino.tolino_cloud import PARTNERS
    print(PARTNERS)
 
-Unfortunately, the only supported partner now is 'www.buecher.de', because it has a different way of connection... So for now, the only solution is to create an account there and link it to your original account.
-
+for now, only orelfuessli is supported, but it should be easy to include the others (but always need of a manual login)
 
 
 Features
 ========
 
-* login to tolino partner (for now works only with buecher.de)
-* register device
-* unregister device
 * upload ebook
 * delete ebook from the cloud
 * add a book to a collection
 * download inventory
 * upload metadata
-* more to come...
 
 
 License

pytolino-2.0/README.rst

@@ -0,0 +1,92 @@
+UPDATE
+===========
+
+because of heavy anti-bot protection, it is no longer possible to make a fully automatic login. One can however reuse authorization token after a manual login. The token can then be refreshed automatically, for example with a cronjob (at least once per hour.)
+
+
+pytolino
+===================
+
+A client to interact (login, upload, delete ebooks, etc..) with the tolino cloud with python. thanks to https://github.com/darkphoenix/tolino-calibre-sync for the inspiration.
+
+One difference is that I aim to create a python package from it and to put it on pypi, so that one can use this python module in other projects.
+
+Installation
+============
+
+.. code-block:: bash
+
+    pip install pytolino
+
+Usage
+=====
+
+First, login manually, and use an inspector tool in the browser to inspect the requests. After connecting to the digital libray of tolino, there is POST request (named token). From the request response, copy the value of the refresh token (and the expiration time in seconds). Then, in a PATH request, in the request header, find the device_id number.
+
+You can then store the token:
+
+.. code-block:: python
+
+    from pytolino.tolino_cloud import Client
+    partner = 'orellfuessli'
+    account_name = 'any name for reference'
+    client = Client(partner)
+    print('login on your browser and get the token.')
+    refresh_token = input('refresh token:\n')
+    expires_in = int(input('expires_in:\n'))
+    hardware_id = input('hardware id:\n')
+    Client.store_token(
+    account_name, refresh_token, expires_in, hardware_id)
+
+Then, get a new access token. It will expires in 1 hours, so you might want to create a crontab job to do it regularely:
+
+.. code-block:: python
+
+    from pytolino.tolino_cloud import Client
+    partner = 'orellfuessli'
+    account_name = 'any name for reference'
+    client = Client(partner)
+    client.get_new_token(account_name)
+
+After this, instead of login, you only need to retrieve the access token that is stored on disk and upload, delete books, etc...
+
+.. code-block:: python
+
+    from pytolino.tolino_cloud import Client
+    partner = 'orellfuessli'
+    account_name = 'any name for reference'
+    client = Client(partner)
+    client.retrieve_token(account_name)
+
+    ebook_id = client.upload(EPUB_FILE_PATH) # return a unique id that can be used for reference
+    client.add_collection(epub_id, 'science fiction') # add the previous book to the collection science-fiction
+    client.add_cover(epub_id, cover_path) # to upload a cover on the book.
+    client.delete_ebook(epub_id) # delete the previousely uploaded ebook
+    inventory = client.get_inventory() # get a list of all the books on the cloud and their metadata
+    client.upload_metadata(epub_id, title='my title', author='someone') # you can upload various kind of metadata
+
+
+To get a list of the supported partners:
+
+.. code-block:: python
+
+   from pytolino.tolino_cloud import PARTNERS
+   print(PARTNERS)
+
+for now, only orelfuessli is supported, but it should be easy to include the others (but always need of a manual login)
+
+
+Features
+========
+
+* upload ebook
+* delete ebook from the cloud
+* add a book to a collection
+* download inventory
+* upload metadata
+
+
+License
+=======
+
+The project is licensed under GNU GENERAL PUBLIC LICENSE v3.0

{pytolino-1.7 → pytolino-2.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "pytolino"
-version = "1.7"
+version = "2.0"
 authors = [
     {name="Imam Usmani"},
 ]
@@ -21,6 +21,8 @@ description = "client for tolino cloud"
 dependencies = [
     'requests',
     'mechanize',
+    'curl_cffi',
+    'varboxes',
 ]
 
 [project.optional-dependencies]
@@ -31,7 +33,7 @@ dev = [
 	'sphinx',
 	'build',
 	'twine',
-	'sphinx-rtd-theme'
+	'sphinx-rtd-theme',
 ]
 
 [project.urls]

pytolino-2.0/src/pytolino/tolino_cloud.py

@@ -0,0 +1,414 @@
+#!/usr/bin/env python3
+
+
+import logging
+import json
+import time
+import tomllib
+from pathlib import Path
+
+
+import requests
+import curl_cffi
+from varboxes import VarBox
+
+
+class PytolinoException(Exception):
+    pass
+
+
+class ExpirationError(PytolinoException):
+    pass
+
+
+SERVERS_SETTINGS_FN = 'servers_settings.toml'
+SERVERS_SETTINGS_FP = Path(__file__).parent / SERVERS_SETTINGS_FN
+servers_settings = tomllib.loads(SERVERS_SETTINGS_FP.read_text())
+PARTNERS = servers_settings.keys()
+
+
+def main():
+    for partner in PARTNERS:
+        print(partner)
+        for key, val in servers_settings[partner].items():
+            print(key, val)
+
+
+class Client(object):
+
+    """create a client to communicate with a tolino partner (login, etc..)"""
+
+    def store_token(
+                    account_name,
+                    refresh_token: str,
+                    expires_in: int,
+                    hardward_id: str,
+                    access_token='',
+                    ):
+        """after one has connected in a browser,
+        one can store the token for the app.
+
+        :account_name: internal name for reference
+        :refresh_token: given by server after a token POST request
+        :expires_in: time in seconds
+        :hardward_id: present in payload for every request to API.
+
+        """
+        vb = VarBox('pytolino', app_name=account_name)
+        vb.refresh_token = refresh_token
+        vb.hardware_id = hardward_id
+        vb.expires_in = expires_in
+        vb.timestamp = time.time()
+        vb.access_token = access_token
+
+    @property
+    def refresh_token(self) -> str:
+        """refresh token to get new access token"""
+        return self._refresh_token
+
+    @property
+    def hardware_id(self) -> str:
+        """hardware id that is sent in request payloads"""
+        return self._hardware_id
+
+    def __init__(self, server_name='orellfuessli'):
+
+        if server_name not in servers_settings:
+            raise PytolinoException(
+                    f'the partner {server_name} was not found.'
+                    f'please choose one of the list: {PARTNERS}')
+
+        self._access_token = None
+        self._refresh_token = None
+        self._token_expires = None
+        self._hardware_id = None
+
+        self._server_settings = servers_settings[server_name]
+        self._session = requests.Session()
+        self._session_cffi = curl_cffi.Session()
+        self._server_name = server_name
+
+    def retrieve_token(
+            self,
+            account_name,
+            ) -> tuple[str, str]:
+        """get the token and data that were stored previousely.
+        raise error if expired
+
+        :account_name: internal name under which token was stored
+        :returns: refresh_token, hardware_id
+
+        """
+        vb = VarBox('pytolino', app_name=account_name)
+        if not hasattr(vb, 'refresh_token'):
+            raise PytolinoException(
+                    'there was no refresh token stored for that name')
+        now = time.time()
+        expiration_time = vb.timestamp + vb.expires_in
+        if now > expiration_time:
+            raise ExpirationError('the refresh token has expired')
+        else:
+            self._refresh_token = vb.refresh_token
+            self._hardware_id = vb.hardware_id
+            self._access_token = vb.access_token
+
+    def get_new_token(self, account_name):
+        """look at the store token, and get a new access and refresh tokens.
+
+        :account_name: TODO
+        :returns: TODO
+
+        """
+        self.retrieve_token(account_name)
+
+        headers = {
+                'Referer': 'https://webreader.mytolino.com/',
+                }
+        payload = {
+            'client_id': 'webreader',
+            'grant_type': 'refresh_token',
+            'refresh_token': self.refresh_token,
+            'scope': 'SCOPE_BOSH',
+        }
+        host_response = self._session_cffi.post(
+                self._server_settings['token_url'],
+                data=payload,
+                verify=True,
+                allow_redirects=True,
+                headers=headers,
+                impersonate='chrome',
+                )
+        if not host_response.ok:
+            msg = str(host_response)
+            logging.error('failed to get a new token')
+            logging.error(msg)
+            raise PytolinoException('failed to get a new token')
+        else:
+            j = host_response.json()
+            self._access_token = j['access_token']
+            self._refresh_token = j['refresh_token']
+            self._token_expires = int(j['expires_in'])
+            Client.store_token(
+                    account_name,
+                    self.refresh_token,
+                    self._token_expires,
+                    self.hardware_id,
+                    access_token=self._access_token,
+                    )
+            logging.info('got a new access token!')
+
+    def login(self, username, password, fp=None):
+        """login to the partner and get access token.
+
+        """
+        msg = 'login does not work anymore because of bot protection'
+        'connect manualy (once) and use store_token and retrieve token'
+        'methods instead'
+        raise NotImplementedError(msg)
+
+    def logout(self):
+        """logout from tolino partner host
+
+        """
+        raise NotImplementedError('logout is not necessary with tokens')
+
+    def register(self):
+        raise NotImplementedError('register is not necessary with tokens')
+
+    def unregister(self, device_id=None):
+        raise NotImplementedError('unregister is not necessary with tokens')
+
+    def get_inventory(self):
+        """download a list of the books on the cloud and their information
+        :returns: list of dict describing the book, with a epubMetaData dict
+
+        """
+
+        host_response = self._session.get(
+                self._server_settings['inventory_url'],
+                params={'strip': 'true'},
+                headers={
+                    't_auth_token': self._access_token,
+                    'hardware_id': self._hardware_id,
+                    'reseller_id': self._server_settings['partner_id'],
+                    }
+                )
+
+        if not host_response.ok:
+            raise PytolinoException(
+                    f'inventory request failed {host_response}')
+
+        try:
+            j = host_response.json()
+        except requests.JSONDecodeError:
+            raise PytolinoException(
+                    'inventory list request failed because of json error.'
+                    )
+        else:
+            try:
+                publication_inventory = j['PublicationInventory']
+                uploaded_ebooks = publication_inventory['edata']
+                purchased_ebook = publication_inventory['ebook']
+            except KeyError:
+                raise PytolinoException(
+                        'inventory list request failed because',
+                        'of key error in json.',
+                        )
+            else:
+                inventory = uploaded_ebooks + purchased_ebook
+                return inventory
+
+    def add_to_collection(self, book_id, collection_name):
+        """add a book to a collection on the cloud
+
+        :book_id: identify the book on the cloud
+        :collection_name: str name
+
+        """
+
+        payload = {
+                "revision": None,
+                "patches": [{
+                    "op": "add",
+                    "value": {
+                        "modified": round(time.time() * 1000),
+                        "name": collection_name,
+                        "category": "collection",
+                    },
+                    "path": f"/publications/{book_id}/tags"
+                    }]
+                }
+
+        host_response = self._session.patch(
+                self._server_settings['sync_data_url'],
+                data=json.dumps(payload),
+                headers={
+                    'content-type': 'application/json',
+                    't_auth_token': self._access_token,
+                    'hardware_id': self.hardware_id,
+                    'reseller_id': self._server_settings['partner_id'],
+                    'client_type': 'TOLINO_WEBREADER',
+                    }
+                )
+
+        if not host_response.ok:
+            raise PytolinoException(
+                    f'collection add failed {host_response}')
+
+    def upload_metadata(self, book_id, **new_metadata):
+        """upload some metadata to a specific book on the cloud
+
+        :book_id: ref on the cloud of the book
+        :**meta_data: dict of metadata than can be changed
+
+        """
+
+        url = self._server_settings['meta_url'] + f'/?deliverableId={book_id}'
+        host_response = self._session.get(
+                url,
+                headers={
+                    't_auth_token': self._access_token,
+                    'hardware_id': self.hardware_id,
+                    'reseller_id': self._server_settings['partner_id'],
+                  }
+                )
+
+        book = host_response.json()
+        if not host_response.ok:
+            raise PytolinoException(f'metadata upload failed {host_response}')
+
+        for key, value in new_metadata.items():
+            book['metadata'][key] = value
+
+        payload = {
+                'uploadMetaData': book['metadata']
+                }
+
+        host_response = self._session.put(
+                url,
+                data=json.dumps(payload),
+                headers={
+                    'content-type': 'application/json',
+                    't_auth_token': self._access_token,
+                    'hardware_id': self.hardware_id,
+                    'reseller_id': self._server_settings['partner_id'],
+                    }
+                )
+
+        if not host_response.ok:
+            raise PytolinoException(f'metadata upload failed {host_response}')
+
+    def upload(self, file_path, name=None, extension=None):
+        """upload an ebook to your cloud
+
+        :file_path: str path to the ebook to upload
+        :name: str name of book if different from filename
+        :extension: epub or pdf, if not in filename
+        :returns: epub_id on the server
+
+        """
+
+        if name is None:
+            name = file_path.split('/')[-1]
+        if extension is None:
+            extension = file_path.split('.')[-1]
+
+        mime = {
+                'pdf': 'application/pdf',
+                'epub': 'application/epub+zip',
+                }.get(extension.lower(), 'application/pdf')
+
+        host_response = self._session.post(
+                self._server_settings['upload_url'],
+                files=[(
+                    'file',
+                    (
+                        name,
+                        open(file_path, 'rb'),
+                        mime,
+                        ),
+                    )],
+                headers={
+                    't_auth_token': self._access_token,
+                    'hardware_id': self.hardware_id,
+                    'reseller_id': self._server_settings['partner_id'],
+                    }
+                )
+
+        if not host_response.ok:
+            raise PytolinoException(f'upload failed {host_response}')
+
+        try:
+            j = host_response.json()
+        except requests.JSONDecodeError:
+            raise PytolinoException('file upload failed.')
+        else:
+            try:
+                return j['metadata']['deliverableId']
+            except KeyError:
+                raise PytolinoException('file upload failed.')
+
+    def delete_ebook(self, ebook_id):
+        """delete an ebook present on your cloud
+
+        :ebook_id: id of the ebook to delete.
+        :returns: None
+
+        """
+        host_response = self._session.get(
+                self._server_settings['delete_url'],
+                params={'deliverableId': ebook_id},
+                headers={
+                    't_auth_token': self._access_token,
+                    'hardware_id': self.hardware_id,
+                    'reseller_id': self._server_settings['partner_id'],
+                    }
+                )
+
+        if not host_response.ok:
+            try:
+                j = host_response.json()
+                raise PytolinoException(
+                        f"delete {ebook_id} failed: ",
+                        f"{j['ResponseInfo']['message']}"
+                        )
+            except KeyError:
+                raise PytolinoException(
+                        f'delete {ebook_id} failed: reason unknown.')
+
+    def add_cover(self, book_id, filepath, file_ext=None):
+        """upload a a cover to a book on the cloud
+
+        :book_id: id of the book on the serveer
+        :filepath: path to the cover file
+        :file_ext: png, jpg or jpeg. only necessary if the
+        filepath has no extension
+
+        """
+
+        if file_ext is None:
+            ext = filepath.split('.')[-1]
+
+        mime = {
+                'png': 'image/png',
+                'jpeg': 'image/jpeg',
+                'jpg': 'image/jpeg'
+                }.get(ext.lower(), 'application/jpeg')
+
+        host_response = self._session.post(
+                self._server_settings['cover_url'],
+                files=[('file', ('1092560016', open(filepath, 'rb'), mime))],
+                data={'deliverableId': book_id},
+                headers={
+                    't_auth_token': self._access_token,
+                    'hardware_id': self.hardware_id,
+                    'reseller_id': self._server_settings['partner_id'],
+                    },
+                )
+
+        if not host_response.ok:
+            raise PytolinoException(f'cover upload failed. {host_response}')
+
+
+if __name__ == '__main__':
+    main()