overleaf-sync 1.2.0__tar.gz → 2.0.0a2__tar.gz

overleaf_sync-2.0.0a2/PKG-INFO

@@ -0,0 +1,148 @@
+Metadata-Version: 2.4
+Name: overleaf-sync
+Version: 2.0.0a2
+Summary: Overleaf Two-Way Sync Tool
+Keywords: overleaf,sync,latex,tex
+Author-email: Moritz Glöckl <moritzgloeckl@users.noreply.github.com>
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+License-File: LICENSE
+Requires-Dist: requests >=2.31,<3
+Requires-Dist: beautifulsoup4 >=4.12,<5
+Requires-Dist: yaspin >=3,<4
+Requires-Dist: python-dateutil >=2.8,<3
+Requires-Dist: click >=8.1,<9
+Requires-Dist: socketIO-client ==0.5.7.2
+Requires-Dist: websocket-client >=1,<2
+Requires-Dist: pytest >=8 ; extra == "dev"
+Requires-Dist: pytest-cov >=5 ; extra == "dev"
+Requires-Dist: responses >=0.25 ; extra == "dev"
+Requires-Dist: ruff >=0.6 ; extra == "dev"
+Requires-Dist: PySide6 >=6.5,<7 ; extra == "gui"
+Project-URL: Homepage, https://github.com/moritzgloeckl/overleaf-sync
+Project-URL: Issues, https://github.com/moritzgloeckl/overleaf-sync/issues
+Provides-Extra: dev
+Provides-Extra: gui
+
+# Overleaf-Sync
+### Easy Overleaf Two-Way Synchronization
+
+![Made In Austria](https://img.shields.io/badge/Made%20in-Austria-%23ED2939.svg) ![PyPI - License](https://img.shields.io/pypi/l/overleaf-sync.svg) ![PyPI](https://img.shields.io/pypi/v/overleaf-sync.svg) ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/overleaf-sync.svg)
+
+This tool provides an easy way to synchronize Overleaf projects from and to your local computer. No paid account necessary.
+
+----
+
+## Features
+- Sync your locally modified `.tex` (and other) files to your Overleaf projects
+- Sync your remotely modified `.tex` (and other) files to computer
+- Works with free Overleaf account
+- No Git or Dropbox required
+- Does not steal or store your login credentials (works with a persisted cookie, logging in is done on the original Overleaf website)
+
+## How To Use
+### Install
+The package is available via [PyPI](https://pypi.org/project/overleaf-sync/) and needs Python 3.11 or newer. Install it including the login window (`gui` extra):
+
+```
+moritz@github:~/test$ pip3 install "overleaf-sync[gui]"
+```
+
+Depending on your local Python installation, you might need to use `pip` instead of `pip3`. [pipx](https://pipx.pypa.io/) works too: `pipx install "overleaf-sync[gui]"`.
+
+> **2.0 pre-release:** Overleaf changed its website in a way that broke version 1.2.0. The fixes are available as a pre-release: `pip3 install --pre "overleaf-sync[gui]"`. Please report any problems in the [issues](https://github.com/moritzgloeckl/overleaf-sync/issues).
+
+### Prerequisites
+- Create your project on [Overleaf](https://www.overleaf.com/project), for example a project named `test`. Overleaf-sync is not able to create projects (yet).
+- Create a folder, preferably with the same name as the project (`test`) on your computer.
+- Execute the script from that folder (`test`).
+- If you do not specify the project name, overleaf-sync uses the current folder's name as the project name.
+
+### Usage
+#### Login
+```
+moritz@github:~/test$ ols login [--path]
+Login successful. Cookie persisted as `.olauth`. You may now sync your project.
+```
+
+Logging in will be handled by a mini web browser opening on your device (using Qt via PySide6, from the `gui` extra). You can then enter your username and password securely on the official Overleaf website. You might get asked to solve a CAPTCHA in the process. Your credentials are sent to Overleaf over HTTPS.
+
+It then stores your *cookie* (**not** your login credentials) in a hidden file called `.olauth` in the same folder you run the command from. It is possible to store the cookie elsewhere using the `--path` option. The cookie file will not be synced to or from Overleaf.
+
+Keep the `.olauth` file save, as it can be used to log in into your account.
+
+#### Login without the login window
+On a server without a display, or if the login window doesn't work for your account (e.g. single sign-on via ORCID or your institution), log in with a session cookie instead:
+
+1. Log in to Overleaf in your normal browser.
+2. Open the developer tools → Application (Firefox: Storage) → Cookies → `https://www.overleaf.com` and copy the value of `overleaf_session2`.
+3. Run `ols login --cookie` and paste the value at the (hidden) prompt, or provide it via the `OLSYNC_COOKIE` environment variable.
+
+This works without installing the `gui` extra.
+
+### Listing all projects
+```
+moritz@github:~/test$ ols list [--store-path -v/--verbose]
+10/31/2021, 01:23:45 - Project A
+09/21/2020, 01:23:45 - Project B
+08/11/2019, 01:23:45 - Project C
+07/01/2018, 01:23:45 - Project D
+```
+
+Use `ols list` to conveniently list all projects in your account available for syncing. 
+
+### Downloading project's PDF
+```
+moritz@github:~/test$ ols download [--name --download-path --store-path -v/--verbose]
+```
+
+Use `ols download` to compile and download your project's PDF. Specify a download path if you do not want to store the PDF file in the current folder. Currently only downloads the first PDF file it finds.
+
+### Syncing
+```
+moritz@github:~/test$ ols [-l/--local-only -r/--remote-only --store-path -p/--path -i/--olignore]
+```
+
+Just calling `ols` will two-way sync your project. When there are changes both locally, and remotely you will be asked which file to keep. Using the `-l` or `-r` option you can specify to either sync local project files to Overleaf only or Overleaf files to local ones only respectively. When using these options you can also sync deleted files. If a file has been deleted it can either be deleted on the target (remote when `-l`, local when `-r`) as well, restored on the source (local when `-l`, remote when `-r`) or ignored.
+
+The option `--store-path` specifies the path of the cookie file created by the `login` command. If you did not change its path, you do not need to specify this argument. The `-p/--path` option allows you to specify a different sync folder than the one you're calling `ols` from. The `-i/--olignore` option allows you to specify the path of an `.olignore` file. It uses `fnmatch` internally, so it may have some similarity to `.gitignore` but doesn't work exactly the same. For example, if you wish to exclude a specific folder named `out`, you need to specify it as `out/*`. See [here](https://docs.python.org/3/library/fnmatch.html) for more information.
+
+Sample Output:
+
+```
+Project queried successfully.
+✅  Querying project
+Project downloaded successfully.
+✅  Downloading project
+
+Syncing files from remote to local
+====================
+
+[SYNCING] report.tex
+report.tex does not exist on local. Creating file.
+
+[SYNCING] other-report.tex
+other-report.tex does not exist on local. Creating file.
+
+
+✅  Syncing files from remote to local
+```
+
+## Known Bugs
+- When modifying a file on Overleaf and immediately syncing afterwards, the tool might not detect the changes. Please allow 1-2 minutes after modifying a file on Overleaf before syncing it to your local computer.
+
+## Contributing
+
+All pull requests and change/feature requests are welcome.
+
+## Disclaimer
+THE AUTHOR OF THIS SOFTWARE AND THIS SOFTWARE IS NOT ENDORSED BY, DIRECTLY AFFILIATED WITH, MAINTAINED, AUTHORIZED, OR SPONSORED BY OVERLEAF OR WRITELATEX LIMITED. ALL PRODUCT AND COMPANY NAMES ARE THE REGISTERED TRADEMARKS OF THEIR ORIGINAL OWNERS. THE USE OF ANY TRADE NAME OR TRADEMARK IS FOR IDENTIFICATION AND REFERENCE PURPOSES ONLY AND DOES NOT IMPLY ANY ASSOCIATION WITH THE TRADEMARK HOLDER OF THEIR PRODUCT BRAND.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+THIS SOFTWARE WAS DESIGNED TO BE USED ONLY FOR RESEARCH PURPOSES. THIS SOFTWARE COMES WITH NO WARRANTIES OF ANY KIND WHATSOEVER. USE IT AT YOUR OWN RISK! IF THESE TERMS ARE NOT ACCEPTABLE, YOU AREN'T ALLOWED TO USE THE CODE.
+
+

{overleaf-sync-1.2.0 → overleaf_sync-2.0.0a2}/README.md

@@ -16,13 +16,15 @@ This tool provides an easy way to synchronize Overleaf projects from and to your
 
 ## How To Use
 ### Install
-The package is available via [PyPI](https://pypi.org/project/overleaf-sync/). Just run:
+The package is available via [PyPI](https://pypi.org/project/overleaf-sync/) and needs Python 3.11 or newer. Install it including the login window (`gui` extra):
 
 ```
-moritz@github:~/test$ pip3 install overleaf-sync
+moritz@github:~/test$ pip3 install "overleaf-sync[gui]"
 ```
 
-That's it! Depending on your local Python installation, you might need to use `pip` instead of `pip3`.
+Depending on your local Python installation, you might need to use `pip` instead of `pip3`. [pipx](https://pipx.pypa.io/) works too: `pipx install "overleaf-sync[gui]"`.
+
+> **2.0 pre-release:** Overleaf changed its website in a way that broke version 1.2.0. The fixes are available as a pre-release: `pip3 install --pre "overleaf-sync[gui]"`. Please report any problems in the [issues](https://github.com/moritzgloeckl/overleaf-sync/issues).
 
 ### Prerequisites
 - Create your project on [Overleaf](https://www.overleaf.com/project), for example a project named `test`. Overleaf-sync is not able to create projects (yet).
@@ -37,12 +39,21 @@ moritz@github:~/test$ ols login [--path]
 Login successful. Cookie persisted as `.olauth`. You may now sync your project.
 ```
 
-Logging in will be handled by a mini web browser opening on your device (using Qt5). You can then enter your username and password securely on the official Overleaf website. You might get asked to solve a CAPTCHA in the process. Your credentials are sent to Overleaf over HTTPS.
+Logging in will be handled by a mini web browser opening on your device (using Qt via PySide6, from the `gui` extra). You can then enter your username and password securely on the official Overleaf website. You might get asked to solve a CAPTCHA in the process. Your credentials are sent to Overleaf over HTTPS.
 
 It then stores your *cookie* (**not** your login credentials) in a hidden file called `.olauth` in the same folder you run the command from. It is possible to store the cookie elsewhere using the `--path` option. The cookie file will not be synced to or from Overleaf.
 
 Keep the `.olauth` file save, as it can be used to log in into your account.
 
+#### Login without the login window
+On a server without a display, or if the login window doesn't work for your account (e.g. single sign-on via ORCID or your institution), log in with a session cookie instead:
+
+1. Log in to Overleaf in your normal browser.
+2. Open the developer tools → Application (Firefox: Storage) → Cookies → `https://www.overleaf.com` and copy the value of `overleaf_session2`.
+3. Run `ols login --cookie` and paste the value at the (hidden) prompt, or provide it via the `OLSYNC_COOKIE` environment variable.
+
+This works without installing the `gui` extra.
+
 ### Listing all projects
 ```
 moritz@github:~/test$ ols list [--store-path -v/--verbose]

{overleaf-sync-1.2.0 → overleaf_sync-2.0.0a2}/olsync/__init__.py

@@ -1,3 +1,3 @@
 """Overleaf Two-Way Sync Tool"""
 
-__version__ = '1.2.0'
+__version__ = '2.0.0a2'

{overleaf-sync-1.2.0 → overleaf_sync-2.0.0a2}/olsync/olbrowserlogin.py

@@ -6,7 +6,6 @@
 # Description: Overleaf Browser Login Utility
 # Author: Moritz Glöckl
 # License: MIT
-# Version: 1.2.0
 ##################################################
 
 from PySide6.QtCore import *

{overleaf-sync-1.2.0 → overleaf_sync-2.0.0a2}/olsync/olclient.py

@@ -6,16 +6,24 @@
 # Description: Overleaf API Wrapper
 # Author: Moritz Glöckl
 # License: MIT
-# Version: 1.2.0
 ##################################################
 
 import requests as reqs
 from bs4 import BeautifulSoup
 import json
+import ssl
 import uuid
+import websocket
 from socketIO_client import SocketIO
 import time
 
+# socketIO_client/transports.py has `except websocket.SSLError`, an attribute
+# recent websocket-client (1.9.0+) no longer exposes; that raises AttributeError
+# and masks whatever the actual connection error is. Keep this shim for as long
+# as we depend on socketIO-client.
+if not hasattr(websocket, "SSLError"):
+    websocket.SSLError = ssl.SSLError
+
 # Where to get the CSRF Token and where to send the login request to
 LOGIN_URL = "https://www.overleaf.com/login"
 PROJECT_URL = "https://www.overleaf.com/project"  # The dashboard URL
@@ -23,7 +31,7 @@ PROJECT_URL = "https://www.overleaf.com/project"  # The dashboard URL
 DOWNLOAD_URL = "https://www.overleaf.com/project/{}/download/zip"
 UPLOAD_URL = "https://www.overleaf.com/project/{}/upload"  # The URL to upload files
 FOLDER_URL = "https://www.overleaf.com/project/{}/folder"  # The URL to create folders
-DELETE_URL = "https://www.overleaf.com/project/{}/doc/{}"  # The URL to delete files
+DELETE_URL = "https://www.overleaf.com/project/{}/{}/{}"  # The URL to delete files (entity type: doc or file)
 COMPILE_URL = "https://www.overleaf.com/project/{}/compile?enable_pdf_caching=true"  # The URL to compile the project
 BASE_URL = "https://www.overleaf.com"  # The Overleaf Base URL
 PATH_SEP = "/"  # Use hardcoded path separator for both windows and posix system
@@ -82,14 +90,45 @@ class OverleafClient(object):
 
             return {"cookie": self._cookie, "csrf": self._csrf}
 
+    @staticmethod
+    def _extract_projects(page_content):
+        """
+        Extract the project list from the Overleaf dashboard page.
+
+        Overleaf used to embed the list as a bare JSON array in a
+        <meta name="ol-projects"> tag. It now embeds
+        {"totalSize": N, "projects": [...]} in a
+        <meta name="ol-prefetchedProjectsBlob"> tag instead. Support both so
+        this keeps working if Overleaf reverts or if an older cookie/page is
+        cached somewhere.
+        """
+        soup = BeautifulSoup(page_content, 'html.parser')
+        tag = soup.find('meta', {'name': 'ol-prefetchedProjectsBlob'}) or soup.find('meta', {'name': 'ol-projects'})
+        json_content = json.loads(tag.get('content'))
+        if isinstance(json_content, dict):
+            json_content = json_content.get('projects', [])
+        return json_content
+
+    def fetch_csrf(self):
+        """
+        Get the CSRF token from the project dashboard, e.g. after logging in with a cookie copied from a browser
+        Returns: the CSRF token, or None if the session cookie is not (or no longer) valid
+        """
+        projects_page = reqs.get(PROJECT_URL, cookies=self._cookie, allow_redirects=False)
+        if not projects_page.ok or projects_page.is_redirect:
+            return None
+        csrf_tag = BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-csrfToken'})
+        if csrf_tag is None:
+            return None
+        return csrf_tag.get('content')
+
     def all_projects(self):
         """
         Get all of a user's active projects (= not archived and not trashed)
         Returns: List of project objects
         """
         projects_page = reqs.get(PROJECT_URL, cookies=self._cookie)
-        json_content = json.loads(
-            BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-projects'}).get('content'))
+        json_content = OverleafClient._extract_projects(projects_page.content)
         return list(OverleafClient.filter_projects(json_content))
 
     def get_project(self, project_name):
@@ -100,8 +139,7 @@ class OverleafClient(object):
         """
 
         projects_page = reqs.get(PROJECT_URL, cookies=self._cookie)
-        json_content = json.loads(
-            BeautifulSoup(projects_page.content, 'html.parser').find('meta', {'name': 'ol-projects'}).get('content'))
+        json_content = OverleafClient._extract_projects(projects_page.content)
         return next(OverleafClient.filter_projects(json_content, {"name": project_name}), None)
 
     def download_project(self, project_id):
@@ -138,8 +176,11 @@ class OverleafClient(object):
 
         if r.ok:
             return json.loads(r.content)
-        elif r.status_code == str(400):
-            # Folder already exists
+        elif r.status_code == 400:
+            # Folder already exists. status_code is an int; comparing to
+            # str(400) was always False, so this branch never actually
+            # fired -- a real folder-exists response fell through to the
+            # HTTPError below instead of silently continuing.
             return
         else:
             raise reqs.HTTPError()
@@ -155,23 +196,50 @@ class OverleafClient(object):
         """
         project_infos = None
 
-        # Callback function for the joinProject emitter
-        def set_project_infos(a, project_infos_dict, c, d):
-            # Set project_infos variable in outer scope
+        # Callback for the joinProjectResponse event the server pushes right
+        # after a successful connect+auto-join (see below) -- delivered as a
+        # plain named event, not as an ack to an explicit emit, and shaped as
+        # {"publicId": ..., "project": {...}}. Downstream code (upload_file,
+        # delete_file) indexes project_infos['rootFolder'][0][...] directly,
+        # so unwrap to the inner "project" dict here rather than returning
+        # the wrapper.
+        def set_project_infos(data):
             nonlocal project_infos
-            project_infos = project_infos_dict
-
-        # Convert cookie from CookieJar to string
-        cookie = "GCLB={}; overleaf_session2={}" \
-            .format(
-            self._cookie["GCLB"],
-            self._cookie["overleaf_session2"]
+            project_infos = data.get('project')
+
+        # The Socket.IO handshake below (done again internally by the SocketIO()
+        # constructor) has Overleaf issue a *fresh* load-balancer affinity cookie
+        # (currently named GCLB, see olbrowserlogin.COOKIE_NAMES and commit
+        # 0cc61e7) on its response -- but socketIO_client calls requests.get()
+        # directly rather than through a Session, so that Set-Cookie is silently
+        # dropped, and the WebSocket upgrade that follows then 502s without it.
+        # Perform the same handshake ourselves first, purely to capture that
+        # cookie, and merge it into the header used for the real connection.
+        # (Built from whatever cookies are present rather than hardcoded names:
+        # a session may not carry every cookie -- this previously raised
+        # KeyError here even though overleaf_session2 alone is enough to connect.)
+        handshake = reqs.get(
+            "{}/socket.io/1/".format(BASE_URL),
+            params={"t": int(time.time() * 1000)},
+            cookies=self._cookie,
+        )
+        cookie_dict = dict(self._cookie)
+        cookie_dict.update(handshake.cookies.get_dict())
+        cookie = "; ".join(
+            "{}={}".format(name, value) for name, value in cookie_dict.items()
         )
 
-        # Connect to Overleaf Socket.IO, send a time parameter and the cookies
+        # projectId must be a query parameter on THIS handshake -- the one
+        # socketIO_client performs internally when SocketIO() connects, not
+        # the pre-fetch above. Overleaf now binds the session to a project at
+        # handshake time and rejects the connection otherwise
+        # (connectionRejected: "missing/bad ?projectId=... query flag on
+        # handshake"); confirmed it rejects even when projectId is only on
+        # the WebSocket upgrade URL -- it must be on the initial
+        # /socket.io/1/ GET specifically.
         socket_io = SocketIO(
             BASE_URL,
-            params={'t': int(time.time())},
+            params={'t': int(time.time()), 'projectId': project_id},
             headers={'Cookie': cookie}
         )
 
@@ -179,9 +247,19 @@ class OverleafClient(object):
         socket_io.on('connect', lambda: None)
         socket_io.wait_for_callbacks()
 
-        # Send the joinProject event and receive the project infos
-        socket_io.emit('joinProject', {'project_id': project_id}, set_project_infos)
-        socket_io.wait_for_callbacks()
+        # With projectId on the handshake, the server auto-joins and pushes
+        # the result as a joinProjectResponse event -- no explicit joinProject
+        # emit needed (confirmed live: the emit's ack callback never fires
+        # since the server responds with a named event, not an ack).
+        # wait_for_callbacks() only waits for a pending *ack* callback, which
+        # .on() never registers -- it would return immediately without
+        # reading anything. Wait in short slices instead, stopping as soon as
+        # the event has actually been processed.
+        socket_io.on('joinProjectResponse', set_project_infos)
+        for _ in range(20):
+            if project_infos is not None:
+                break
+            socket_io.wait(seconds=1)
 
         # Disconnect from the socket if still connected
         if socket_io.connected:
@@ -233,13 +311,26 @@ class OverleafClient(object):
             "qqtotalfilesize": file_size,
         }
         files = {
+            # Overleaf's upload endpoint now rejects the request with 422
+            # {"success":false,"error":"invalid_filename"} unless the
+            # filename is also present as a plain multipart field, not just
+            # in qqfilename above. Confirmed live: identical request without
+            # this field gets 422; with it, 200 and the file is actually
+            # updated. It must be the bare file name: Overleaf rejects names
+            # containing "/" (SafePath), the folder is chosen via folder_id.
+            "name": (None, file_name.split(PATH_SEP)[-1]),
             "qqfile": file
         }
 
         # Upload the file to the predefined folder
         r = reqs.post(UPLOAD_URL.format(project_id), cookies=self._cookie, params=params, files=files)
 
-        return r.status_code == str(200) and json.loads(r.content)["success"]
+        # status_code is an int; comparing to str(200) was always False here,
+        # so this reported failure on every call regardless of the real
+        # result. Harmless today only because callers don't check the return
+        # value, but worth fixing since it's the only signal this method
+        # gives back.
+        return r.status_code == 200 and json.loads(r.content)["success"]
 
     def delete_file(self, project_id, project_infos, file_name):
         """
@@ -252,23 +343,21 @@ class OverleafClient(object):
         Returns: True on success, False on fail
         """
 
-        file = None
-
-        # The file name contains path separators, check folders
-        if PATH_SEP in file_name:
-            local_folders = file_name.split(PATH_SEP)[:-1]  # Remove last item since this is the file name
-            current_overleaf_folder = project_infos['rootFolder'][0]['folders']  # Set the current remote folder
-
-            for local_folder in local_folders:
-                for remote_folder in current_overleaf_folder:
-                    if local_folder.lower() == remote_folder['name'].lower():
-                        file = next((v for v in remote_folder['docs'] if v['name'] == file_name.split(PATH_SEP)[-1]),
-                                    None)
-                        current_overleaf_folder = remote_folder['folders']
-                        break
-        # File is in root folder
-        else:
-            file = next((v for v in project_infos['rootFolder'][0]['docs'] if v['name'] == file_name), None)
+        # Walk down to the folder that contains the file (folder names match case-insensitively, as in upload_file)
+        *local_folders, name = file_name.split(PATH_SEP)
+        current_folder = project_infos['rootFolder'][0]
+        for local_folder in local_folders:
+            current_folder = next((f for f in current_folder['folders'] if f['name'].lower() == local_folder.lower()),
+                                  None)
+            if current_folder is None:
+                return False
+
+        # Text files are "docs", binary files (images, PDFs, ...) are "fileRefs" with their own endpoint
+        file = next((v for v in current_folder.get('docs', []) if v['name'] == name), None)
+        entity_type = "doc"
+        if file is None:
+            file = next((v for v in current_folder.get('fileRefs', []) if v['name'] == name), None)
+            entity_type = "file"
 
         # File not found!
         if file is None:
@@ -278,9 +367,13 @@ class OverleafClient(object):
             "X-Csrf-Token": self._csrf
         }
 
-        r = reqs.delete(DELETE_URL.format(project_id, file['_id']), cookies=self._cookie, headers=headers, json={})
+        r = reqs.delete(DELETE_URL.format(project_id, entity_type, file['_id']), cookies=self._cookie,
+                        headers=headers, json={})
 
-        return r.status_code == str(204)
+        # status_code is an int; comparing to str(204) was always False
+        # regardless of the real result, same bug class as upload_file and
+        # create_folder above.
+        return r.status_code == 204
 
     def download_pdf(self, project_id):
         """
@@ -299,7 +392,7 @@ class OverleafClient(object):
             "check": "silent",
             "draft": False,
             "incrementalCompilesEnabled": True,
-            "rootDoc_id": "",
+            # No rootDoc_id: Overleaf validates it as an optional ObjectId and rejects "" (400)
             "stopOnFirstError": False
         }
 
@@ -315,7 +408,15 @@ class OverleafClient(object):
 
         pdf_file = next(v for v in compile_result['outputFiles'] if v['type'] == 'pdf')
 
-        download_req = reqs.get(BASE_URL + pdf_file['url'], cookies=self._cookie, headers=headers)
+        # The output is only served (otherwise 404) when the compile group and CLSI server are passed along,
+        # like Overleaf's own frontend does. Self-hosted servers may not send them.
+        params = {}
+        if compile_result.get("compileGroup"):
+            params["compileGroup"] = compile_result["compileGroup"]
+        if compile_result.get("clsiServerId"):
+            params["clsiserverid"] = compile_result["clsiServerId"]
+
+        download_req = reqs.get(BASE_URL + pdf_file['url'], cookies=self._cookie, headers=headers, params=params)
 
         if download_req.ok:
             return pdf_file['path'], download_req.content

{overleaf-sync-1.2.0 → overleaf_sync-2.0.0a2}/olsync/olsync.py

@@ -6,11 +6,11 @@
 # Description: Overleaf Two-Way Sync
 # Author: Moritz Glöckl
 # License: MIT
-# Version: 1.2.0
 ##################################################
 
 import click
 import os
+import sys
 from yaspin import yaspin
 import pickle
 import zipfile
@@ -24,11 +24,21 @@ from pathlib import Path
 try:
     # Import for pip installation / wheel
     from olsync.olclient import OverleafClient
-    import olsync.olbrowserlogin as olbrowserlogin
 except ImportError:
     # Import for development
     from olclient import OverleafClient
-    import olbrowserlogin
+
+
+def configure_console():
+    """
+    Windows consoles often default to a legacy codepage (e.g. cp1252) that can't
+    encode the emoji/braille characters used for status output and the spinner,
+    which crashes stdout.write() with UnicodeEncodeError. Force UTF-8 output.
+    Done when the CLI starts rather than on import, so importing olsync has no side effects.
+    """
+    for stream in (sys.stdout, sys.stderr):
+        if hasattr(stream, "reconfigure"):
+            stream.reconfigure(encoding="utf-8", errors="replace")
 
 
 @click.group(invoke_without_command=True)
@@ -48,6 +58,7 @@ except ImportError:
 @click.version_option(package_name='overleaf-sync')
 @click.pass_context
 def main(ctx, local, remote, project_name, cookie_path, sync_path, olignore_path, verbose):
+    configure_console()
     if ctx.invoked_subcommand is None:
         if not os.path.isfile(cookie_path):
             raise click.ClickException(
@@ -87,9 +98,11 @@ def main(ctx, local, remote, project_name, cookie_path, sync_path, olignore_path
         sync = not (local or remote)
 
         if remote or sync:
+            remote_files = zip_file.namelist()
+            deleted_files = [] if sync else [f for f in olignore_keep_list(olignore_path) if f not in remote_files]
             sync_func(
-                files_from=zip_file.namelist(),
-                deleted_files=[f for f in olignore_keep_list(olignore_path) if f not in zip_file.namelist() and not sync],
+                files_from=remote_files,
+                deleted_files=deleted_files,
                 create_file_at_to=lambda name: write_file(name, zip_file.read(name)),
                 delete_file_at_to=lambda name: delete_file(name),
                 create_file_at_from=lambda name: overleaf_client.upload_file(
@@ -102,9 +115,12 @@ def main(ctx, local, remote, project_name, cookie_path, sync_path, olignore_path
                 to_name="local",
                 verbose=verbose)
         if local or sync:
+            # Computed once (#60), and after the remote -> local pass so newly downloaded files are included
+            local_files = olignore_keep_list(olignore_path)
+            deleted_files = [] if sync else [f for f in zip_file.namelist() if f not in local_files]
             sync_func(
-                files_from=olignore_keep_list(olignore_path),
-                deleted_files=[f for f in zip_file.namelist() if f not in olignore_keep_list(olignore_path) and not sync],
+                files_from=local_files,
+                deleted_files=deleted_files,
                 create_file_at_to=lambda name: overleaf_client.upload_file(
                     project["id"], project_infos, name, os.path.getsize(name), open(name, 'rb')),
                 delete_file_at_to=lambda name: overleaf_client.delete_file(project["id"], project_infos, name),
@@ -121,13 +137,27 @@ def main(ctx, local, remote, project_name, cookie_path, sync_path, olignore_path
 @main.command()
 @click.option('--path', 'cookie_path', default=".olauth", type=click.Path(exists=False),
               help="Path to store the persisted Overleaf cookie.")
+@click.option('--cookie', 'use_cookie', is_flag=True,
+              help="Log in without the login window, using the `overleaf_session2` cookie from a browser that is "
+                   "logged in to Overleaf (asked for at a hidden prompt, or read from OLSYNC_COOKIE). Useful on "
+                   "servers without a display and for single sign-on accounts.")
 @click.option('-v', '--verbose', 'verbose', is_flag=True, help="Enable extended error logging.")
-def login(cookie_path, verbose):
+def login(cookie_path, use_cookie, verbose):
     if os.path.isfile(cookie_path) and not click.confirm(
             'Persisted Overleaf cookie already exist. Do you want to override it?'):
         return
+    if use_cookie:
+        # Never accept the cookie as a command line argument, it would end up in the shell history
+        cookie = os.environ.get("OLSYNC_COOKIE") or click.prompt("Value of the overleaf_session2 cookie",
+                                                                 hide_input=True)
+        execute_action(lambda: cookie_login_handler(cookie.strip(), cookie_path), "Login",
+                       "Login successful. Cookie persisted as `" + click.format_filename(
+                           cookie_path) + "`. You may now sync your project.",
+                       "Login failed: the cookie is not valid (expired, or copied incompletely).", verbose)
+        return
+    olbrowserlogin = load_browser_login()
     click.clear()
-    execute_action(lambda: login_handler(cookie_path), "Login",
+    execute_action(lambda: login_handler(olbrowserlogin, cookie_path), "Login",
                    "Login successful. Cookie persisted as `" + click.format_filename(
                        cookie_path) + "`. You may now sync your project.",
                    "Login failed. Please try again.", verbose)
@@ -203,7 +233,36 @@ def download_pdf(project_name, download_path, cookie_path, verbose):
                    "Downloading project's PDF failed. Please try again.", verbose)
 
 
-def login_handler(path):
+def load_browser_login():
+    """
+    Import the login window module. PySide6 is an optional dependency (the "gui" extra),
+    so it is only imported when logging in.
+    """
+    try:
+        try:
+            # Import for pip installation / wheel
+            import olsync.olbrowserlogin as olbrowserlogin
+        except ImportError as e:
+            if not (e.name or "").startswith("olsync"):
+                raise
+            # Import for development (running olsync.py from inside the olsync folder)
+            import olbrowserlogin
+    except ImportError as e:
+        if (e.name or "").startswith("PySide6"):
+            raise click.ClickException(
+                "The login window needs PySide6. Install it with: pip install \"overleaf-sync[gui]\"\n"
+                "Alternatively, log in without the window: ols login --cookie")
+        # PySide6 is installed but can't be loaded, e.g. because another Qt installation (such as Anaconda's)
+        # conflicts with it (#59)
+        raise click.ClickException(
+            f"The login window could not be loaded: {e}\n"
+            "This usually means another Qt installation (e.g. from Anaconda) conflicts with PySide6. Installing "
+            "overleaf-sync into a fresh virtual environment based on a non-conda Python usually helps.\n"
+            "Alternatively, log in without the window: ols login --cookie")
+    return olbrowserlogin
+
+
+def login_handler(olbrowserlogin, path):
     store = olbrowserlogin.login()
     if store is None:
         return False
@@ -212,6 +271,16 @@ def login_handler(path):
     return True
 
 
+def cookie_login_handler(cookie, path):
+    cookies = {"overleaf_session2": cookie}
+    csrf = OverleafClient(cookies).fetch_csrf()
+    if csrf is None:
+        return False
+    with open(path, 'wb+') as f:
+        pickle.dump({"cookie": cookies, "csrf": csrf}, f)
+    return True
+
+
 def delete_file(path):
     _dir = os.path.dirname(path)
     if _dir == path:
@@ -376,7 +445,8 @@ def olignore_keep_list(olignore_path):
     else:
         click.echo("\n.olignore: using %s to filter items" % olignore_path)
         with open(olignore_path, 'r') as f:
-            ignore_pattern = f.read().splitlines()
+            # "./main.tex" and "main.tex" mean the same file, matched paths have no "./" prefix (#72)
+            ignore_pattern = [pattern.removeprefix("./") for pattern in f.read().splitlines()]
 
         keep_list = [f for f in files if not any(
             fnmatch.fnmatch(f, ignore) for ignore in ignore_pattern)]

overleaf_sync-2.0.0a2/pyproject.toml

@@ -0,0 +1,60 @@
+[build-system]
+requires = ["flit_core >=3.4,<4"]
+build-backend = "flit_core.buildapi"
+
+[project]
+name = "overleaf-sync"
+authors = [{ name = "Moritz Glöckl", email = "moritzgloeckl@users.noreply.github.com" }]
+readme = "README.md"
+license = { file = "LICENSE" }
+classifiers = [
+    "License :: OSI Approved :: MIT License",
+    "Intended Audience :: Science/Research",
+    "Programming Language :: Python :: 3",
+]
+keywords = ["overleaf", "sync", "latex", "tex"]
+requires-python = ">=3.11"
+dynamic = ["version", "description"]
+dependencies = [
+    "requests >=2.31,<3",
+    "beautifulsoup4 >=4.12,<5",
+    "yaspin >=3,<4",
+    "python-dateutil >=2.8,<3",
+    "click >=8.1,<9",
+    "socketIO-client ==0.5.7.2",  # Do not upgrade, see AGENTS.md
+    "websocket-client >=1,<2",
+]
+
+[project.optional-dependencies]
+gui = ["PySide6 >=6.5,<7"]
+dev = ["pytest >=8", "pytest-cov >=5", "responses >=0.25", "ruff >=0.6"]
+
+[project.urls]
+Homepage = "https://github.com/moritzgloeckl/overleaf-sync"
+Issues = "https://github.com/moritzgloeckl/overleaf-sync/issues"
+
+[project.scripts]
+ols = "olsync.olsync:main"
+overleaf-sync = "olsync.olsync:main"
+
+[tool.flit.module]
+name = "olsync"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-m 'not integration and not smoke'"
+filterwarnings = ["ignore:color, on_color and attrs are not supported:UserWarning:yaspin"]
+markers = [
+    "integration: needs the Overleaf Community Edition container (tests/integration/compose.yml)",
+    "smoke: runs against overleaf.com with a real session, local only (see tests/smoke/)",
+]
+
+[tool.ruff]
+target-version = "py311"
+line-length = 120
+
+[tool.ruff.lint]
+select = ["E9", "F"]
+
+[tool.ruff.lint.per-file-ignores]
+"olsync/olbrowserlogin.py" = ["F403", "F405"]

overleaf-sync-1.2.0/.gitignore

@@ -1,118 +0,0 @@
-# Byte-compiled / optimized / DLL files
-__pycache__/
-*.py[cod]
-*$py.class
-
-# C extensions
-*.so
-
-# Distribution / packaging
-.Python
-build/
-develop-eggs/
-dist/
-downloads/
-eggs/
-.eggs/
-lib/
-lib64/
-parts/
-sdist/
-var/
-wheels/
-*.egg-info/
-.installed.cfg
-*.egg
-MANIFEST
-pip-wheel-metadata/
-
-# PyInstaller
-#  Usually these files are written by a python script from a template
-#  before PyInstaller builds the exe, so as to inject date/other infos into it.
-*.manifest
-*.spec
-
-# Installer logs
-pip-log.txt
-pip-delete-this-directory.txt
-
-# Unit test / coverage reports
-htmlcov/
-.tox/
-.coverage
-.coverage.*
-.cache
-nosetests.xml
-coverage.xml
-*.cover
-.hypothesis/
-.pytest_cache/
-
-# Translations
-*.mo
-*.pot
-
-# Django stuff:
-*.log
-local_settings.py
-db.sqlite3
-
-# Flask stuff:
-instance/
-.webassets-cache
-
-# Scrapy stuff:
-.scrapy
-
-# Sphinx documentation
-docs/_build/
-
-# PyBuilder
-target/
-
-# Jupyter Notebook
-.ipynb_checkpoints
-
-# pyenv
-.python-version
-
-# celery beat schedule file
-celerybeat-schedule
-
-# SageMath parsed files
-*.sage.py
-
-# Environments
-.env
-.venv
-env/
-venv/
-ENV/
-env.bak/
-venv.bak/
-
-# Spyder project settings
-.spyderproject
-.spyproject
-
-# Rope project settings
-.ropeproject
-
-# mkdocs documentation
-/site
-
-# mypy
-.mypy_cache/
-
-#PyCharm
-.idea/
-
-# vscode
-.vscode
-.DS_Store
-
-# dev testing
-setup.py
-test*
-.olauth
-.olignore

overleaf-sync-1.2.0/PKG-INFO

@@ -1,7 +0,0 @@
-Metadata-Version: 1.1
-Name: overleaf-sync
-Version: 1.2.0
-Summary: Overleaf Two-Way Sync Tool
-Home-page: https://github.com/moritzgloeckl/overleaf-sync
-Author: Moritz Glöckl
-Author-email: moritzgloeckl@users.noreply.github.com

overleaf-sync-1.2.0/pyproject.toml

@@ -1,26 +0,0 @@
-[build-system]
-requires = ["flit"]
-build-backend = "flit.buildapi"
-
-[tool.flit.metadata]
-module = "olsync"
-dist-name = "overleaf-sync"
-description-file = "README.md"
-author = "Moritz Glöckl"
-author-email = "moritzgloeckl@users.noreply.github.com"
-home-page = "https://github.com/moritzgloeckl/overleaf-sync"
-classifiers = ["License :: OSI Approved :: MIT License", "Intended Audience :: Science/Research", "Programming Language :: Python :: 3"]
-requires-python = ">=3"
-requires = [
-    "requests == 2.*",
-    "beautifulsoup4 == 4.11.1",
-    "yaspin == 2.*",
-    "python-dateutil~=2.8.1",
-    "click == 8.*",
-    "socketIO-client == 0.5.7.2",
-    "PySide6 == 6.*"
-]
-keywords = "overleaf sync latex tex"
-
-[tool.flit.scripts]
-ols = "olsync.olsync:main"

overleaf-sync-1.2.0/requirements.txt

@@ -1,7 +0,0 @@
-requests==2.*
-beautifulsoup4==4.11.1
-yaspin==2.*
-python-dateutil~=2.8.1
-click==8.*
-socketIO-client==0.5.7.2 # Do not upgrade!
-PySide6==6.*

overleaf-sync-1.2.0/setup.py

@@ -1,35 +0,0 @@
-#!/usr/bin/env python
-# setup.py generated by flit for tools that don't yet use PEP 517
-
-from distutils.core import setup
-
-packages = \
-['olsync']
-
-package_data = \
-{'': ['*']}
-
-install_requires = \
-['requests == 2.*',
- 'beautifulsoup4 == 4.11.1',
- 'yaspin == 2.*',
- 'python-dateutil~=2.8.1',
- 'click == 8.*',
- 'socketIO-client == 0.5.7.2',
- 'PySide6 == 6.*']
-
-entry_points = \
-{'console_scripts': ['ols = olsync.olsync:main']}
-
-setup(name='overleaf-sync',
-      version='1.2.0',
-      description='Overleaf Two-Way Sync Tool',
-      author='Moritz Glöckl',
-      author_email='moritzgloeckl@users.noreply.github.com',
-      url='https://github.com/moritzgloeckl/overleaf-sync',
-      packages=packages,
-      package_data=package_data,
-      install_requires=install_requires,
-      entry_points=entry_points,
-      python_requires='>=3',
-     )