LiveSync 0.2.1__py3-none-any.whl → 0.3.0__py3-none-any.whl

LiveSync-0.3.0.dist-info/METADATA

@@ -0,0 +1,100 @@
+Metadata-Version: 2.1
+Name: LiveSync
+Version: 0.3.0
+Summary: Repeatedly synchronize local workspace with a (slow) remote machine
+Home-page: https://github.com/zauberzeug/livesync
+Author: Zauberzeug GmbH
+Author-email: info@zauberzeug.com
+License: MIT
+Keywords: sync remote watch filesystem development deploy live hot reload
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pathspec
+Requires-Dist: pyjson5
+Requires-Dist: watchfiles
+
+# LiveSync
+
+Repeatedly synchronize local workspace with a (slow) remote machine.
+It is available as [PyPI package](https://pypi.org/project/livesync/) and hosted on [GitHub](https://github.com/zauberzeug/livesync).
+
+[![PyPI version](https://badge.fury.io/py/livesync.svg)](https://pypi.org/project/livesync/)
+[![PyPI - Downloads](https://img.shields.io/pypi/dm/livesync)](https://pypi.org/project/livesync/)
+[![GitHub commit activity](https://img.shields.io/github/commit-activity/m/zauberzeug/livesync)](https://github.com/zauberzeug/livesync/graphs/commit-activity)
+[![GitHub issues](https://img.shields.io/github/issues/zauberzeug/livesync)](https://github.com/zauberzeug/livesync/issues)
+[![GitHub license](https://img.shields.io/github/license/zauberzeug/livesync)](https://github.com/zauberzeug/livesync/blob/main/LICENSE)
+
+## Use Case
+
+[VS Code Remote Development](https://code.visualstudio.com/docs/remote/remote-overview) and similar tools are great as long as your remote machine is powerful enough.
+But if your target is a Raspberry Pi, Jetson Nano/Xavier/Orin, Beagle Board or similar, it feels like coding in jelly.
+Especially if you run powerful extensions like Pylance.
+LiveSync solves this by watching your code for changes and just copying the modifications to the slow remote machine.
+It works best if you have some kind of reload mechanism in place on the target ([NiceGUI](https://nicegui.io), [FastAPI](https://fastapi.tiangolo.com/) or [Flask](https://flask.palletsprojects.com/) for example).
+
+## Usage
+
+```bash
+livesync <source> <username>@<host>
+```
+
+LiveSync uses rsync (SSH) to copy the files, so the `<username>@<host>` must be accessible via SSH (ideally by key, not password or passphrase, because it will be called over and over).
+
+Press `CTRL-C` to abort the synchronization.
+
+Positional arguments:
+
+- `<source>`
+  local folder or VSCode workspace file
+- `<username>@<host>`
+  target user and host (e.g. username@hostname)
+
+Options:
+
+- `--target-root TARGET_ROOT`
+  subfolder on target to synchronize to (default: "")
+- `--target-port TARGET_PORT`
+  SSH port on target (default: 22)
+- `--on-change ON_CHANGE`
+  command to be executed on remote host after any file change (default: None)
+- `--mutex-interval MUTEX_INTERVAL`
+  interval in which mutex is updated (default: 10 seconds)
+
+### Notes
+
+- We suggest you have some auto-reloading in place on the (slow) target machine, like [NiceGUI](https://nicegui.io).
+- Only one user per target host should run LiveSync at a time. Therefore LiveSync provides a mutex mechanism.
+- You can create a `.syncignore` file in any source directory to skip additional files and directories from syncing.
+- If a `.syncignore` file doesn't exist, it is automatically created containing `.git/`, `__pycache__/`, `.DS_Store`, `*.tmp`, and `.env`.
+- If you pass a VSCode workspace file as `source`, LiveSync will synchronize each directory listed in the `folders` section.
+
+## Installation
+
+```bash
+python3 -m pip install livesync
+```
+
+## Development
+
+For development we suggest to use the following instructions instead of the normal pip installation:
+
+```bash
+git clone git@github.com:zauberzeug/livesync.git
+cd livesync
+python3 -m pip uninstall livesync # remove previous installed version
+python3 -m pip install -e .
+```
+
+Now you can change the code and call the `livesync` command from your `$PATH` variable with the modified code.
+
+## Testing
+
+We have build a small testing infrastructure with two docker containers.
+See [tests/README.md](https://github.com/zauberzeug/livesync/blob/main/tests/README.md) for details.
+
+## Releases
+
+Just create and push a new tag with the new version name (v0.2.1 for example).
+After a successful build a new release will be created.
+This should be edited to describe the changes in the release notes.

LiveSync-0.3.0.dist-info/RECORD

@@ -0,0 +1,10 @@
+livesync/__init__.py,sha256=V9nt0JRzIeiiLDVNDMZ20SqLMSqFh8uPViPsp2WYK6g,60
+livesync/folder.py,sha256=NhorutBdi6_0bhLCPjrVi8TIPZ3TPDQxJRQ0L0-3kRU,3619
+livesync/livesync.py,sha256=PKaw-WD4WOZb-X2n4ihj0C8jEOff4s5dopOGiOM3xI4,2586
+livesync/mutex.py,sha256=C3Az3Exfgj1ohKnhWl77TDvuJlv3q2qieSgtJF__Wdc,1646
+LiveSync-0.3.0.dist-info/LICENSE,sha256=QcBlwggRQYhvfTAE481AAMFQfMS_N6Bj8Svh1T6dsnI,1072
+LiveSync-0.3.0.dist-info/METADATA,sha256=pjVKS1x7LaYr_emVuhAwMUIQyK-WKxJYzPUhnVMs9fc,4285
+LiveSync-0.3.0.dist-info/WHEEL,sha256=yQN5g4mg4AybRjkgi-9yy4iQEFibGQmlz78Pik5Or-A,92
+LiveSync-0.3.0.dist-info/entry_points.txt,sha256=4dn5YR27lUlJWea3yqLKLqR4MwqjcqzMR5Q4jVFnytI,52
+LiveSync-0.3.0.dist-info/top_level.txt,sha256=mLwExc6wTUGqxvUkYMio5rxGS1h8bvxpNsR2ebfjSL4,9
+LiveSync-0.3.0.dist-info/RECORD,,

{LiveSync-0.2.1.dist-info → LiveSync-0.3.0.dist-info}/WHEEL

@@ -1,5 +1,5 @@
 Wheel-Version: 1.0
-Generator: bdist_wheel (0.37.1)
+Generator: bdist_wheel (0.41.2)
 Root-Is-Purelib: true
 Tag: py3-none-any
 

livesync/__init__.py

@@ -1,2 +1,2 @@
-from .folder import Folder
+from .folder import Folder, Target
 from .mutex import Mutex

livesync/folder.py

@@ -1,63 +1,76 @@
 import asyncio
-import os
 import subprocess
+import sys
+from dataclasses import dataclass
+from pathlib import Path
 from typing import List, Optional
 
 import pathspec
 import watchfiles
 
+KWONLY_SLOTS = {'kw_only': True, 'slots': True} if sys.version_info >= (3, 10) else {}
+DEFAULT_IGNORES = ['.git/', '__pycache__/', '.DS_Store', '*.tmp', '.env']
+
+
+def run_subprocess(command: str, *, quiet: bool = False) -> None:
+    result = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, check=True)
+    if not quiet:
+        print(result.stdout.decode())
+
+
+@dataclass(**KWONLY_SLOTS)
+class Target:
+    host: str
+    port: int
+    root: Path
+
+    def make_target_root_directory(self) -> None:
+        print(f'make target root directory {self.root}')
+        run_subprocess(f'ssh {self.host} -p {self.port} "mkdir -p {self.root}"')
+
 
 class Folder:
 
-    def __init__(self, local_dir: str, target_host: str) -> None:
-        self.local_dir = os.path.abspath(local_dir)
-        self.target_host = target_host
+    def __init__(self, local_dir: Path, target: Target) -> None:
+        self.local_path = local_dir.resolve()  # one should avoid `absolute` if Python < 3.11
+        self.target = target
 
-        # https://stackoverflow.com/a/22090594/3419103
-        self._ignore_spec = pathspec.PathSpec.from_lines(pathspec.patterns.GitWildMatchPattern, self.get_excludes())
+        # from https://stackoverflow.com/a/22090594/3419103
+        match_pattern = pathspec.patterns.gitwildmatch.GitWildMatchPattern
+        self._ignore_spec = pathspec.PathSpec.from_lines(match_pattern, self.get_ignores())
 
         self._stop_watching = asyncio.Event()
 
     @property
-    def target_path(self) -> str:
-        return os.path.basename(os.path.realpath(self.local_dir))
-
-    @property
-    def ssh_target(self) -> str:
-        return f'{self.target_host}:{self.target_path}'
+    def target_path(self) -> Path:
+        return self.target.root / self.local_path.stem
 
     @property
-    def is_valid(self) -> bool:
-        return os.path.isdir(self.local_dir)
-
-    def get_excludes(self) -> List[str]:
-        return ['.git/', '__pycache__/', '.DS_Store'] + \
-            self._parse_ignore_file(f'{self.local_dir}/.syncignore') + \
-            self._parse_ignore_file(f'{self.local_dir}/.gitignore')
+    def ssh_path(self) -> str:
+        return f'{self.target.host}:{self.target_path}'
 
-    @staticmethod
-    def _parse_ignore_file(path: str) -> List[str]:
-        if not os.path.isfile(path):
-            return []
-        with open(path) as f:
-            return [line.strip() for line in f.readlines() if not line.startswith('#')]
+    def get_ignores(self) -> List[str]:
+        path = self.local_path / '.syncignore'
+        if not path.is_file():
+            path.write_text('\n'.join(DEFAULT_IGNORES))
+        return [line.strip() for line in path.read_text().splitlines() if not line.startswith('#')]
 
     def get_summary(self) -> str:
-        summary = f'{self.local_dir} --> {self.ssh_target}\n'
-        if not os.path.exists(os.path.join(self.local_dir, '.git')):
+        summary = f'{self.local_path} --> {self.ssh_path}\n'
+        if not (self.local_path / '.git').exists():
             return summary
         try:
             cmd = ['git', 'log', '--pretty=format:[%h]\n', '-n', '1']
-            summary += subprocess.check_output(cmd, cwd=self.local_dir).decode()
+            summary += subprocess.check_output(cmd, cwd=self.local_path).decode()
             cmd = ['git', 'status', '--short', '--branch']
-            summary += subprocess.check_output(cmd, cwd=self.local_dir).decode().strip()
-        except:
+            summary += subprocess.check_output(cmd, cwd=self.local_path).decode().strip() + '\n'
+        except Exception:
             pass  # maybe git is not installed
         return summary
 
     async def watch(self, on_change_command: Optional[str]) -> None:
         try:
-            async for changes in watchfiles.awatch(self.local_dir, stop_event=self._stop_watching,
+            async for changes in watchfiles.awatch(self.local_path, stop_event=self._stop_watching,
                                                    watch_filter=lambda _, filepath: not self._ignore_spec.match_file(filepath)):
                 for change, filepath in changes:
                     print('?+U-'[change], filepath)
@@ -71,8 +84,9 @@ class Folder:
 
     def sync(self, post_sync_command: Optional[str] = None) -> None:
         args = '--prune-empty-dirs --delete -avz --checksum --no-t'
-        args += ''.join(f' --exclude="{e}"' for e in self.get_excludes())
-        command = f'rsync {args} {self.local_dir}/ {self.ssh_target}'
+        # args += ' --mkdirs'  # INFO: this option is not available in rsync < 3.2.3
+        args += ''.join(f' --exclude="{e}"' for e in self.get_ignores())
+        args += f' -e "ssh -p {self.target.port}"'
+        run_subprocess(f'rsync {args} {self.local_path}/ {self.ssh_path}/', quiet=True)
         if post_sync_command:
-            command += f'; ssh {self.target_host} "cd {self.target_path}; {post_sync_command}"'
-        subprocess.run(command, shell=True, stdout=subprocess.DEVNULL)
+            run_subprocess(f'ssh {self.target.host} -p {self.target.port} "cd {self.target_path}; {post_sync_command}"')

livesync/livesync.py

@@ -1,12 +1,13 @@
 #!/usr/bin/env python3
 import argparse
 import asyncio
-import json
 import sys
-from glob import glob
+from pathlib import Path
 from typing import List
 
-from livesync import Folder, Mutex
+import pyjson5
+
+from livesync import Folder, Mutex, Target
 
 
 def git_summary(folders: List[Folder]) -> str:
@@ -14,36 +15,45 @@ def git_summary(folders: List[Folder]) -> str:
 
 
 async def async_main() -> None:
-    parser = argparse.ArgumentParser(description='Repeatedly synchronize local workspace with remote machine')
+    parser = argparse.ArgumentParser(
+        description='Repeatedly synchronize local directories with remote machine',
+        formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+    parser.add_argument('source', type=str, help='local source folder or VSCode workspace file')
+    parser.add_argument('--target-root', type=str, default='', help='subfolder on target to synchronize to')
+    parser.add_argument('--target-port', type=int, default=22, help='SSH port on target')
     parser.add_argument('--on-change', type=str, help='command to be executed on remote host after any file change')
-    parser.add_argument('--source', type=str, help='source folder on local host instead of vscode workspace file')
-    parser.add_argument('--mutex-interval', type=int, nargs='?', default=10, help='interval in which mutex is updated')
+    parser.add_argument('--mutex-interval', type=int, default=10, help='interval in which mutex is updated')
     parser.add_argument('host', type=str, help='the target host (e.g. username@hostname)')
     args = parser.parse_args()
+    source = Path(args.source)
+    target = Target(host=args.host, port=args.target_port, root=Path(args.target_root))
 
     folders: List[Folder] = []
-    workspaces = glob('*.code-workspace')
-    if args.source is None and workspaces:
-        print(f'Reading vscode workspace file {workspaces[0]} ...')
-        try:
-            with open(workspaces[0]) as f:
-                workspace = json.load(f)
-                folders = [Folder(folder['path'], args.host) for folder in workspace['folders']]
-        except IndexError:
-            print('No vscode workspace file found; provide --source parameter or start in dir with *.code-workspace file')
-            sys.exit(1)
+    if source.is_file():
+        workspace = pyjson5.decode(source.read_text())
+        paths = [Path(f['path']) for f in workspace['folders']]
+        folders = [Folder(p, target) for p in paths if p.is_dir()]
     else:
-        folders = [Folder(args.source or '.', args.host)]
+        folders = [Folder(source, target)]
+
+    for folder in folders:
+        if not folder.local_path.is_dir():
+            print(f'Invalid path: {folder.local_path}')
+            sys.exit(1)
 
     print('Checking mutex...')
-    mutex = Mutex(args.host)
+    mutex = Mutex(target)
     if not mutex.set(git_summary(folders)):
         print(f'Target is in use by {mutex.occupant}')
         sys.exit(1)
 
+    if args.target_root:
+        print('Creating target root directory...')
+        target.make_target_root_directory()
+
     print('Initial sync...')
     for folder in folders:
-        print(f'  {folder.local_dir} --> {folder.ssh_target}')
+        print(f'  {folder.local_path} --> {folder.ssh_path}')
         folder.sync(post_sync_command=args.on_change)
 
     print('Watching for file changes...')

livesync/mutex.py

@@ -2,28 +2,30 @@ import logging
 import socket
 import subprocess
 from datetime import datetime, timedelta
+from typing import Optional
+
+from .folder import Target
 
 MUTEX_FILEPATH = '~/.livesync_mutex'
 
 
 class Mutex:
 
-    def __init__(self, host: str) -> None:
-        self.host = host
-        self.occupant: str = None
+    def __init__(self, target: Target) -> None:
+        self.target = target
+        self.occupant: Optional[str] = None
         self.user_id = socket.gethostname()
 
     def is_free(self, info: str) -> bool:
         try:
-            command = ['ssh', self.host, f'cat {MUTEX_FILEPATH} || echo "{self.tag}\n{info}"']
-            output = subprocess.check_output(command, stderr=subprocess.DEVNULL).decode().splitlines()[0]
+            output = self._run_ssh_command(f'cat {MUTEX_FILEPATH} || echo "{self.tag}\n{info}"').splitlines()[0]
             words = output.strip().split()
             self.occupant = words[0]
             occupant_ok = self.occupant == self.user_id
             mutex_datetime = datetime.fromisoformat(words[1])
             mutex_expired = datetime.now() - mutex_datetime > timedelta(seconds=15)
             return occupant_ok or mutex_expired
-        except:
+        except Exception:
             logging.exception('Could not access target system')
             return False
 
@@ -31,8 +33,7 @@ class Mutex:
         if not self.is_free(info):
             return False
         try:
-            command = ['ssh', self.host, f'echo "{self.tag}\n{info}" > {MUTEX_FILEPATH}']
-            subprocess.check_output(command, stderr=subprocess.DEVNULL)
+            self._run_ssh_command(f'echo "{self.tag}\n{info}" > {MUTEX_FILEPATH}')
             return True
         except subprocess.CalledProcessError:
             print('Could not write mutex file')
@@ -41,3 +42,7 @@ class Mutex:
     @property
     def tag(self) -> str:
         return f'{self.user_id} {datetime.now().isoformat()}'
+
+    def _run_ssh_command(self, command: str) -> str:
+        ssh_command = ['ssh', self.target.host, '-p', str(self.target.port), command]
+        return subprocess.check_output(ssh_command, stderr=subprocess.DEVNULL).decode()

LiveSync-0.2.1.dist-info/METADATA

@@ -1,67 +0,0 @@
-Metadata-Version: 2.1
-Name: LiveSync
-Version: 0.2.1
-Summary: Repeatedly synchronize local workspace with a (slow) remote machine
-Home-page: https://github.com/zauberzeug/livesync
-Author: Zauberzeug GmbH
-Author-email: info@zauberzeug.com
-License: MIT
-Keywords: sync remote watch filesystem development deploy live hot reload
-Requires-Python: >=3.7
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: watchfiles
-Requires-Dist: pathspec
-
-# LiveSync
-
-Repeatedly synchronize local workspace with a (slow) remote machine.
-
-## Use Case
-
-[VS Code Remote Development](https://code.visualstudio.com/docs/remote/remote-overview) and similar tools are great as long as your remote machine is powerful enough.
-But if your target is a Raspberry Pi, Jetson Nano/Xavier/Orin, Beagle Board or similar, it feels like coding in yelly.
-Especially if you run powerful extensions like Pylance.
-LiveSync solves this by watching your code and just copying the changed files to the slow remote machine.
-It works best if you have some kind of reloading mechanism in place on the target.
-We obviously recommend [NiceGUI](https://nicegui.io).
-
-## Install
-
-```bash
-python3 -m pip install livesync
-```
-
-## Usage
-
-```bash
-cd <my_folder_with_vscode_workspace>
-livesync <username>@<host>
-```
-
-LiveSync uses rsync (SSH) to copy the files, so the `<username>@<host>` must be accessible via SSH (ideally by key, not password or passphrase, because it will be called over and over).
-
-Press `CTRL-C` to abort the synchronization.
-
-### Notes
-
-- We suggest you have some auto-reloading in place on the (slow) target machine, like [NiceGUI](https://nicegui.io).
-- Only one user per target host can run LiveSync at a time.
-- By default `.git/` folders are not synchronized.
-- All files and directories from the `.gitignore` of any source directory are also excluded from synchronization.
-- You can create a `.syncignore` file in any source directory to skip additional files and directories from syncing.
-
-## Development
-
-```bash
-git clone git@github.com:zauberzeug/livesync.git
-cd livesync
-python3 -m pip uninstall livesync # remove previous installed version
-python3 -m pip install -e .
-```
-
-Now you can change the code and still use the `livesync` command from your `$PATH` variable.
-
-## Releases
-
-Just create and push a new tag with the new version name.

LiveSync-0.2.1.dist-info/RECORD

@@ -1,10 +0,0 @@
-livesync/__init__.py,sha256=8EPS0Mjy54QwW5ak8L8uv13ZCub0-WCYa1KB0UqilGI,52
-livesync/folder.py,sha256=qJv3oEOfZUANle1GAFpiYWOZAIBPIp_DgV0C9kflI7Y,3010
-livesync/livesync.py,sha256=SNUuijoFRFBJfmUyAK3nQxExFTbL87JbLwJlhQT9eU0,2264
-livesync/mutex.py,sha256=venXO1Nnj8msN185e2QqhCpc_saNY-lQFNeSlZNMiPE,1500
-LiveSync-0.2.1.dist-info/LICENSE,sha256=QcBlwggRQYhvfTAE481AAMFQfMS_N6Bj8Svh1T6dsnI,1072
-LiveSync-0.2.1.dist-info/METADATA,sha256=8RyxXLrHYssPDWd_jF-IgOOX-0akFPPYmEhWiya7dTo,2342
-LiveSync-0.2.1.dist-info/WHEEL,sha256=G16H4A3IeoQmnOrYV4ueZGKSjhipXx8zc8nu9FGlvMA,92
-LiveSync-0.2.1.dist-info/entry_points.txt,sha256=4dn5YR27lUlJWea3yqLKLqR4MwqjcqzMR5Q4jVFnytI,52
-LiveSync-0.2.1.dist-info/top_level.txt,sha256=mLwExc6wTUGqxvUkYMio5rxGS1h8bvxpNsR2ebfjSL4,9
-LiveSync-0.2.1.dist-info/RECORD,,