rclone-api 1.0.15__tar.gz → 1.0.17__tar.gz

Sign up to get free protection for your applications and to get access to all the features.
Files changed (54) hide show
  1. {rclone_api-1.0.15 → rclone_api-1.0.17}/.gitignore +3 -1
  2. rclone_api-1.0.17/PKG-INFO +149 -0
  3. rclone_api-1.0.17/README.md +133 -0
  4. {rclone_api-1.0.15 → rclone_api-1.0.17}/pyproject.toml +1 -1
  5. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/__init__.py +2 -0
  6. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/exec.py +13 -0
  7. rclone_api-1.0.17/src/rclone_api/process.py +124 -0
  8. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/rclone.py +39 -0
  9. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/util.py +2 -2
  10. rclone_api-1.0.17/src/rclone_api.egg-info/PKG-INFO +149 -0
  11. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api.egg-info/SOURCES.txt +2 -0
  12. rclone_api-1.0.17/tests/test_mount.py +105 -0
  13. rclone_api-1.0.15/PKG-INFO +0 -34
  14. rclone_api-1.0.15/README.md +0 -18
  15. rclone_api-1.0.15/src/rclone_api.egg-info/PKG-INFO +0 -34
  16. {rclone_api-1.0.15 → rclone_api-1.0.17}/.aiderignore +0 -0
  17. {rclone_api-1.0.15 → rclone_api-1.0.17}/.github/workflows/lint.yml +0 -0
  18. {rclone_api-1.0.15 → rclone_api-1.0.17}/.github/workflows/push_macos.yml +0 -0
  19. {rclone_api-1.0.15 → rclone_api-1.0.17}/.github/workflows/push_ubuntu.yml +0 -0
  20. {rclone_api-1.0.15 → rclone_api-1.0.17}/.github/workflows/push_win.yml +0 -0
  21. {rclone_api-1.0.15 → rclone_api-1.0.17}/.pylintrc +0 -0
  22. {rclone_api-1.0.15 → rclone_api-1.0.17}/.vscode/launch.json +0 -0
  23. {rclone_api-1.0.15 → rclone_api-1.0.17}/.vscode/settings.json +0 -0
  24. {rclone_api-1.0.15 → rclone_api-1.0.17}/.vscode/tasks.json +0 -0
  25. {rclone_api-1.0.15 → rclone_api-1.0.17}/LICENSE +0 -0
  26. {rclone_api-1.0.15 → rclone_api-1.0.17}/MANIFEST.in +0 -0
  27. {rclone_api-1.0.15 → rclone_api-1.0.17}/clean +0 -0
  28. {rclone_api-1.0.15 → rclone_api-1.0.17}/install +0 -0
  29. {rclone_api-1.0.15 → rclone_api-1.0.17}/lint +0 -0
  30. {rclone_api-1.0.15 → rclone_api-1.0.17}/requirements.testing.txt +0 -0
  31. {rclone_api-1.0.15 → rclone_api-1.0.17}/setup.cfg +0 -0
  32. {rclone_api-1.0.15 → rclone_api-1.0.17}/setup.py +0 -0
  33. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/assets/example.txt +0 -0
  34. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/cli.py +0 -0
  35. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/config.py +0 -0
  36. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/convert.py +0 -0
  37. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/dir.py +0 -0
  38. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/dir_listing.py +0 -0
  39. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/file.py +0 -0
  40. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/filelist.py +0 -0
  41. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/remote.py +0 -0
  42. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/rpath.py +0 -0
  43. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api/walk.py +0 -0
  44. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api.egg-info/dependency_links.txt +0 -0
  45. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api.egg-info/requires.txt +0 -0
  46. {rclone_api-1.0.15 → rclone_api-1.0.17}/src/rclone_api.egg-info/top_level.txt +0 -0
  47. {rclone_api-1.0.15 → rclone_api-1.0.17}/test +0 -0
  48. {rclone_api-1.0.15 → rclone_api-1.0.17}/tests/test_copy.py +0 -0
  49. {rclone_api-1.0.15 → rclone_api-1.0.17}/tests/test_is_synced.py +0 -0
  50. {rclone_api-1.0.15 → rclone_api-1.0.17}/tests/test_ls.py +0 -0
  51. {rclone_api-1.0.15 → rclone_api-1.0.17}/tests/test_remotes.py +0 -0
  52. {rclone_api-1.0.15 → rclone_api-1.0.17}/tests/test_walk.py +0 -0
  53. {rclone_api-1.0.15 → rclone_api-1.0.17}/tox.ini +0 -0
  54. {rclone_api-1.0.15 → rclone_api-1.0.17}/upload_package.sh +0 -0
@@ -142,4 +142,6 @@ uv.lock
142
142
  .aider*
143
143
 
144
144
  !.aider.conf.yml
145
- !.aiderignore
145
+ !.aiderignore
146
+
147
+ rclone.conf
@@ -0,0 +1,149 @@
1
+ Metadata-Version: 2.2
2
+ Name: rclone_api
3
+ Version: 1.0.17
4
+ Summary: rclone api in python
5
+ Home-page: https://github.com/zackees/rclone-api
6
+ Maintainer: Zachary Vorhies
7
+ License: BSD 3-Clause License
8
+ Keywords: template-python-cmd
9
+ Classifier: Programming Language :: Python :: 3
10
+ Requires-Python: >=3.10
11
+ Description-Content-Type: text/markdown
12
+ License-File: LICENSE
13
+ Requires-Dist: python-dotenv>=1.0.0
14
+ Dynamic: home-page
15
+ Dynamic: maintainer
16
+
17
+ # rclone-api
18
+
19
+ [![Linting](https://github.com/zackees/rclone-api/actions/workflows/lint.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/lint.yml)
20
+ [![MacOS_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml)
21
+ [![Ubuntu_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml)
22
+ [![Win_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml)
23
+
24
+ Api version of rclone. It's well tested. It's a pretty low level api without the bells and whistles of other apis, but it will get the job done.
25
+
26
+ You will need to have rclone installed and on your path.
27
+
28
+ One of the benefits of this api is that it does not use `shell=True`, which can keep `rclone` running in some instances even when try to kill the process.
29
+
30
+ # Install
31
+
32
+ `pip install rclone-api`
33
+
34
+
35
+ # Examples
36
+
37
+ You can use env variables or use a `.env` file to store your secrets.
38
+
39
+
40
+ # Rclone API Usage Examples
41
+
42
+ This script demonstrates how to interact with DigitalOcean Spaces using `rclone_api`.
43
+
44
+ ## Setup & Usage
45
+
46
+ Ensure you have set the required environment variables:
47
+
48
+ - `BUCKET_NAME`
49
+ - `BUCKET_KEY_PUBLIC`
50
+ - `BUCKET_KEY_SECRET`
51
+ - `BUCKET_URL`
52
+
53
+ Then, run the following Python script:
54
+
55
+ ```python
56
+ import os
57
+ from rclone_api import Config, DirListing, File, Rclone, Remote
58
+
59
+ # Load environment variables
60
+ BUCKET_NAME = os.getenv("BUCKET_NAME")
61
+ BUCKET_KEY_PUBLIC = os.getenv("BUCKET_KEY_PUBLIC")
62
+ BUCKET_KEY_SECRET = os.getenv("BUCKET_KEY_SECRET")
63
+ BUCKET_URL = "sfo3.digitaloceanspaces.com"
64
+
65
+ # Generate Rclone Configuration
66
+ def generate_rclone_config() -> Config:
67
+ config_text = f"""
68
+ [dst]
69
+ type = s3
70
+ provider = DigitalOcean
71
+ access_key_id = {BUCKET_KEY_PUBLIC}
72
+ secret_access_key = {BUCKET_KEY_SECRET}
73
+ endpoint = {BUCKET_URL}
74
+ """
75
+ return Config(config_text)
76
+
77
+ rclone = Rclone(generate_rclone_config())
78
+
79
+ # List Available Remotes
80
+ print("\n=== Available Remotes ===")
81
+ remotes = rclone.listremotes()
82
+ for remote in remotes:
83
+ print(remote)
84
+
85
+ # List Contents of the Root Bucket
86
+ print("\n=== Listing Root Bucket ===")
87
+ listing = rclone.ls(f"dst:{BUCKET_NAME}", max_depth=-1)
88
+
89
+ print("\nDirectories:")
90
+ for dir in listing.dirs:
91
+ print(dir)
92
+
93
+ print("\nFiles:")
94
+ for file in listing.files:
95
+ print(file)
96
+
97
+ # List a Specific Subdirectory
98
+ print("\n=== Listing 'zachs_video' Subdirectory ===")
99
+ path = f"dst:{BUCKET_NAME}/zachs_video"
100
+ listing = rclone.ls(path)
101
+ print(listing)
102
+
103
+ # List PNG Files in a Subdirectory
104
+ print("\n=== Listing PNG Files ===")
105
+ listing = rclone.ls(path, glob="*.png")
106
+
107
+ if listing.files:
108
+ for file in listing.files:
109
+ print(file)
110
+
111
+ # Copy a File
112
+ print("\n=== Copying a File ===")
113
+ if listing.files:
114
+ file = listing.files[0]
115
+ new_path = f"dst:{BUCKET_NAME}/zachs_video/{file.name}_copy"
116
+ rclone.copyfile(file, new_path)
117
+ print(f"Copied {file.name} to {new_path}")
118
+
119
+ # Copy Multiple Files
120
+ print("\n=== Copying Multiple Files ===")
121
+ if listing.files:
122
+ file_mapping = {file.name: file.name + "_copy" for file in listing.files[:2]}
123
+ rclone.copyfiles(file_mapping)
124
+ print(f"Copied files: {file_mapping}")
125
+
126
+ # Delete a File
127
+ print("\n=== Deleting a File ===")
128
+ file_to_delete = f"dst:{BUCKET_NAME}/zachs_video/sample.png_copy"
129
+ rclone.deletefiles([file_to_delete])
130
+ print(f"Deleted {file_to_delete}")
131
+
132
+ # Walk Through a Directory
133
+ print("\n=== Walking Through a Directory ===")
134
+ for dirlisting in rclone.walk(f"dst:{BUCKET_NAME}", max_depth=1):
135
+ print(dirlisting)
136
+
137
+ print("Done.")
138
+ ```
139
+
140
+
141
+ To develop software, run `. ./activate`
142
+
143
+ # Windows
144
+
145
+ This environment requires you to use `git-bash`.
146
+
147
+ # Linting
148
+
149
+ Run `./lint`
@@ -0,0 +1,133 @@
1
+ # rclone-api
2
+
3
+ [![Linting](https://github.com/zackees/rclone-api/actions/workflows/lint.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/lint.yml)
4
+ [![MacOS_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml)
5
+ [![Ubuntu_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml)
6
+ [![Win_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml)
7
+
8
+ Api version of rclone. It's well tested. It's a pretty low level api without the bells and whistles of other apis, but it will get the job done.
9
+
10
+ You will need to have rclone installed and on your path.
11
+
12
+ One of the benefits of this api is that it does not use `shell=True`, which can keep `rclone` running in some instances even when try to kill the process.
13
+
14
+ # Install
15
+
16
+ `pip install rclone-api`
17
+
18
+
19
+ # Examples
20
+
21
+ You can use env variables or use a `.env` file to store your secrets.
22
+
23
+
24
+ # Rclone API Usage Examples
25
+
26
+ This script demonstrates how to interact with DigitalOcean Spaces using `rclone_api`.
27
+
28
+ ## Setup & Usage
29
+
30
+ Ensure you have set the required environment variables:
31
+
32
+ - `BUCKET_NAME`
33
+ - `BUCKET_KEY_PUBLIC`
34
+ - `BUCKET_KEY_SECRET`
35
+ - `BUCKET_URL`
36
+
37
+ Then, run the following Python script:
38
+
39
+ ```python
40
+ import os
41
+ from rclone_api import Config, DirListing, File, Rclone, Remote
42
+
43
+ # Load environment variables
44
+ BUCKET_NAME = os.getenv("BUCKET_NAME")
45
+ BUCKET_KEY_PUBLIC = os.getenv("BUCKET_KEY_PUBLIC")
46
+ BUCKET_KEY_SECRET = os.getenv("BUCKET_KEY_SECRET")
47
+ BUCKET_URL = "sfo3.digitaloceanspaces.com"
48
+
49
+ # Generate Rclone Configuration
50
+ def generate_rclone_config() -> Config:
51
+ config_text = f"""
52
+ [dst]
53
+ type = s3
54
+ provider = DigitalOcean
55
+ access_key_id = {BUCKET_KEY_PUBLIC}
56
+ secret_access_key = {BUCKET_KEY_SECRET}
57
+ endpoint = {BUCKET_URL}
58
+ """
59
+ return Config(config_text)
60
+
61
+ rclone = Rclone(generate_rclone_config())
62
+
63
+ # List Available Remotes
64
+ print("\n=== Available Remotes ===")
65
+ remotes = rclone.listremotes()
66
+ for remote in remotes:
67
+ print(remote)
68
+
69
+ # List Contents of the Root Bucket
70
+ print("\n=== Listing Root Bucket ===")
71
+ listing = rclone.ls(f"dst:{BUCKET_NAME}", max_depth=-1)
72
+
73
+ print("\nDirectories:")
74
+ for dir in listing.dirs:
75
+ print(dir)
76
+
77
+ print("\nFiles:")
78
+ for file in listing.files:
79
+ print(file)
80
+
81
+ # List a Specific Subdirectory
82
+ print("\n=== Listing 'zachs_video' Subdirectory ===")
83
+ path = f"dst:{BUCKET_NAME}/zachs_video"
84
+ listing = rclone.ls(path)
85
+ print(listing)
86
+
87
+ # List PNG Files in a Subdirectory
88
+ print("\n=== Listing PNG Files ===")
89
+ listing = rclone.ls(path, glob="*.png")
90
+
91
+ if listing.files:
92
+ for file in listing.files:
93
+ print(file)
94
+
95
+ # Copy a File
96
+ print("\n=== Copying a File ===")
97
+ if listing.files:
98
+ file = listing.files[0]
99
+ new_path = f"dst:{BUCKET_NAME}/zachs_video/{file.name}_copy"
100
+ rclone.copyfile(file, new_path)
101
+ print(f"Copied {file.name} to {new_path}")
102
+
103
+ # Copy Multiple Files
104
+ print("\n=== Copying Multiple Files ===")
105
+ if listing.files:
106
+ file_mapping = {file.name: file.name + "_copy" for file in listing.files[:2]}
107
+ rclone.copyfiles(file_mapping)
108
+ print(f"Copied files: {file_mapping}")
109
+
110
+ # Delete a File
111
+ print("\n=== Deleting a File ===")
112
+ file_to_delete = f"dst:{BUCKET_NAME}/zachs_video/sample.png_copy"
113
+ rclone.deletefiles([file_to_delete])
114
+ print(f"Deleted {file_to_delete}")
115
+
116
+ # Walk Through a Directory
117
+ print("\n=== Walking Through a Directory ===")
118
+ for dirlisting in rclone.walk(f"dst:{BUCKET_NAME}", max_depth=1):
119
+ print(dirlisting)
120
+
121
+ print("Done.")
122
+ ```
123
+
124
+
125
+ To develop software, run `. ./activate`
126
+
127
+ # Windows
128
+
129
+ This environment requires you to use `git-bash`.
130
+
131
+ # Linting
132
+
133
+ Run `./lint`
@@ -14,7 +14,7 @@ dependencies = [
14
14
  "python-dotenv>=1.0.0"
15
15
  ]
16
16
  # Change this with the version number bump.
17
- version = "1.0.15"
17
+ version = "1.0.17"
18
18
 
19
19
  [tool.setuptools]
20
20
  package-dir = {"" = "src"}
@@ -3,6 +3,7 @@ from .dir import Dir
3
3
  from .dir_listing import DirListing
4
4
  from .file import File
5
5
  from .filelist import FileList
6
+ from .process import Process
6
7
  from .rclone import Rclone
7
8
  from .remote import Remote
8
9
  from .rpath import RPath
@@ -16,4 +17,5 @@ __all__ = [
16
17
  "RPath",
17
18
  "DirListing",
18
19
  "FileList",
20
+ "Process",
19
21
  ]
@@ -3,6 +3,7 @@ from dataclasses import dataclass
3
3
  from pathlib import Path
4
4
 
5
5
  from rclone_api.config import Config
6
+ from rclone_api.process import Process, ProcessArgs
6
7
 
7
8
 
8
9
  @dataclass
@@ -17,3 +18,15 @@ class RcloneExec:
17
18
  from rclone_api.util import rclone_execute
18
19
 
19
20
  return rclone_execute(cmd, self.rclone_config, self.rclone_exe, check=check)
21
+
22
+ def launch_process(self, cmd: list[str]) -> Process:
23
+ """Launch rclone process."""
24
+
25
+ args: ProcessArgs = ProcessArgs(
26
+ cmd=cmd,
27
+ rclone_conf=self.rclone_config,
28
+ rclone_exe=self.rclone_exe,
29
+ cmd_list=cmd,
30
+ )
31
+ process = Process(args)
32
+ return process
@@ -0,0 +1,124 @@
1
+ import os
2
+ import subprocess
3
+ from dataclasses import dataclass
4
+ from pathlib import Path
5
+ from tempfile import TemporaryDirectory
6
+ from typing import Any
7
+
8
+ from rclone_api.config import Config
9
+ from rclone_api.util import get_verbose
10
+
11
+ # def rclone_launch_process(
12
+ # cmd: list[str],
13
+ # rclone_conf: Path | Config,
14
+ # rclone_exe: Path,
15
+ # verbose: bool | None = None,
16
+ # ) -> subprocess.Popen:
17
+ # tempdir: TemporaryDirectory | None = None
18
+ # verbose = _get_verbose(verbose)
19
+ # assert verbose is not None
20
+
21
+ # try:
22
+ # if isinstance(rclone_conf, Config):
23
+ # tempdir = TemporaryDirectory()
24
+ # tmpfile = Path(tempdir.name) / "rclone.conf"
25
+ # tmpfile.write_text(rclone_conf.text, encoding="utf-8")
26
+ # rclone_conf = tmpfile
27
+ # cmd = (
28
+ # [str(rclone_exe.resolve())] + ["--config", str(rclone_conf.resolve())] + cmd
29
+ # )
30
+ # if verbose:
31
+ # cmd_str = subprocess.list2cmdline(cmd)
32
+ # print(f"Running: {cmd_str}")
33
+ # cp = subprocess.Popen(
34
+ # cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=False
35
+ # )
36
+ # return cp
37
+ # finally:
38
+ # if tempdir:
39
+ # try:
40
+ # tempdir.cleanup()
41
+ # except Exception as e:
42
+ # print(f"Error cleaning up tempdir: {e}")
43
+
44
+
45
+ def _get_verbose(verbose: bool | None) -> bool:
46
+ if verbose is not None:
47
+ return verbose
48
+ # get it from the environment
49
+ return bool(int(os.getenv("RCLONE_API_VERBOSE", "0")))
50
+
51
+
52
+ @dataclass
53
+ class ProcessArgs:
54
+ cmd: list[str]
55
+ rclone_conf: Path | Config
56
+ rclone_exe: Path
57
+ cmd_list: list[str]
58
+ verbose: bool | None = None
59
+
60
+
61
+ class Process:
62
+ def __init__(self, args: ProcessArgs) -> None:
63
+ assert args.rclone_exe.exists()
64
+ self.args = args
65
+ self.tempdir: TemporaryDirectory | None = None
66
+ verbose = get_verbose(args.verbose)
67
+ if isinstance(args.rclone_conf, Config):
68
+ self.tempdir = TemporaryDirectory()
69
+ tmpfile = Path(self.tempdir.name) / "rclone.conf"
70
+ tmpfile.write_text(args.rclone_conf.text, encoding="utf-8")
71
+ rclone_conf = tmpfile
72
+ else:
73
+ rclone_conf = args.rclone_conf
74
+
75
+ assert rclone_conf.exists()
76
+
77
+ self.cmd = (
78
+ [str(args.rclone_exe.resolve())]
79
+ + ["--config", str(rclone_conf.resolve())]
80
+ + args.cmd
81
+ )
82
+ if verbose:
83
+ cmd_str = subprocess.list2cmdline(self.cmd)
84
+ print(f"Running: {cmd_str}")
85
+ self.process = subprocess.Popen(self.cmd, shell=False)
86
+
87
+ def cleanup(self) -> None:
88
+ if self.tempdir:
89
+ try:
90
+ self.tempdir.cleanup()
91
+ except Exception as e:
92
+ print(f"Error cleaning up tempdir: {e}")
93
+
94
+ def __del__(self) -> None:
95
+ self.cleanup()
96
+
97
+ def kill(self) -> None:
98
+ self.cleanup()
99
+ return self.process.kill()
100
+
101
+ def terminate(self) -> None:
102
+ self.cleanup()
103
+ return self.process.terminate()
104
+
105
+ @property
106
+ def returncode(self) -> int | None:
107
+ return self.process.returncode
108
+
109
+ @property
110
+ def stdout(self) -> Any:
111
+ return self.process.stdout
112
+
113
+ @property
114
+ def stderr(self) -> Any:
115
+ return self.process.stderr
116
+
117
+ def poll(self) -> int | None:
118
+ return self.process.poll()
119
+
120
+ def wait(self) -> int:
121
+ return self.process.wait()
122
+
123
+ def send_signal(self, signal: int) -> None:
124
+ return self.process.send_signal(signal)
@@ -3,6 +3,7 @@ Unit test file.
3
3
  """
4
4
 
5
5
  import subprocess
6
+ import time
6
7
  from concurrent.futures import ThreadPoolExecutor
7
8
  from fnmatch import fnmatch
8
9
  from pathlib import Path
@@ -15,6 +16,7 @@ from rclone_api.dir_listing import DirListing
15
16
  from rclone_api.exec import RcloneExec
16
17
  from rclone_api.file import File
17
18
  from rclone_api.filelist import FileList
19
+ from rclone_api.process import Process
18
20
  from rclone_api.remote import Remote
19
21
  from rclone_api.rpath import RPath
20
22
  from rclone_api.util import get_rclone_exe, to_path
@@ -33,6 +35,9 @@ class Rclone:
33
35
  def _run(self, cmd: list[str], check: bool = True) -> subprocess.CompletedProcess:
34
36
  return self._exec.execute(cmd, check=check)
35
37
 
38
+ def _launch_process(self, cmd: list[str]) -> Process:
39
+ return self._exec.launch_process(cmd)
40
+
36
41
  def ls(
37
42
  self,
38
43
  path: Dir | Remote | str,
@@ -232,3 +237,37 @@ class Rclone:
232
237
  if args is not None:
233
238
  cmd_list += args
234
239
  return self._run(cmd_list)
240
+
241
+ def mount(
242
+ self, src: Remote | Dir | str, outdir: Path, allow_writes=False, use_links=True
243
+ ) -> Process:
244
+ """Mount a remote or directory to a local path.
245
+
246
+ Args:
247
+ src: Remote or directory to mount
248
+ outdir: Local path to mount to
249
+
250
+ Returns:
251
+ CompletedProcess from the mount command execution
252
+
253
+ Raises:
254
+ subprocess.CalledProcessError: If the mount operation fails
255
+ """
256
+ if outdir.exists():
257
+ is_empty = not list(outdir.iterdir())
258
+ if not is_empty:
259
+ raise ValueError(
260
+ f"Mount directory already exists and is not empty: {outdir}"
261
+ )
262
+ outdir.rmdir()
263
+ src_str = convert_to_str(src)
264
+ cmd_list: list[str] = ["mount", src_str, str(outdir)]
265
+ if not allow_writes:
266
+ cmd_list.append("--read-only")
267
+ if use_links:
268
+ cmd_list.append("--links")
269
+ proc = self._launch_process(cmd_list)
270
+ time.sleep(2) # give it a moment to mount
271
+ if proc.poll() is not None:
272
+ raise ValueError("Mount process failed to start")
273
+ return proc
@@ -54,7 +54,7 @@ def to_path(item: Dir | Remote | str, rclone: Any) -> RPath:
54
54
  raise ValueError(f"Invalid type for item: {type(item)}")
55
55
 
56
56
 
57
- def _get_verbose(verbose: bool | None) -> bool:
57
+ def get_verbose(verbose: bool | None) -> bool:
58
58
  if verbose is not None:
59
59
  return verbose
60
60
  # get it from the environment
@@ -79,7 +79,7 @@ def rclone_execute(
79
79
  verbose: bool | None = None,
80
80
  ) -> subprocess.CompletedProcess:
81
81
  tempdir: TemporaryDirectory | None = None
82
- verbose = _get_verbose(verbose)
82
+ verbose = get_verbose(verbose)
83
83
  assert verbose is not None
84
84
 
85
85
  try:
@@ -0,0 +1,149 @@
1
+ Metadata-Version: 2.2
2
+ Name: rclone_api
3
+ Version: 1.0.17
4
+ Summary: rclone api in python
5
+ Home-page: https://github.com/zackees/rclone-api
6
+ Maintainer: Zachary Vorhies
7
+ License: BSD 3-Clause License
8
+ Keywords: template-python-cmd
9
+ Classifier: Programming Language :: Python :: 3
10
+ Requires-Python: >=3.10
11
+ Description-Content-Type: text/markdown
12
+ License-File: LICENSE
13
+ Requires-Dist: python-dotenv>=1.0.0
14
+ Dynamic: home-page
15
+ Dynamic: maintainer
16
+
17
+ # rclone-api
18
+
19
+ [![Linting](https://github.com/zackees/rclone-api/actions/workflows/lint.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/lint.yml)
20
+ [![MacOS_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml)
21
+ [![Ubuntu_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml)
22
+ [![Win_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml)
23
+
24
+ Api version of rclone. It's well tested. It's a pretty low level api without the bells and whistles of other apis, but it will get the job done.
25
+
26
+ You will need to have rclone installed and on your path.
27
+
28
+ One of the benefits of this api is that it does not use `shell=True`, which can keep `rclone` running in some instances even when try to kill the process.
29
+
30
+ # Install
31
+
32
+ `pip install rclone-api`
33
+
34
+
35
+ # Examples
36
+
37
+ You can use env variables or use a `.env` file to store your secrets.
38
+
39
+
40
+ # Rclone API Usage Examples
41
+
42
+ This script demonstrates how to interact with DigitalOcean Spaces using `rclone_api`.
43
+
44
+ ## Setup & Usage
45
+
46
+ Ensure you have set the required environment variables:
47
+
48
+ - `BUCKET_NAME`
49
+ - `BUCKET_KEY_PUBLIC`
50
+ - `BUCKET_KEY_SECRET`
51
+ - `BUCKET_URL`
52
+
53
+ Then, run the following Python script:
54
+
55
+ ```python
56
+ import os
57
+ from rclone_api import Config, DirListing, File, Rclone, Remote
58
+
59
+ # Load environment variables
60
+ BUCKET_NAME = os.getenv("BUCKET_NAME")
61
+ BUCKET_KEY_PUBLIC = os.getenv("BUCKET_KEY_PUBLIC")
62
+ BUCKET_KEY_SECRET = os.getenv("BUCKET_KEY_SECRET")
63
+ BUCKET_URL = "sfo3.digitaloceanspaces.com"
64
+
65
+ # Generate Rclone Configuration
66
+ def generate_rclone_config() -> Config:
67
+ config_text = f"""
68
+ [dst]
69
+ type = s3
70
+ provider = DigitalOcean
71
+ access_key_id = {BUCKET_KEY_PUBLIC}
72
+ secret_access_key = {BUCKET_KEY_SECRET}
73
+ endpoint = {BUCKET_URL}
74
+ """
75
+ return Config(config_text)
76
+
77
+ rclone = Rclone(generate_rclone_config())
78
+
79
+ # List Available Remotes
80
+ print("\n=== Available Remotes ===")
81
+ remotes = rclone.listremotes()
82
+ for remote in remotes:
83
+ print(remote)
84
+
85
+ # List Contents of the Root Bucket
86
+ print("\n=== Listing Root Bucket ===")
87
+ listing = rclone.ls(f"dst:{BUCKET_NAME}", max_depth=-1)
88
+
89
+ print("\nDirectories:")
90
+ for dir in listing.dirs:
91
+ print(dir)
92
+
93
+ print("\nFiles:")
94
+ for file in listing.files:
95
+ print(file)
96
+
97
+ # List a Specific Subdirectory
98
+ print("\n=== Listing 'zachs_video' Subdirectory ===")
99
+ path = f"dst:{BUCKET_NAME}/zachs_video"
100
+ listing = rclone.ls(path)
101
+ print(listing)
102
+
103
+ # List PNG Files in a Subdirectory
104
+ print("\n=== Listing PNG Files ===")
105
+ listing = rclone.ls(path, glob="*.png")
106
+
107
+ if listing.files:
108
+ for file in listing.files:
109
+ print(file)
110
+
111
+ # Copy a File
112
+ print("\n=== Copying a File ===")
113
+ if listing.files:
114
+ file = listing.files[0]
115
+ new_path = f"dst:{BUCKET_NAME}/zachs_video/{file.name}_copy"
116
+ rclone.copyfile(file, new_path)
117
+ print(f"Copied {file.name} to {new_path}")
118
+
119
+ # Copy Multiple Files
120
+ print("\n=== Copying Multiple Files ===")
121
+ if listing.files:
122
+ file_mapping = {file.name: file.name + "_copy" for file in listing.files[:2]}
123
+ rclone.copyfiles(file_mapping)
124
+ print(f"Copied files: {file_mapping}")
125
+
126
+ # Delete a File
127
+ print("\n=== Deleting a File ===")
128
+ file_to_delete = f"dst:{BUCKET_NAME}/zachs_video/sample.png_copy"
129
+ rclone.deletefiles([file_to_delete])
130
+ print(f"Deleted {file_to_delete}")
131
+
132
+ # Walk Through a Directory
133
+ print("\n=== Walking Through a Directory ===")
134
+ for dirlisting in rclone.walk(f"dst:{BUCKET_NAME}", max_depth=1):
135
+ print(dirlisting)
136
+
137
+ print("Done.")
138
+ ```
139
+
140
+
141
+ To develop software, run `. ./activate`
142
+
143
+ # Windows
144
+
145
+ This environment requires you to use `git-bash`.
146
+
147
+ # Linting
148
+
149
+ Run `./lint`
@@ -29,6 +29,7 @@ src/rclone_api/dir_listing.py
29
29
  src/rclone_api/exec.py
30
30
  src/rclone_api/file.py
31
31
  src/rclone_api/filelist.py
32
+ src/rclone_api/process.py
32
33
  src/rclone_api/rclone.py
33
34
  src/rclone_api/remote.py
34
35
  src/rclone_api/rpath.py
@@ -43,5 +44,6 @@ src/rclone_api/assets/example.txt
43
44
  tests/test_copy.py
44
45
  tests/test_is_synced.py
45
46
  tests/test_ls.py
47
+ tests/test_mount.py
46
48
  tests/test_remotes.py
47
49
  tests/test_walk.py
@@ -0,0 +1,105 @@
1
+ """
2
+ Unit test file for testing rclone mount functionality.
3
+ """
4
+
5
+ import os
6
+ import subprocess
7
+ import unittest
8
+ from pathlib import Path
9
+
10
+ from dotenv import load_dotenv
11
+
12
+ from rclone_api import Config, Process, Rclone
13
+
14
+ load_dotenv()
15
+
16
+
17
+ def _generate_rclone_config() -> Config:
18
+ # Load environment variables
19
+ BUCKET_KEY_SECRET = os.getenv("BUCKET_KEY_SECRET")
20
+ BUCKET_KEY_PUBLIC = os.getenv("BUCKET_KEY_PUBLIC")
21
+ BUCKET_URL = "sfo3.digitaloceanspaces.com"
22
+
23
+ config_text = f"""
24
+ [dst]
25
+ type = s3
26
+ provider = DigitalOcean
27
+ access_key_id = {BUCKET_KEY_PUBLIC}
28
+ secret_access_key = {BUCKET_KEY_SECRET}
29
+ endpoint = {BUCKET_URL}
30
+ """
31
+ return Config(config_text)
32
+
33
+
34
+ class RcloneMountTests(unittest.TestCase):
35
+ """Test rclone mount functionality."""
36
+
37
+ def setUp(self) -> None:
38
+ """Check if all required environment variables are set before running tests."""
39
+ required_vars = [
40
+ "BUCKET_NAME",
41
+ "BUCKET_KEY_SECRET",
42
+ "BUCKET_KEY_PUBLIC",
43
+ "BUCKET_URL",
44
+ ]
45
+ missing = [var for var in required_vars if not os.getenv(var)]
46
+ if missing:
47
+ self.skipTest(
48
+ f"Missing required environment variables: {', '.join(missing)}"
49
+ )
50
+
51
+ self.bucket_name = os.getenv("BUCKET_NAME")
52
+ self.mount_point = Path("test_mount")
53
+ # Create mount point directory if it doesn't exist
54
+ # self.mount_point.mkdir(exist_ok=True)
55
+ # make parents
56
+ parent = self.mount_point.parent
57
+ if not parent.exists():
58
+ parent.mkdir(parents=True)
59
+
60
+ os.environ["RCLONE_API_VERBOSE"] = "1"
61
+ self.rclone = Rclone(_generate_rclone_config())
62
+
63
+ def test_mount(self) -> None:
64
+ """Test mounting a remote bucket."""
65
+ remote_path = f"dst:{self.bucket_name}"
66
+ process: Process | None = None
67
+
68
+ try:
69
+ # Start the mount process
70
+ process = self.rclone.mount(remote_path, self.mount_point)
71
+ self.assertIsNone(
72
+ process.poll(), "Mount process should still be running after 2 seconds"
73
+ )
74
+
75
+ # Verify mount point exists and is accessible
76
+ self.assertTrue(self.mount_point.exists())
77
+ self.assertTrue(self.mount_point.is_dir())
78
+
79
+ # Check if we can list contents
80
+ contents = list(self.mount_point.iterdir())
81
+ self.assertGreater(
82
+ len(contents), 0, "Mounted directory should not be empty"
83
+ )
84
+
85
+ except subprocess.CalledProcessError as e:
86
+ self.fail(f"Mount operation failed: {str(e)}")
87
+ finally:
88
+ # Cleanup will happen in tearDown
89
+ if process:
90
+ if process.poll() is None:
91
+ process.kill()
92
+ stdout = process.stdout
93
+ if stdout:
94
+ # stdout is a buffered reader
95
+ for line in stdout:
96
+ print(line)
97
+ stderr = process.stderr
98
+ if stderr:
99
+ for line in stderr:
100
+ print(line)
101
+ process.kill()
102
+
103
+
104
+ if __name__ == "__main__":
105
+ unittest.main()
@@ -1,34 +0,0 @@
1
- Metadata-Version: 2.2
2
- Name: rclone_api
3
- Version: 1.0.15
4
- Summary: rclone api in python
5
- Home-page: https://github.com/zackees/rclone-api
6
- Maintainer: Zachary Vorhies
7
- License: BSD 3-Clause License
8
- Keywords: template-python-cmd
9
- Classifier: Programming Language :: Python :: 3
10
- Requires-Python: >=3.10
11
- Description-Content-Type: text/markdown
12
- License-File: LICENSE
13
- Requires-Dist: python-dotenv>=1.0.0
14
- Dynamic: home-page
15
- Dynamic: maintainer
16
-
17
- # rclone-api
18
-
19
- [![Linting](https://github.com/zackees/rclone-api/actions/workflows/lint.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/lint.yml)
20
- [![MacOS_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml)
21
- [![Ubuntu_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml)
22
- [![Win_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml)
23
-
24
- Api version of rclone. It's well tested. It's just released so this readme is a little to be desired.
25
-
26
- To develop software, run `. ./activate`
27
-
28
- # Windows
29
-
30
- This environment requires you to use `git-bash`.
31
-
32
- # Linting
33
-
34
- Run `./lint`
@@ -1,18 +0,0 @@
1
- # rclone-api
2
-
3
- [![Linting](https://github.com/zackees/rclone-api/actions/workflows/lint.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/lint.yml)
4
- [![MacOS_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml)
5
- [![Ubuntu_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml)
6
- [![Win_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml)
7
-
8
- Api version of rclone. It's well tested. It's just released so this readme is a little to be desired.
9
-
10
- To develop software, run `. ./activate`
11
-
12
- # Windows
13
-
14
- This environment requires you to use `git-bash`.
15
-
16
- # Linting
17
-
18
- Run `./lint`
@@ -1,34 +0,0 @@
1
- Metadata-Version: 2.2
2
- Name: rclone_api
3
- Version: 1.0.15
4
- Summary: rclone api in python
5
- Home-page: https://github.com/zackees/rclone-api
6
- Maintainer: Zachary Vorhies
7
- License: BSD 3-Clause License
8
- Keywords: template-python-cmd
9
- Classifier: Programming Language :: Python :: 3
10
- Requires-Python: >=3.10
11
- Description-Content-Type: text/markdown
12
- License-File: LICENSE
13
- Requires-Dist: python-dotenv>=1.0.0
14
- Dynamic: home-page
15
- Dynamic: maintainer
16
-
17
- # rclone-api
18
-
19
- [![Linting](https://github.com/zackees/rclone-api/actions/workflows/lint.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/lint.yml)
20
- [![MacOS_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_macos.yml)
21
- [![Ubuntu_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_ubuntu.yml)
22
- [![Win_Tests](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml/badge.svg)](https://github.com/zackees/rclone-api/actions/workflows/push_win.yml)
23
-
24
- Api version of rclone. It's well tested. It's just released so this readme is a little to be desired.
25
-
26
- To develop software, run `. ./activate`
27
-
28
- # Windows
29
-
30
- This environment requires you to use `git-bash`.
31
-
32
- # Linting
33
-
34
- Run `./lint`
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes