copyparty 1.19.9__tar.gz → 1.19.11__tar.gz

{copyparty-1.19.9 → copyparty-1.19.11}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: copyparty
-Version: 1.19.9
+Version: 1.19.11
 Summary:   Portable file server with accelerated resumable uploads, deduplication, WebDAV, FTP, zeroconf, media indexer, video thumbnails, audio transcoding, and write-only folders
 Author-email: ed <copyparty@ocv.me>
 License: MIT
@@ -1418,7 +1418,7 @@ general usage:
 on macos, connect from finder:
 * [Go] -> [Connect to Server...] -> http://192.168.123.1:3923/
 
-in order to grant full write-access to webdav clients, the volflag `daw` must be set and the account must also have delete-access (otherwise the client won't be allowed to replace the contents of existing files, which is how webdav works)
+to upload or edit files with WebDAV clients, enable the `daw` volflag (because most WebDAV clients expect this) and give your account the delete-permission. This avoids getting several copies of the same file on the server. HOWEVER: This will also make all PUT-uploads overwrite existing files if the user has delete-access, so use with caution.
 
 > note: if you have enabled [IdP authentication](#identity-providers) then that may cause issues for some/most webdav clients; see [the webdav section in the IdP docs](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients)
 
@@ -2771,7 +2771,7 @@ below are some tweaks roughly ordered by usefulness:
   * and pypy can sometimes crash on startup with `-j0` (TODO make issue)
 
 * if you are running the copyparty server **on Windows or Macos:**
-  * `--casechk=y` makes it much faster, but also awakens [the usual surprises](https://github.com/9001/copyparty/issues/781) you expect from a case-insensitive filesystem
+  * `--casechk=n` makes it much faster, but also awakens [the usual surprises](https://github.com/9001/copyparty/issues/781) you expect from a case-insensitive filesystem
     * this is the same as `casechk: n` in a config-file
 
 
@@ -3088,7 +3088,7 @@ first install one of the following:
 and then copypaste the following command into `a-Shell`:
 
 ```sh
-curl https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
+curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
 ```
 
 what this does:

{copyparty-1.19.9 → copyparty-1.19.11}/README.md

@@ -1353,7 +1353,7 @@ general usage:
 on macos, connect from finder:
 * [Go] -> [Connect to Server...] -> http://192.168.123.1:3923/
 
-in order to grant full write-access to webdav clients, the volflag `daw` must be set and the account must also have delete-access (otherwise the client won't be allowed to replace the contents of existing files, which is how webdav works)
+to upload or edit files with WebDAV clients, enable the `daw` volflag (because most WebDAV clients expect this) and give your account the delete-permission. This avoids getting several copies of the same file on the server. HOWEVER: This will also make all PUT-uploads overwrite existing files if the user has delete-access, so use with caution.
 
 > note: if you have enabled [IdP authentication](#identity-providers) then that may cause issues for some/most webdav clients; see [the webdav section in the IdP docs](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients)
 
@@ -2706,7 +2706,7 @@ below are some tweaks roughly ordered by usefulness:
   * and pypy can sometimes crash on startup with `-j0` (TODO make issue)
 
 * if you are running the copyparty server **on Windows or Macos:**
-  * `--casechk=y` makes it much faster, but also awakens [the usual surprises](https://github.com/9001/copyparty/issues/781) you expect from a case-insensitive filesystem
+  * `--casechk=n` makes it much faster, but also awakens [the usual surprises](https://github.com/9001/copyparty/issues/781) you expect from a case-insensitive filesystem
     * this is the same as `casechk: n` in a config-file
 
 
@@ -3023,7 +3023,7 @@ first install one of the following:
 and then copypaste the following command into `a-Shell`:
 
 ```sh
-curl https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
+curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
 ```
 
 what this does:

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/__main__.py

@@ -1212,6 +1212,7 @@ def add_upload(ap):
     ap2.add_argument("--chmod-d", metavar="UGO", type=u, default="755", help="unix file permissions to use when creating directories; see --help-chmod. Examples: [\033[32m755\033[0m] = owner-RW + all-R, [\033[32m777\033[0m] = full-yolo (volflag=chmod_d)")
     ap2.add_argument("--uid", metavar="N", type=int, default=-1, help="unix user-id to chown new files/folders to; default = -1 = do-not-change (volflag=uid)")
     ap2.add_argument("--gid", metavar="N", type=int, default=-1, help="unix group-id to chown new files/folders to; default = -1 = do-not-change (volflag=gid)")
+    ap2.add_argument("--wram", action="store_true", help="allow uploading even if a volume is inside a ramdisk, meaning that all data will be lost on the next server reboot (volflag=wram)")
     ap2.add_argument("--dedup", action="store_true", help="enable symlink-based upload deduplication (volflag=dedup)")
     ap2.add_argument("--safe-dedup", metavar="N", type=int, default=50, help="how careful to be when deduplicating files; [\033[32m1\033[0m] = just verify the filesize, [\033[32m50\033[0m] = verify file contents have not been altered (volflag=safededup)")
     ap2.add_argument("--hardlink", action="store_true", help="enable hardlink-based dedup; will fallback on symlinks when that is impossible (across filesystems) (volflag=hardlink)")
@@ -1569,6 +1570,9 @@ def add_logging(ap):
     ap2.add_argument("--ihead", metavar="HEADER", type=u, action='append', help="print request \033[33mHEADER\033[0m; [\033[32m*\033[0m]=all")
     ap2.add_argument("--ohead", metavar="HEADER", type=u, action='append', help="print response \033[33mHEADER\033[0m; [\033[32m*\033[0m]=all")
     ap2.add_argument("--lf-url", metavar="RE", type=u, default=r"^/\.cpr/|[?&]th=[wjp]|/\.(_|ql_|DS_Store$|localized$)", help="dont log URLs matching regex \033[33mRE\033[0m")
+    ap2.add_argument("--scan-st-r", metavar="SEC", type=float, default=0.1, help="fs-indexing: wait \033[33mSEC\033[0m between each status-message")
+    ap2.add_argument("--scan-pr-r", metavar="SEC", type=float, default=10, help="fs-indexing: wait \033[33mSEC\033[0m between each 'progress:' message")
+    ap2.add_argument("--scan-pr-s", metavar="MiB", type=float, default=1, help="fs-indexing: say 'file: <name>' when a file larger than \033[33mMiB\033[0m is about to be hashed")
 
 
 def add_admin(ap):

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/__version__.py

@@ -1,8 +1,8 @@
 # coding: utf-8
 
-VERSION = (1, 19, 9)
+VERSION = (1, 19, 11)
 CODENAME = "usernames"
-BUILD_DT = (2025, 9, 15)
+BUILD_DT = (2025, 9, 20)
 
 S_VERSION = ".".join(map(str, VERSION))
 S_BUILD_DT = "{0:04d}-{1:02d}-{2:02d}".format(*BUILD_DT)

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/broker_mpw.py

@@ -12,6 +12,7 @@ import queue
 from .__init__ import ANYWIN
 from .authsrv import AuthSrv
 from .broker_util import BrokerCli, ExceptionalQueue, NotExQueue
+from .fsutil import ramdisk_chk
 from .httpsrv import HttpSrv
 from .util import FAKE_MP, Daemon, HMaccas
 
@@ -50,6 +51,7 @@ class MpWorker(BrokerCli):
 
         # starting to look like a good idea
         self.asrv = AuthSrv(args, None, False)
+        ramdisk_chk(self.asrv)
 
         # instantiate all services here (TODO: inheritance?)
         self.iphash = HMaccas(os.path.join(self.args.E.cfg, "iphash"), 8)
@@ -93,6 +95,7 @@ class MpWorker(BrokerCli):
             if dest == "reload":
                 self.logw("mpw.asrv reloading")
                 self.asrv.reload()
+                ramdisk_chk(self.asrv)
                 self.logw("mpw.asrv reloaded")
                 continue
 

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/cfg.py

@@ -57,6 +57,7 @@ def vf_bmap()   :
         "rmagic",
         "rss",
         "wo_up_readme",
+        "wram",
         "xdev",
         "xlink",
         "xvol",
@@ -187,6 +188,7 @@ flagcats = {
         "chmod_f=644": "unix-permission for new files",
         "uid=573": "change owner of new files/folders to unix-user 573",
         "gid=999": "change owner of new files/folders to unix-group 999",
+        "wram": "allow uploading into ramdisks",
         "sparse": "force use of sparse files, mainly for s3-backed storage",
         "nosparse": "deny use of sparse files, mainly for slow storage",
         "daw": "enable full WebDAV write support (dangerous);\nPUT-operations will now \033[1;31mOVERWRITE\033[0;35m existing files",

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/fsutil.py

@@ -7,27 +7,30 @@ import re
 import time
 
 from .__init__ import ANYWIN, MACOS
-from .authsrv import AXS, VFS
+from .authsrv import AXS, VFS, AuthSrv
 from .bos import bos
 from .util import chkcmd, min_ex, undot
 
 class Fstab(object):
-    def __init__(self, log , args ):
+    def __init__(self, log , args , verbose ):
         self.log_func = log
+        self.verbose = verbose
 
         self.warned = False
         self.trusted = False
         self.tab  = None
         self.oldtab  = None
         self.srctab = "a"
-        self.cache   = {}
+        self.cache    = {}
         self.age = 0.0
         self.maxage = args.mtab_age
 
     def log(self, msg , c   = 0)  :
+        if not c or self.verbose:
+            return
         self.log_func("fstab", msg, c)
 
-    def get(self, path )  :
+    def get(self, path )   :
         now = time.time()
         if now - self.age > self.maxage or len(self.cache) > 9000:
             self.age = now
@@ -35,6 +38,7 @@ class Fstab(object):
             self.tab = None
             self.cache = {}
 
+        mp = ""
         fs = "ext4"
         msg = "failed to determine filesystem at %r; assuming %s\n%s"
 
@@ -44,7 +48,7 @@ class Fstab(object):
                 path = self._winpath(path)
             except:
                 self.log(msg % (path, fs, min_ex()), 3)
-                return fs
+                return fs, ""
 
         path = undot(path)
         try:
@@ -53,14 +57,14 @@ class Fstab(object):
             pass
 
         try:
-            fs = self.get_w32(path) if ANYWIN else self.get_unix(path)
+            fs, mp = self.get_w32(path) if ANYWIN else self.get_unix(path)
         except:
             self.log(msg % (path, fs, min_ex()), 3)
 
         fs = fs.lower()
-        self.cache[path] = fs
-        self.log("found %s at %r" % (fs, path))
-        return fs
+        self.cache[path] = (fs, mp)
+        self.log("found %s at %r, %r" % (fs, mp, path))
+        return fs, mp
 
     def _winpath(self, path )  :
         # try to combine volume-label + st_dev (vsn)
@@ -75,34 +79,49 @@ class Fstab(object):
         self.tab = VFS(self.log_func, "idk", "/", "/", AXS(), {})
         self.trusted = False
 
-    def build_tab(self)  :
-        self.log("inspecting mtab for changes")
-
+    def _from_sp_mount(self)   :
         sptn = r"^.*? on (.*) type ([^ ]+) \(.*"
         if MACOS:
             sptn = r"^.*? on (.*) \(([^ ]+), .*"
 
         ptn = re.compile(sptn)
         so, _ = chkcmd(["mount"])
-        tab1   = []
-        atab = []
+        dtab   = {}
         for ln in so.split("\n"):
             m = ptn.match(ln)
             if not m:
                 continue
 
             zs1, zs2 = m.groups()
-            tab1.append((str(zs1), str(zs2)))
-            atab.append(ln)
+            dtab[str(zs1)] = str(zs2)
+
+        return dtab
+
+    def _from_proc(self)   :
+        ret   = {}
+        with open("/proc/self/mounts", "rb", 262144) as f:
+            src = f.read(262144).decode("utf-8", "replace").split("\n")
+        for zsl in [x.split(" ") for x in src]:
+            if len(zsl) < 3:
+                continue
+            zs = zsl[1]
+            zs = zs.replace("\\011", "\t").replace("\\040", " ").replace("\\134", "\\")
+            ret[zs] = zsl[2]
+        return ret
+
+    def build_tab(self)  :
+        self.log("inspecting mtab for changes")
+        dtab = self._from_sp_mount() if MACOS else self._from_proc()
 
         # keep empirically-correct values if mounttab unchanged
-        srctab = "\n".join(sorted(atab))
+        srctab = str(sorted(dtab.items()))
         if srctab == self.srctab:
             self.tab = self.oldtab
             return
 
         self.log("mtab has changed; reevaluating support for sparse files")
 
+        tab1 = list(dtab.items())
         tab1.sort(key=lambda x: (len(x[0]), x[0]))
         path1, fs1 = tab1[0]
         tab = VFS(self.log_func, fs1, path1, path1, AXS(), {})
@@ -139,7 +158,7 @@ class Fstab(object):
             vn.realpath = ptn.sub(nval, vn.realpath)
             visit.extend(list(vn.nodes.values()))
 
-    def get_unix(self, path )  :
+    def get_unix(self, path )   :
         if not self.tab:
             try:
                 self.build_tab()
@@ -148,18 +167,42 @@ class Fstab(object):
                 # prisonparty or other restrictive environment
                 if not self.warned:
                     self.warned = True
-                    self.log("failed to build tab:\n{}".format(min_ex()), 3)
+                    t = "failed to associate fs-mounts with the VFS (this is fine):\n%s"
+                    self.log(t % (min_ex(),), 6)
                 self.build_fallback()
 
         ret = self.tab._find(path)[0]
         if self.trusted or path == ret.vpath:
-            return ret.realpath.split("/")[0]
+            return ret.realpath.split("/")[0], ret.vpath
         else:
-            return "idk"
+            return "idk", ""
 
-    def get_w32(self, path )  :
+    def get_w32(self, path )   :
         if not self.tab:
             self.build_fallback()
 
         ret = self.tab._find(path)[0]
-        return ret.realpath
+        return ret.realpath, ""
+
+
+def ramdisk_chk(asrv )  :
+    # should have been in authsrv but that's a circular import
+    mods = []
+    ramfs = ("tmpfs", "overlay")
+    log = asrv.log_func or print
+    fstab = Fstab(log, asrv.args, False)
+    for vn in asrv.vfs.all_nodes.values():
+        if not vn.axs.uwrite or "wram" in vn.flags:
+            continue
+        ap = vn.realpath
+        if not ap or os.path.isfile(ap):
+            continue
+        fs, mp = fstab.get(ap)
+        mp = "/" + mp.strip("/")
+        if fs == "tmpfs" or (mp == "/" and fs in ramfs):
+            mods.append((vn.vpath, ap, fs, mp))
+            vn.axs.uwrite.clear()
+    if mods:
+        t = "WARNING: write-access was removed from the following volumes because they are not mapped to an actual HDD for storage! All uploaded data would live in RAM only, and all uploaded files would be LOST on next reboot. To allow uploading and ignore this hazard, enable the 'wram' option (global/volflag). List of affected volumes:"
+        t2 = ["\n  volume=[/%s], abspath=%r, type=%s, root=%r" % x for x in mods]
+        log("vfs", t + "".join(t2) + "\n", 1)

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/ftpd.py

@@ -275,6 +275,10 @@ class FtpFs(AbstractedFS):
     def chdir(self, path )  :
         nwd = join(self.cwd, path)
         vfs, rem = self.hub.asrv.vfs.get(nwd, self.uname, False, False)
+        if not vfs.realpath:
+            self.cwd = nwd
+            return
+
         ap = vfs.canonical(rem)
         try:
             st = bos.stat(ap)
@@ -284,12 +288,9 @@ class FtpFs(AbstractedFS):
             # returning 550 is library-default and suitable
             raise FSE("No such file or directory")
 
-        if vfs.realpath:
-            avfs = vfs.chk_ap(ap, st)
-            if not avfs:
-                raise FSE("Permission denied", 1)
-        else:
-            avfs = vfs
+        avfs = vfs.chk_ap(ap, st)
+        if not avfs:
+            raise FSE("Permission denied", 1)
 
         self.cwd = nwd
 

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/mtag.py

@@ -426,9 +426,8 @@ def _get_cover_from_epub2(
 )  :
     # <meta name="cover" content="id-to-cover-image"> in <metadata>, then
     # <item> in <manifest>
-    cover_id = package_root.find("./metadata/meta[@name='cover']", package_ns).get(
-        "content"
-    )
+    xn = package_root.find("./metadata/meta[@name='cover']", package_ns)
+    cover_id = xn.get("content") if xn is not None else None
 
     if not cover_id:
         return None

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/svchub.py

@@ -24,6 +24,7 @@ from .__init__ import ANYWIN, EXE, MACOS, PY2, TYPE_CHECKING, E, EnvParams, unic
 from .authsrv import BAD_CFG, AuthSrv, n_du_who, n_ver_who
 from .bos import bos
 from .cert import ensure_cert
+from .fsutil import ramdisk_chk
 from .mtag import HAVE_FFMPEG, HAVE_FFPROBE, HAVE_MUTAGEN
 from .pwhash import HAVE_ARGON2
 from .tcpsrv import TcpSrv
@@ -304,6 +305,7 @@ class SvcHub(object):
 
         # initiate all services to manage
         self.asrv = AuthSrv(self.args, self.log, dargs=self.dargs)
+        ramdisk_chk(self.asrv)
 
         if args.cgen:
             self.asrv.cgen()
@@ -1349,6 +1351,7 @@ class SvcHub(object):
         with self.reload_mutex:
             self.log("root", "reloading config")
             self.asrv.reload(9 if up2k else 4)
+            ramdisk_chk(self.asrv)
             if up2k:
                 self.up2k.reload(rescan_all_vols)
                 t += "; volumes are now reinitializing"

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/up2k.py

@@ -209,7 +209,7 @@ class Up2k(object):
             t = "could not initialize sqlite3, will use in-memory registry only"
             self.log(t, 3)
 
-        self.fstab = Fstab(self.log_func, self.args)
+        self.fstab = Fstab(self.log_func, self.args, True)
         self.gen_fk = self._gen_fk if self.args.log_fk else gen_filekey
 
         if self.args.hash_mt < 2:
@@ -1491,6 +1491,7 @@ class Up2k(object):
 
         th_cvd = self.args.th_coversd
         th_cvds = self.args.th_coversd_set
+        scan_pr_s = self.args.scan_pr_s
 
         self.pp.msg = "a%d %s" % (self.pp.n, cdir)
 
@@ -1702,7 +1703,7 @@ class Up2k(object):
             if nohash or not sz:
                 wark = up2k_wark_from_metadata(self.salt, sz, lmod, rd, fn)
             else:
-                if sz > 1024 * 1024:
+                if sz > 1024 * 1024 * scan_pr_s:
                     self.log("file: %r" % (abspath,))
 
                 try:
@@ -2991,7 +2992,7 @@ class Up2k(object):
 
         # check if filesystem supports sparse files;
         # refuse out-of-order / multithreaded uploading if sprs False
-        sprs = self.fstab.get(pdir) != "ng"
+        sprs = self.fstab.get(pdir)[0] != "ng"
 
         if True:
             jcur = self.cur.get(ptop)
@@ -5155,7 +5156,7 @@ class Up2k(object):
                     sprs = False
 
             if not ANYWIN and sprs and sz > 1024 * 1024:
-                fs = self.fstab.get(pdir)
+                fs, mnt = self.fstab.get(pdir)
                 if fs == "ok":
                     pass
                 elif "nosparse" in vf:

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty/util.py

@@ -525,6 +525,8 @@ def py_desc()  :
     ofs = py_ver.find(".final.")
     if ofs > 0:
         py_ver = py_ver[:ofs]
+    if "free-threading" in sys.version:
+        py_ver += "t"
 
     host_os = platform.system()
     compiler = platform.python_compiler().split("http")[0]
@@ -1040,16 +1042,18 @@ class ProgressPrinter(threading.Thread):
         sigblock()
         tp = 0
         msg = None
-        no_stdout = self.args.q
+        slp_pr = self.args.scan_pr_r
+        slp_ps = min(slp_pr, self.args.scan_st_r)
+        no_stdout = self.args.q or slp_pr == slp_ps
         fmt = " {}\033[K\r" if VT100 else " {} $\r"
         while not self.end:
-            time.sleep(0.1)
+            time.sleep(slp_ps)
             if msg == self.msg or self.end:
                 continue
 
             msg = self.msg
             now = time.time()
-            if msg and now - tp > 10:
+            if msg and now - tp >= slp_pr:
                 tp = now
                 self.log("progress: %r" % (msg,), 6)
 

{copyparty-1.19.9 → copyparty-1.19.11}/copyparty.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: copyparty
-Version: 1.19.9
+Version: 1.19.11
 Summary:   Portable file server with accelerated resumable uploads, deduplication, WebDAV, FTP, zeroconf, media indexer, video thumbnails, audio transcoding, and write-only folders
 Author-email: ed <copyparty@ocv.me>
 License: MIT
@@ -1418,7 +1418,7 @@ general usage:
 on macos, connect from finder:
 * [Go] -> [Connect to Server...] -> http://192.168.123.1:3923/
 
-in order to grant full write-access to webdav clients, the volflag `daw` must be set and the account must also have delete-access (otherwise the client won't be allowed to replace the contents of existing files, which is how webdav works)
+to upload or edit files with WebDAV clients, enable the `daw` volflag (because most WebDAV clients expect this) and give your account the delete-permission. This avoids getting several copies of the same file on the server. HOWEVER: This will also make all PUT-uploads overwrite existing files if the user has delete-access, so use with caution.
 
 > note: if you have enabled [IdP authentication](#identity-providers) then that may cause issues for some/most webdav clients; see [the webdav section in the IdP docs](https://github.com/9001/copyparty/blob/hovudstraum/docs/idp.md#connecting-webdav-clients)
 
@@ -2771,7 +2771,7 @@ below are some tweaks roughly ordered by usefulness:
   * and pypy can sometimes crash on startup with `-j0` (TODO make issue)
 
 * if you are running the copyparty server **on Windows or Macos:**
-  * `--casechk=y` makes it much faster, but also awakens [the usual surprises](https://github.com/9001/copyparty/issues/781) you expect from a case-insensitive filesystem
+  * `--casechk=n` makes it much faster, but also awakens [the usual surprises](https://github.com/9001/copyparty/issues/781) you expect from a case-insensitive filesystem
     * this is the same as `casechk: n` in a config-file
 
 
@@ -3088,7 +3088,7 @@ first install one of the following:
 and then copypaste the following command into `a-Shell`:
 
 ```sh
-curl https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
+curl -L https://github.com/9001/copyparty/raw/refs/heads/hovudstraum/contrib/setup-ashell.sh | sh
 ```
 
 what this does: