lfss 0.9.0__py3-none-any.whl → 0.9.2__py3-none-any.whl

Readme.md

@@ -9,6 +9,7 @@ My experiment on a lightweight and high-performance file/object storage service.
 - Pagination and sorted file listing for vast number of files.  
 - High performance: high concurrency, near-native speed on stress tests.
 - Support range requests, so you can stream large files / resume download.
+- WebDAV compatible ([NOTE](./docs/Webdav.md)).
 
 It stores small files and metadata in sqlite, large files in the filesystem.  
 Tested on 2 million files, and it is still fast.
@@ -30,9 +31,15 @@ lfss-panel --open
 Or, you can start a web server at `/frontend` and open `index.html` in your browser. 
 
 The API usage is simple, just `GET`, `PUT`, `DELETE` to the `/<username>/file/url` path.  
-Authentication via `Authorization` header with the value `Bearer <token>`, or through the `token` query parameter.  
+The authentication can be acheived through one of the following methods:
+1. `Authorization` header with the value `Bearer sha256(<username><password>)`.
+2. `token` query parameter with the value `sha256(<username><password>)`.
+3. HTTP Basic Authentication with the username and password (If WebDAV is enabled).
+
 You can refer to `frontend` as an application example, `lfss/api/connector.py` for more APIs. 
 
 By default, the service exposes all files to the public for `GET` requests, 
 but file-listing is restricted to the user's own files.  
-Please refer to [docs/Permission.md](./docs/Permission.md) for more details on the permission system.
+Please refer to [docs/Permission.md](./docs/Permission.md) for more details on the permission system. 
+
+More can be found in the [docs](./docs) directory.

docs/Enviroment_variables.md

@@ -0,0 +1,12 @@
+
+# Enviroment variables
+
+**Server**
+- `LFSS_DATA`: The directory to store the data. Default is `.storage_data`.
+- `LFSS_WEBDAV`: Enable WebDAV support. Default is `0`, set to `1` to enable.
+- `LFSS_LARGE_FILE`: The size limit of the file to store in the database. Default is `8m`.
+- `LFSS_DEBUG`: Enable debug mode for more verbose logging. Default is `0`, set to `1` to enable.
+
+**Client**
+- `LFSS_ENDPOINT`: The fallback server endpoint. Default is `http://localhost:8000`.
+- `LFSS_TOKEN`: The fallback token to authenticate. Should be `sha256(<username><password>)`.

docs/Known_issues.md

@@ -1 +1,3 @@
-[Safari 中文输入法回车捕获](https://github.com/anse-app/anse/issues/127)
+[Safari 中文输入法回车捕获](https://github.com/anse-app/anse/issues/127)
+
+[Word 临时文件](https://answers.microsoft.com/en-us/msoffice/forum/all/mac-os-word-temp-sb-folders-created-on-smb-share/40fda56c-c77c-4365-8fa3-eb87ac814207?page=1)

docs/Webdav.md

@@ -0,0 +1,22 @@
+# WebDAV
+
+It is convinient to make LFSS WebDAV compatible, because they both use HTTP `GET`, `PUT`, `DELETE` methods to interact with files.
+
+However, WebDAV utilize more HTTP methods, 
+which are disabled by default in LFSS, because they may not be supported by many middlewares or clients.   
+
+The WebDAV support can be enabled by setting the `LFSS_WEBDAV` environment variable to `1`. 
+i.e. 
+```sh
+LFSS_WEBDAV=1 lfss-serve
+```
+Please note:
+1. **WebDAV support is experimental, and is currently not well-tested.**
+2. LFSS not allow creating files in the root directory, however some client such as [Finder](https://sabre.io/dav/clients/finder/) will try to create files in the root directory. Thus, it is safer to mount the user directory only, e.g. `http://localhost:8000/<username>/`.
+3. LFSS not allow directory creation, instead it creates directoy implicitly when a file is uploaded to a non-exist directory. 
+   i.e. `PUT http://localhost:8000/<username>/dir/file.txt` will create the `dir` directory if it does not exist. 
+   However, the WebDAV `MKCOL` method requires the directory to be created explicitly, so WebDAV `MKCOL` method instead create a decoy file on the path (`.lfss-keep`), and hide the file from the file listing by `PROPFIND` method. 
+   This leads to: 
+   1) You may see a `.lfss-keep` file in the directory with native file listing (e.g. `/_api/list-files`), but it is hidden in WebDAV clients.
+   2) The directory may be deleted if there is no file in it and the `.lfss-keep` file is not created by WebDAV client.  
+

frontend/api.js

@@ -384,6 +384,27 @@ export default class Connector {
         }
     }
 
+    /**
+     * @param {string} srcPath - file path(url)
+     * @param {string} dstPath - new file path(url)
+     */
+    async copy(srcPath, dstPath){
+        if (srcPath.startsWith('/')){ srcPath = srcPath.slice(1); }
+        if (dstPath.startsWith('/')){ dstPath = dstPath.slice(1); }
+        const dst = new URL(this.config.endpoint + '/_api/copy');
+        dst.searchParams.append('src', srcPath);
+        dst.searchParams.append('dst', dstPath);
+        const res = await fetch(dst.toString(), {
+            method: 'POST',
+            headers: {
+                'Authorization': 'Bearer ' + this.config.token,
+                'Content-Type': 'application/www-form-urlencoded'
+            },
+        });
+        if (!(res.status == 200 || res.status == 201)){
+            throw new Error(`Failed to copy file, status code: ${res.status}, message: ${await fmtFailedResponse(res)}`);
+        }
+    }
 }
 
 /**

frontend/scripts.js

@@ -347,7 +347,7 @@ async function refreshFileList(){
                                 }
                             );
                         }, {
-                            text: 'Enter the destination path: ',
+                            text: 'Enter the destination path (Move): ',
                             placeholder: 'Destination path',
                             value: decodePathURI(dirurl), 
                             select: "last-pathname"
@@ -355,6 +355,30 @@ async function refreshFileList(){
                     });
                     actContainer.appendChild(moveButton);
 
+                    const copyButton = document.createElement('a');
+                    copyButton.textContent = 'Copy';
+                    copyButton.style.cursor = 'pointer';
+                    copyButton.addEventListener('click', () => {
+                        showFloatingWindowLineInput((dstPath) => {
+                            dstPath = encodePathURI(dstPath);
+                            console.log("Copying", dirurl, "to", dstPath);
+                            conn.copy(dirurl, dstPath)
+                                .then(() => {
+                                    refreshFileList();
+                                }, 
+                                (err) => {
+                                    showPopup('Failed to copy path: ' + err, {level: 'error'});
+                                }
+                            );
+                        }, {
+                            text: 'Enter the destination path (Copy): ',
+                            placeholder: 'Destination path',
+                            value: decodePathURI(dirurl), 
+                            select: "last-pathname"
+                        });
+                    });
+                    actContainer.appendChild(copyButton);
+
                     const downloadButton = document.createElement('a');
                     downloadButton.textContent = 'Download';
                     downloadButton.href = conn.config.endpoint + '/_api/bundle?' + 
@@ -478,7 +502,7 @@ async function refreshFileList(){
                                 }
                             );
                         }, {
-                            text: 'Enter the destination path: ',
+                            text: 'Enter the destination path (Move): ',
                             placeholder: 'Destination path',
                             value: decodePathURI(file.url), 
                             select: "last-filename"
@@ -486,6 +510,29 @@ async function refreshFileList(){
                     });
                     actContainer.appendChild(moveButton);
 
+                    const copyButton = document.createElement('a');
+                    copyButton.textContent = 'Copy';
+                    copyButton.style.cursor = 'pointer';
+                    copyButton.addEventListener('click', () => {
+                        showFloatingWindowLineInput((dstPath) => {
+                            dstPath = encodePathURI(dstPath);
+                            conn.copy(file.url, dstPath)
+                                .then(() => {
+                                    refreshFileList();
+                                }, 
+                                (err) => {
+                                    showPopup('Failed to copy file: ' + err, {level: 'error'});
+                                }
+                            );
+                        }, {
+                            text: 'Enter the destination path (Copy): ',
+                            placeholder: 'Destination path',
+                            value: decodePathURI(file.url), 
+                            select: "last-filename"
+                        });
+                    });
+                    actContainer.appendChild(copyButton);
+
                     const downloadBtn = document.createElement('a');
                     downloadBtn.textContent = 'Download';
                     downloadBtn.href = conn.config.endpoint + '/' + file.url + '?download=true&token=' + conn.config.token;

lfss/api/__init__.py

@@ -1,9 +1,9 @@
 import os, time, pathlib
 from threading import Lock
 from .connector import Connector
-from ..src.datatype import FileRecord
-from ..src.utils import decode_uri_compnents
-from ..src.bounded_pool import BoundedThreadPoolExecutor
+from ..eng.datatype import FileRecord
+from ..eng.utils import decode_uri_compnents
+from ..eng.bounded_pool import BoundedThreadPoolExecutor
 
 def upload_file(
     connector: Connector, 

lfss/api/connector.py

@@ -5,12 +5,12 @@ import requests
 import requests.adapters
 import urllib.parse
 from tempfile import SpooledTemporaryFile
-from lfss.src.error import PathNotFoundError
-from lfss.src.datatype import (
+from lfss.eng.error import PathNotFoundError
+from lfss.eng.datatype import (
     FileReadPermission, FileRecord, DirectoryRecord, UserRecord, PathContents, 
     FileSortKey, DirSortKey
     )
-from lfss.src.utils import ensure_uri_compnents
+from lfss.eng.utils import ensure_uri_compnents
 
 _default_endpoint = os.environ.get('LFSS_ENDPOINT', 'http://localhost:8000')
 _default_token = os.environ.get('LFSS_TOKEN', '')
@@ -270,6 +270,12 @@ class Connector:
         self._fetch_factory('POST', '_api/meta', {'path': path, 'new_path': new_path})(
             headers = {'Content-Type': 'application/www-form-urlencoded'}
         )
+    
+    def copy(self, src: str, dst: str):
+        """Copy file from src to dst."""
+        self._fetch_factory('POST', '_api/copy', {'src': src, 'dst': dst})(
+            headers = {'Content-Type': 'application/www-form-urlencoded'}
+        )
         
     def whoami(self) -> UserRecord:
         """Gets information about the current user."""

lfss/cli/balance.py

@@ -2,14 +2,14 @@
 Balance the storage by ensuring that large file thresholds are met.
 """
 
-from lfss.src.config import LARGE_BLOB_DIR, LARGE_FILE_BYTES
+from lfss.eng.config import LARGE_BLOB_DIR, LARGE_FILE_BYTES
 import argparse, time, itertools
 from functools import wraps
 from asyncio import Semaphore
 import aiofiles, asyncio
 import aiofiles.os
-from lfss.src.database import transaction, unique_cursor
-from lfss.src.connection_pool import global_entrance
+from lfss.eng.database import transaction, unique_cursor
+from lfss.eng.connection_pool import global_entrance
 
 sem: Semaphore
 

lfss/cli/cli.py

@@ -1,8 +1,8 @@
 from pathlib import Path
 import argparse, typing
 from lfss.api import Connector, upload_directory, upload_file, download_file, download_directory
-from lfss.src.datatype import FileReadPermission, FileSortKey, DirSortKey
-from lfss.src.utils import decode_uri_compnents
+from lfss.eng.datatype import FileReadPermission, FileSortKey, DirSortKey
+from lfss.eng.utils import decode_uri_compnents
 from . import catch_request_error, line_sep
 
 def parse_permission(s: str) -> FileReadPermission:

lfss/cli/panel.py

@@ -2,6 +2,7 @@
 import uvicorn
 from fastapi import FastAPI
 from fastapi.staticfiles import StaticFiles
+from fastapi.middleware.cors import CORSMiddleware
 
 import argparse
 from contextlib import asynccontextmanager
@@ -27,6 +28,13 @@ assert (__frontend_dir / "index.html").exists(), "Frontend panel not found"
 
 app = FastAPI(lifespan=app_lifespan)
 app.mount("/", StaticFiles(directory=__frontend_dir, html=True), name="static")
+app.add_middleware(
+    CORSMiddleware,
+    allow_origins=["*"],
+    allow_credentials=True,
+    allow_methods=["*"],
+    allow_headers=["*"],
+)
 
 def main():
     parser = argparse.ArgumentParser(description="Serve frontend panel")

lfss/cli/serve.py

@@ -1,7 +1,9 @@
 import argparse
 from uvicorn import Config, Server
 from uvicorn.config import LOGGING_CONFIG
-from ..src.server import *
+from ..eng.config import DEBUG_MODE
+from ..svc.app_base import logger
+from ..svc.app import app
 
 def main():
     parser = argparse.ArgumentParser()
@@ -19,7 +21,7 @@ def main():
         app=app,
         host=args.host,
         port=args.port,
-        access_log=False,
+        access_log=True if DEBUG_MODE else False,
         workers=args.workers,
         log_config=default_logging_config
     )

lfss/cli/user.py

@@ -1,10 +1,10 @@
 import argparse, asyncio, os
 from contextlib import asynccontextmanager
 from .cli import parse_permission, FileReadPermission
-from ..src.utils import parse_storage_size, fmt_storage_size
-from ..src.datatype import AccessLevel
-from ..src.database import Database, FileReadPermission, transaction, UserConn, unique_cursor, FileConn
-from ..src.connection_pool import global_entrance
+from ..eng.utils import parse_storage_size, fmt_storage_size
+from ..eng.datatype import AccessLevel
+from ..eng.database import Database, FileReadPermission, transaction, UserConn, unique_cursor, FileConn
+from ..eng.connection_pool import global_entrance
 
 def parse_access_level(s: str) -> AccessLevel:
     for p in AccessLevel:

lfss/cli/vacuum.py

@@ -2,17 +2,17 @@
 Vacuum the database and external storage to ensure that the storage is consistent and minimal.
 """
 
-from lfss.src.config import LARGE_BLOB_DIR
+from lfss.eng.config import LARGE_BLOB_DIR
 import argparse, time
 from functools import wraps
 from asyncio import Semaphore
 import aiofiles, asyncio
 import aiofiles.os
 from contextlib import contextmanager
-from lfss.src.database import transaction, unique_cursor
-from lfss.src.stat import RequestDB
-from lfss.src.utils import now_stamp
-from lfss.src.connection_pool import global_entrance
+from lfss.eng.database import transaction, unique_cursor
+from lfss.svc.request_log import RequestDB
+from lfss.eng.utils import now_stamp
+from lfss.eng.connection_pool import global_entrance
 
 sem: Semaphore
 

lfss/{src → eng}/config.py

@@ -21,6 +21,7 @@ else:
 MAX_MEM_FILE_BYTES = 128 * 1024 * 1024   # 128MB
 MAX_BUNDLE_BYTES = 512 * 1024 * 1024   # 512MB
 CHUNK_SIZE = 1024 * 1024   # 1MB chunks for streaming (on large files)
+DEBUG_MODE = os.environ.get('LFSS_DEBUG', '0') == '1'
 
 THUMB_DB = DATA_HOME / 'thumbs.db'
 THUMB_SIZE = (48, 48)

lfss/{src → eng}/database.py

@@ -19,7 +19,7 @@ from .datatype import (
     )
 from .config import LARGE_BLOB_DIR, CHUNK_SIZE, LARGE_FILE_BYTES, MAX_MEM_FILE_BYTES
 from .log import get_logger
-from .utils import decode_uri_compnents, hash_credential, concurrent_wrap, debounce_async
+from .utils import decode_uri_compnents, hash_credential, concurrent_wrap, debounce_async, copy_file
 from .error import *
 
 class DBObjectBase(ABC):
@@ -405,18 +405,30 @@ class FileConn(DBObjectBase):
             )
         await self._user_size_inc(owner_id, file_size)
         self.logger.info(f"File {url} created")
-    
-    async def move_file(self, old_url: str, new_url: str):
+
+    # not tested
+    async def copy_file(self, old_url: str, new_url: str, user_id: Optional[int] = None):
         old = await self.get_file_record(old_url)
         if old is None:
             raise FileNotFoundError(f"File {old_url} not found")
         new_exists = await self.get_file_record(new_url)
         if new_exists is not None:
             raise FileExistsError(f"File {new_url} already exists")
-        await self.cur.execute("UPDATE fmeta SET url = ?, create_time = CURRENT_TIMESTAMP WHERE url = ?", (new_url, old_url))
-        self.logger.info(f"Moved file {old_url} to {new_url}")
+        new_fid = str(uuid.uuid4())
+        user_id = old.owner_id if user_id is None else user_id
+        await self.cur.execute(
+            "INSERT INTO fmeta (url, owner_id, file_id, file_size, permission, external, mime_type) VALUES (?, ?, ?, ?, ?, ?, ?)", 
+            (new_url, user_id, new_fid, old.file_size, old.permission, old.external, old.mime_type)
+            )
+        if not old.external:
+            await self.set_file_blob(new_fid, await self.get_file_blob(old.file_id))
+        else:
+            await copy_file(LARGE_BLOB_DIR / old.file_id, LARGE_BLOB_DIR / new_fid)
+        await self._user_size_inc(user_id, old.file_size)
+        self.logger.info(f"Copied file {old_url} to {new_url}")
     
-    async def move_path(self, old_url: str, new_url: str, conflict_handler: Literal['skip', 'overwrite'] = 'overwrite', user_id: Optional[int] = None):
+    # not tested
+    async def copy_path(self, old_url: str, new_url: str, conflict_handler: Literal['skip', 'overwrite'] = 'overwrite', user_id: Optional[int] = None):
         assert old_url.endswith('/'), "Old path must end with /"
         assert new_url.endswith('/'), "New path must end with /"
         if user_id is None:
@@ -426,12 +438,49 @@ class FileConn(DBObjectBase):
             cursor = await self.cur.execute("SELECT * FROM fmeta WHERE url LIKE ? AND owner_id = ?", (old_url + '%', user_id))
             res = await cursor.fetchall()
         for r in res:
-            new_r = new_url + r[0][len(old_url):]
+            old_record = FileRecord(*r)
+            new_r = new_url + old_record.url[len(old_url):]
             if conflict_handler == 'overwrite':
                 await self.cur.execute("DELETE FROM fmeta WHERE url = ?", (new_r, ))
             elif conflict_handler == 'skip':
                 if (await self.cur.execute("SELECT url FROM fmeta WHERE url = ?", (new_r, ))) is not None:
                     continue
+            new_fid = str(uuid.uuid4())
+            user_id = old_record.owner_id if user_id is None else user_id
+            await self.cur.execute(
+                "INSERT INTO fmeta (url, owner_id, file_id, file_size, permission, external, mime_type) VALUES (?, ?, ?, ?, ?, ?, ?)", 
+                (new_r, user_id, new_fid, old_record.file_size, old_record.permission, old_record.external, old_record.mime_type)
+                )
+            if not old_record.external:
+                await self.set_file_blob(new_fid, await self.get_file_blob(old_record.file_id))
+            else:
+                await copy_file(LARGE_BLOB_DIR / old_record.file_id, LARGE_BLOB_DIR / new_fid)
+            await self._user_size_inc(user_id, old_record.file_size)
+    
+    async def move_file(self, old_url: str, new_url: str):
+        old = await self.get_file_record(old_url)
+        if old is None:
+            raise FileNotFoundError(f"File {old_url} not found")
+        new_exists = await self.get_file_record(new_url)
+        if new_exists is not None:
+            raise FileExistsError(f"File {new_url} already exists")
+        await self.cur.execute("UPDATE fmeta SET url = ?, create_time = CURRENT_TIMESTAMP WHERE url = ?", (new_url, old_url))
+        self.logger.info(f"Moved file {old_url} to {new_url}")
+    
+    async def move_path(self, old_url: str, new_url: str, user_id: Optional[int] = None):
+        assert old_url.endswith('/'), "Old path must end with /"
+        assert new_url.endswith('/'), "New path must end with /"
+        if user_id is None:
+            cursor = await self.cur.execute("SELECT * FROM fmeta WHERE url LIKE ?", (old_url + '%', ))
+            res = await cursor.fetchall()
+        else:
+            cursor = await self.cur.execute("SELECT * FROM fmeta WHERE url LIKE ? AND owner_id = ?", (old_url + '%', user_id))
+            res = await cursor.fetchall()
+        for r in res:
+            new_r = new_url + r[0][len(old_url):]
+            if await (await self.cur.execute("SELECT url FROM fmeta WHERE url = ?", (new_r, ))).fetchone():
+                self.logger.error(f"File {new_r} already exists on move path: {old_url} -> {new_url}")
+                raise FileDuplicateError(f"File {new_r} already exists")
             await self.cur.execute("UPDATE fmeta SET url = ?, create_time = CURRENT_TIMESTAMP WHERE url = ?", (new_r, r[0]))
     
     async def log_access(self, url: str):
@@ -633,6 +682,9 @@ class Database:
         async with unique_cursor() as cur:
             user = await get_user(cur, u)
             assert user is not None, f"User {u} not found"
+
+            if await check_path_permission(url, user, cursor=cur) < AccessLevel.WRITE:
+                raise PermissionDeniedError(f"Permission denied: {user.username} cannot write to {url}")
             
             fconn_r = FileConn(cur)
             user_size_used = await fconn_r.user_size(user.id)
@@ -734,14 +786,33 @@ class Database:
             if r is None:
                 raise FileNotFoundError(f"File {old_url} not found")
             if op_user is not None:
-                if await check_path_permission(old_url, op_user) < AccessLevel.WRITE:
+                if await check_path_permission(old_url, op_user, cursor=cur) < AccessLevel.WRITE:
                     raise PermissionDeniedError(f"Permission denied: {op_user.username} cannot move file {old_url}")
+                if await check_path_permission(new_url, op_user, cursor=cur) < AccessLevel.WRITE:
+                    raise PermissionDeniedError(f"Permission denied: {op_user.username} cannot move file to {new_url}")
             await fconn.move_file(old_url, new_url)
 
             new_mime, _ = mimetypes.guess_type(new_url)
             if not new_mime is None:
                 await fconn.update_file_record(new_url, mime_type=new_mime)
     
+    # not tested
+    async def copy_file(self, old_url: str, new_url: str, op_user: Optional[UserRecord] = None):
+        validate_url(old_url)
+        validate_url(new_url)
+
+        async with transaction() as cur:
+            fconn = FileConn(cur)
+            r = await fconn.get_file_record(old_url)
+            if r is None:
+                raise FileNotFoundError(f"File {old_url} not found")
+            if op_user is not None:
+                if await check_path_permission(old_url, op_user, cursor=cur) < AccessLevel.READ:
+                    raise PermissionDeniedError(f"Permission denied: {op_user.username} cannot copy file {old_url}")
+                if await check_path_permission(new_url, op_user, cursor=cur) < AccessLevel.WRITE:
+                    raise PermissionDeniedError(f"Permission denied: {op_user.username} cannot copy file to {new_url}")
+            await fconn.copy_file(old_url, new_url, user_id=op_user.id if op_user is not None else None)
+    
     async def move_path(self, old_url: str, new_url: str, op_user: UserRecord):
         validate_url(old_url, is_file=False)
         validate_url(new_url, is_file=False)
@@ -756,14 +827,38 @@ class Database:
 
         async with unique_cursor() as cur:
             if not (
-                await check_path_permission(old_url, op_user) >= AccessLevel.WRITE and
-                await check_path_permission(new_url, op_user) >= AccessLevel.WRITE
+                await check_path_permission(old_url, op_user, cursor=cur) >= AccessLevel.WRITE and
+                await check_path_permission(new_url, op_user, cursor=cur) >= AccessLevel.WRITE
                 ):
                 raise PermissionDeniedError(f"Permission denied: {op_user.username} cannot move path {old_url} to {new_url}")
 
         async with transaction() as cur:
             fconn = FileConn(cur)
-            await fconn.move_path(old_url, new_url, 'overwrite', op_user.id)
+            await fconn.move_path(old_url, new_url, op_user.id)
+    
+    # not tested
+    async def copy_path(self, old_url: str, new_url: str, op_user: UserRecord):
+        validate_url(old_url, is_file=False)
+        validate_url(new_url, is_file=False)
+        
+        if new_url.startswith('/'):
+            new_url = new_url[1:]
+        if old_url.startswith('/'):
+            old_url = old_url[1:]
+        assert old_url != new_url, "Old and new path must be different"
+        assert old_url.endswith('/'), "Old path must end with /"
+        assert new_url.endswith('/'), "New path must end with /"
+
+        async with unique_cursor() as cur:
+            if not (
+                await check_path_permission(old_url, op_user, cursor=cur) >= AccessLevel.READ and
+                await check_path_permission(new_url, op_user, cursor=cur) >= AccessLevel.WRITE
+                ):
+                raise PermissionDeniedError(f"Permission denied: {op_user.username} cannot copy path {old_url} to {new_url}")
+        
+        async with transaction() as cur:
+            fconn = FileConn(cur)
+            await fconn.copy_path(old_url, new_url, 'overwrite', op_user.id)
 
     async def __batch_delete_file_blobs(self, fconn: FileConn, file_records: list[FileRecord], batch_size: int = 512):
         # https://github.com/langchain-ai/langchain/issues/10321

lfss/{src → eng}/error.py

@@ -2,10 +2,16 @@ import sqlite3
 
 class LFSSExceptionBase(Exception):...
 
+class FileLockedError(LFSSExceptionBase):...
+
+class InvalidOptionsError(LFSSExceptionBase, ValueError):...
+
 class DatabaseLockedError(LFSSExceptionBase, sqlite3.DatabaseError):...
 
 class PathNotFoundError(LFSSExceptionBase, FileNotFoundError):...
 
+class FileDuplicateError(LFSSExceptionBase, FileExistsError):...
+
 class PermissionDeniedError(LFSSExceptionBase, PermissionError):...
 
 class InvalidPathError(LFSSExceptionBase, ValueError):...

lfss/{src → eng}/thumb.py

@@ -1,6 +1,6 @@
-from lfss.src.config import THUMB_DB, THUMB_SIZE
-from lfss.src.database import FileConn
-from lfss.src.connection_pool import unique_cursor
+from lfss.eng.config import THUMB_DB, THUMB_SIZE
+from lfss.eng.database import FileConn
+from lfss.eng.connection_pool import unique_cursor
 from typing import Optional
 from PIL import Image
 from io import BytesIO
@@ -69,12 +69,13 @@ async def get_thumb(path: str) -> Optional[tuple[bytes, str]]:
     async with unique_cursor() as main_c:
         fconn = FileConn(main_c)
         r = await fconn.get_file_record(path)
-        if r is None:
-            async with cache_cursor() as cur:
-                await _delete_cache_thumb(cur, path)
-            raise FileNotFoundError(f'File not found: {path}')
-        if not r.mime_type.startswith('image/'):
-            return None
+
+    if r is None:
+        async with cache_cursor() as cur:
+            await _delete_cache_thumb(cur, path)
+        raise FileNotFoundError(f'File not found: {path}')
+    if not r.mime_type.startswith('image/'):
+        return None
 
     async with cache_cursor() as cur:
         c_time = r.create_time

lfss/{src → eng}/utils.py

@@ -1,8 +1,10 @@
 import datetime, time
 import urllib.parse
-import asyncio
+import pathlib
 import functools
 import hashlib
+import aiofiles
+import asyncio
 from asyncio import Lock
 from collections import OrderedDict
 from concurrent.futures import ThreadPoolExecutor
@@ -11,6 +13,12 @@ from functools import wraps, partial
 from uuid import uuid4
 import os
 
+async def copy_file(source: str|pathlib.Path, destination: str|pathlib.Path):
+    async with aiofiles.open(source, mode='rb') as src:
+        async with aiofiles.open(destination, mode='wb') as dest:
+            while chunk := await src.read(1024):
+                await dest.write(chunk)
+
 def hash_credential(username: str, password: str):
     return hashlib.sha256((username + password).encode()).hexdigest()
 

lfss/svc/app.py

@@ -0,0 +1,9 @@
+from .app_base import ENABLE_WEBDAV
+from .app_native import *
+
+# order matters
+app.include_router(router_api)
+if ENABLE_WEBDAV:
+    from .app_dav import *
+    app.include_router(router_dav)
+app.include_router(router_fs)