Auto-DLP 2025.1.69__tar.gz

auto_dlp-2025.1.69/.gitignore

@@ -0,0 +1,5 @@
+*/__pycache__
+.gitignore
+.idea/
+dist/
+*.sh

auto_dlp-2025.1.69/LICENSE.txt

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 J. Günthner
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

auto_dlp-2025.1.69/PKG-INFO

@@ -0,0 +1,162 @@
+Metadata-Version: 2.4
+Name: Auto-DLP
+Version: 2025.1.69
+Summary: A simple and lightweight system for downloading and managing youtube playlists and songs (based on yt-dlp)
+Author-email: Vorak <5m9mnf37q@mozmail.com>
+License-File: LICENSE.txt
+Keywords: Youtube,dlp,music
+Classifier: Environment :: Console
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: Microsoft
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.8
+Requires-Dist: mutagen>=1.40.0
+Requires-Dist: pycopy>=2025.1.14
+Requires-Dist: requests>=2.20.0
+Requires-Dist: yt-dlp>=2024.12.0
+Description-Content-Type: text/markdown
+
+# Auto-DLP
+A simple command-line utility for automating the process of downloading songs using yt-dlp and sorting them into folders. Can also push files to android devices.
+
+The philosophy of this tool is to download songs and lay them out beautifully in the filesystem, this means that there are a lot of features to get the names of files, folders, etc. correct.
+
+## Getting Started
+Use ```pipx install auto-dlp``` to install the command directly or use
+```pip install auto-dlp``` to install the module and run it using ```<Your python installation> -m auto_dlp```.
+```Your python installation``` is usually one of ```python```, ```python3``` or ```py```.
+
+In the command line try ```auto-dlp -h```. If this runs without an error, you have successfully installed the tool.
+
+Now choose a directory in which you would like your music to be stored, there run the command ```auto-dlp --create-example .```. This will create an example ```auto-dlp.json``` file which will download some songs and playlists. For the more advanced features of this tool, read the following section.
+
+## Advanced Features
+
+### Pushing the files to folders (Syncing)
+```jsonc
+{
+    // 
+    "sync dirs": ["the/directory/you/would like to sync to"]
+    
+    // alternatively, if you have a lot of directories you might
+    // want to sync to:
+    // "sync": [
+    //    "path1",
+    //    "path2",
+    //    ...
+    // ]
+    // The syncing is not invoked if the destination does not exist
+    
+    // Optional entries
+    
+    // Additional folders that should be copied to your device
+    // By default Auto-DLP only copies the artist's folders
+    // "extra sync folders": [...]
+ }
+```
+
+
+### Pushing the files to Android devices
+Use the normal method described in the previous header if possible.
+
+First you must configure the feature in your `auto-dlp.json` file:
+```jsonc
+{
+  // Where various files are cached
+  "config dir": "~/.local/state/youtube-downloader/",
+  // This is where your artists go
+  "artists": {
+    ...
+  },
+  // This is the important line
+  // adb stands for Android-Debug-Bridge and
+  // is an offial tool for debugging (and copying files)
+  "adb push": true
+
+  // The following options are optional
+  
+  // This option can be used to restart the adb service every
+  // time the program is executed, use this if it is making problems
+  // "restart adb": true
+  
+  // Where the files should be copied to 
+  // "adb push dir": "/sdcard/Music"
+  
+  // Additional folders that should be copied to your device
+  // By default Auto-DLP only copies the artist's folders
+  // "extra sync folders": [...]
+}
+```
+
+Then you must plug your phone into your Laptop with the phone's debug mode enabled. Now just start `auto-dlp` normally and it should copy the files over automatically.
+
+### Heeelp!! I get errors on certain playlists/songs
+
+For this there is two configuration options:
+```jsonc
+{
+  // Where various files are cached
+  "config dir": "~/.local/state/youtube-downloader/",
+  // This is where your artists go
+  "artists": {
+    ...
+  },
+  // With this entry you can forcefully rename songs that
+  // are automatically downloaded as part of a playlist.
+  // This is for example a fix for the problem of empty names.
+  "rename": {
+    "<some name>": "<a valid name>",
+    ...
+  },
+  // This key is for playlists that just don't want to be downloaded
+  // and make the program throw the following error:
+  // "The Youtube Api returned the same result twice"
+  // Ironically this is (so far) only on official Youtube Playlists
+  "fragile playlists": [
+    "RDCLAK5uy_lp8LtelM9GiSwRFGGQjctKaGoHcrgQVEU",
+    ...
+  ]
+}
+```
+
+### The names don't look like I want them to
+
+The names for songs in playlists are simply those entered
+on Youtube. These are, however, sometimes shit.
+
+For this there is the following configuration options:
+```jsonc
+{
+  // Where various files are cached
+  "config dir": "~/.local/state/youtube-downloader/",
+  // This is where your artists go
+  "artists": {
+    ...
+  },
+  // This entry specifies this that are searched for in names
+  // and removed. Case is ignored and the values are treated as
+  // python regexes
+  "name rules": [
+    // This value will remove all text from titles that contain a
+    // '(official' followed by anything and then another ')'
+    // i.e '(Official Music Video)'
+    "\\(official.*\\)",
+    ...
+  ],
+  // This entry is somewhat meta. It is intented to make writing
+  // regexes for "name rules" easier. It is a simple find and replace
+  // system.
+  // In the following example the string "<" is replaced by something
+  // that matches '[' or '('
+  // The strings are replaced in the "name rules"
+  // For example we could write the above rule as
+  // "<official.*>"
+  // Now it would also match (and remove) '[Official Video]'
+  "rule macros": {
+    "<": "[\\[\\(]",
+    ">": "[\\]\\)]",
+    ...
+  }
+}
+```

auto_dlp-2025.1.69/README.md

@@ -0,0 +1,143 @@
+# Auto-DLP
+A simple command-line utility for automating the process of downloading songs using yt-dlp and sorting them into folders. Can also push files to android devices.
+
+The philosophy of this tool is to download songs and lay them out beautifully in the filesystem, this means that there are a lot of features to get the names of files, folders, etc. correct.
+
+## Getting Started
+Use ```pipx install auto-dlp``` to install the command directly or use
+```pip install auto-dlp``` to install the module and run it using ```<Your python installation> -m auto_dlp```.
+```Your python installation``` is usually one of ```python```, ```python3``` or ```py```.
+
+In the command line try ```auto-dlp -h```. If this runs without an error, you have successfully installed the tool.
+
+Now choose a directory in which you would like your music to be stored, there run the command ```auto-dlp --create-example .```. This will create an example ```auto-dlp.json``` file which will download some songs and playlists. For the more advanced features of this tool, read the following section.
+
+## Advanced Features
+
+### Pushing the files to folders (Syncing)
+```jsonc
+{
+    // 
+    "sync dirs": ["the/directory/you/would like to sync to"]
+    
+    // alternatively, if you have a lot of directories you might
+    // want to sync to:
+    // "sync": [
+    //    "path1",
+    //    "path2",
+    //    ...
+    // ]
+    // The syncing is not invoked if the destination does not exist
+    
+    // Optional entries
+    
+    // Additional folders that should be copied to your device
+    // By default Auto-DLP only copies the artist's folders
+    // "extra sync folders": [...]
+ }
+```
+
+
+### Pushing the files to Android devices
+Use the normal method described in the previous header if possible.
+
+First you must configure the feature in your `auto-dlp.json` file:
+```jsonc
+{
+  // Where various files are cached
+  "config dir": "~/.local/state/youtube-downloader/",
+  // This is where your artists go
+  "artists": {
+    ...
+  },
+  // This is the important line
+  // adb stands for Android-Debug-Bridge and
+  // is an offial tool for debugging (and copying files)
+  "adb push": true
+
+  // The following options are optional
+  
+  // This option can be used to restart the adb service every
+  // time the program is executed, use this if it is making problems
+  // "restart adb": true
+  
+  // Where the files should be copied to 
+  // "adb push dir": "/sdcard/Music"
+  
+  // Additional folders that should be copied to your device
+  // By default Auto-DLP only copies the artist's folders
+  // "extra sync folders": [...]
+}
+```
+
+Then you must plug your phone into your Laptop with the phone's debug mode enabled. Now just start `auto-dlp` normally and it should copy the files over automatically.
+
+### Heeelp!! I get errors on certain playlists/songs
+
+For this there is two configuration options:
+```jsonc
+{
+  // Where various files are cached
+  "config dir": "~/.local/state/youtube-downloader/",
+  // This is where your artists go
+  "artists": {
+    ...
+  },
+  // With this entry you can forcefully rename songs that
+  // are automatically downloaded as part of a playlist.
+  // This is for example a fix for the problem of empty names.
+  "rename": {
+    "<some name>": "<a valid name>",
+    ...
+  },
+  // This key is for playlists that just don't want to be downloaded
+  // and make the program throw the following error:
+  // "The Youtube Api returned the same result twice"
+  // Ironically this is (so far) only on official Youtube Playlists
+  "fragile playlists": [
+    "RDCLAK5uy_lp8LtelM9GiSwRFGGQjctKaGoHcrgQVEU",
+    ...
+  ]
+}
+```
+
+### The names don't look like I want them to
+
+The names for songs in playlists are simply those entered
+on Youtube. These are, however, sometimes shit.
+
+For this there is the following configuration options:
+```jsonc
+{
+  // Where various files are cached
+  "config dir": "~/.local/state/youtube-downloader/",
+  // This is where your artists go
+  "artists": {
+    ...
+  },
+  // This entry specifies this that are searched for in names
+  // and removed. Case is ignored and the values are treated as
+  // python regexes
+  "name rules": [
+    // This value will remove all text from titles that contain a
+    // '(official' followed by anything and then another ')'
+    // i.e '(Official Music Video)'
+    "\\(official.*\\)",
+    ...
+  ],
+  // This entry is somewhat meta. It is intented to make writing
+  // regexes for "name rules" easier. It is a simple find and replace
+  // system.
+  // In the following example the string "<" is replaced by something
+  // that matches '[' or '('
+  // The strings are replaced in the "name rules"
+  // For example we could write the above rule as
+  // "<official.*>"
+  // Now it would also match (and remove) '[Official Video]'
+  "rule macros": {
+    "<": "[\\[\\(]",
+    ">": "[\\]\\)]",
+    ...
+  }
+}
+```

auto_dlp-2025.1.69/auto_dlp/YoutubeDataAPIv3.py

@@ -0,0 +1,238 @@
+import os.path as path
+import re
+import sys
+from collections import Counter
+from json import dumps
+
+from requests import get
+
+import auto_dlp.file_locations as fs
+import auto_dlp.utils as utils
+
+# def _ask_for_api_key():
+#     key = input("Please enter a valid Google Cloud Youtube API key (This is for getting the playlist item (in the future there might be a different implementation for this feature)): ").strip()
+#     if key == "":
+#         return _ask_for_api_key()
+#
+#     fs.touch_file(fs.yt_api_key_file())
+#     with fs.yt_api_key_file().open("w") as file:
+#         print(key.strip(), file=file)
+#
+#     return key
+
+# Wow, such secure
+_key_url = "https://pastebin.com/8vmQFLbm"
+
+
+def _download_api_key():
+    response = get(_key_url)
+    if response.status_code != 200:
+        raise RuntimeError("Could not reach server to request api key, please try again later")
+
+    pat = re.compile("this is where the key is:(?P<api_key>.*)not any further", flags=re.IGNORECASE)
+    match = pat.search(response.text)
+    key = match.groupdict()["api_key"]
+
+    file = fs.yt_api_key_file()
+    fs.touch_file(file)
+    with open(file, "w") as fhandle:
+        fhandle.write(key)
+
+    return key
+
+
+@utils.lazy
+def API_KEY():
+    file = fs.yt_api_key_file()
+    if not path.exists(file) or path.getsize(file) <= 0:
+        print("Api key file was not found", file=sys.stderr, flush=True)
+        return utils.scramble(_download_api_key()).strip()
+
+    with open(file) as fhandle:
+        return utils.scramble(fhandle.read()).strip()
+
+
+def request(url, params):
+    params["key"] = API_KEY()
+    response = get(url, params)
+
+    if response.status_code == 200 and "error" not in response.json():
+        return response
+
+    try:
+        err_msg = response.json()["error"]["message"]
+    except KeyError:
+        err_msg = "Error message could not be read"
+
+    if response.status_code == 403 or "API key not valid" in err_msg:
+        print("API key was not recognised by Youtube")
+        sys.exit()
+
+    raise RuntimeError(f"Network error: {response.status_code} {err_msg}")
+
+
+# This method is unaware of name cleaning
+def deduplicate_playlist_items(playlist_items):
+    name_counter = Counter()
+
+    for item in playlist_items:
+        name_counter[item["name"]] += 1
+
+    index_names = set()
+
+    for name, count in name_counter.items():
+        if count > 1:
+            index_names.add(name)
+
+    if len(index_names) == 0:
+        return playlist_items
+
+    name_counter = Counter()
+
+    for item in playlist_items:
+        name = item["name"]
+        if item["name"] in index_names:
+            name_counter[name] += 1
+            item["name"] = f"{name} ={name_counter[name]}="
+
+    return playlist_items
+
+
+# currently unused
+# def error_on_name_conflict(config, names, playlist_id):
+#     clean_names = defaultdict(lambda: [])
+#
+#     for name in names:
+#         clean_names[clean_name(config, name)].append(name)
+#
+#     conflict_found = False
+#
+#     for clean, sources in clean_names.items():
+#         if len(sources) > 1:
+#             conflict_found = True
+#             print(f"The specified name rules caused a name conflict: All of\n    {"\n    ".join(sources)}\n    became {clean}")
+#
+#     if conflict_found:
+#         print(f"There are name conflicts in playlist {playlist_id}", file=sys.stderr)
+#         sys.exit(-1)
+
+def get_playlist_items(config, playlist_id):
+    def send_request(params):
+        params["playlist_id"] = playlist_id
+        params["part"] = "contentDetails, snippet"
+        params["maxResults"] = 50
+        response = request("https://www.googleapis.com/youtube/v3/playlistItems", params)
+        items = {}
+
+        try:
+            items_json = response.json()["items"]
+
+            for item in items_json:
+                pos = item["snippet"]["position"]
+                items[pos] = {
+                    "name": item["snippet"]["title"],
+                    "id": item["contentDetails"]["videoId"]
+                }
+        except KeyError:
+            print(dumps(response.json(), indent=4), file=sys.stderr)
+            raise RuntimeError("Failed getting playlist items")
+
+        if "nextPageToken" in response.json():
+            return response.json()["pageInfo"]["totalResults"], response.json()["nextPageToken"], items
+        return response.json()["pageInfo"]["totalResults"], None, items
+
+    received_items = 0
+    result_count, page_token, items_map = send_request({})
+
+    items_list = [None] * result_count
+
+    def place_items():
+        for index, item in items_map.items():
+            if items_list[index] is None:
+                items_list[index] = item
+                continue
+
+            raise RuntimeError("The Youtube Api returned the same result twice")
+
+    place_items()
+    received_items += 50
+
+    while received_items < result_count:
+        _, page_token, items_map = send_request({"pageToken": page_token})
+        try:
+            place_items()
+        except RuntimeError as e:
+            if playlist_id in config.fragile_playlists:
+                return deduplicate_playlist_items(list(filter(lambda x: x is not None, items_list)))
+            raise e
+        received_items += 50
+
+    result = deduplicate_playlist_items(items_list)
+
+    # error_on_name_conflict(config, [item["name"] for item in result], playlist_id)
+
+    return result
+
+
+def get_song_channel(song_id):
+    response = request("https://www.googleapis.com/youtube/v3/videos", {
+        "key": API_KEY(),
+        "part": "snippet",
+        "id": song_id
+    })
+
+    try:
+        channel = response.json()["items"][0]["snippet"]["channelId"]
+    except KeyError:
+        print(dumps(response.json(), indent=4), file=sys.stderr)
+        raise RuntimeError("Failed getting song thumbnails")
+
+    return channel
+
+
+def get_song_thumbnails(song_id):
+    response = request("https://www.googleapis.com/youtube/v3/videos", {
+        "key": API_KEY(),
+        "part": "snippet",
+        "id": song_id
+    })
+
+    try:
+        thumbnails = response.json()["items"][0]["snippet"]["thumbnails"]
+    except KeyError:
+        print(dumps(response.json(), indent=4), file=sys.stderr)
+        raise RuntimeError("Failed getting song thumbnails")
+
+    return thumbnails
+
+
+def get_playlist_thumbnails(playlist_id):
+    response = request("https://www.googleapis.com/youtube/v3/playlists", {
+        "key": API_KEY(),
+        "part": "snippet",
+        "id": playlist_id
+    })
+
+    try:
+        thumbnails = response.json()["items"][0]["snippet"]["thumbnails"]
+    except KeyError:
+        print(dumps(response.json(), indent=4), file=sys.stderr)
+        raise RuntimeError("Failed getting song thumbnails")
+
+    return thumbnails
+
+
+def get_channel_thumbnails(channel_id):
+    response = request("https://www.googleapis.com/youtube/v3/channels", {
+        "key": API_KEY(),
+        "part": "snippet",
+        "id": channel_id
+    })
+
+    try:
+        thumbnails = response.json()["items"][0]["snippet"]["thumbnails"]
+    except KeyError:
+        print(dumps(response.json(), indent=4), file=sys.stderr)
+        raise RuntimeError("Failed getting song thumbnails")
+
+    return thumbnails

auto_dlp-2025.1.69/auto_dlp/__init__.py

@@ -0,0 +1,106 @@
+import argparse
+import json
+import os
+import re
+import sys
+from pathlib import Path
+from traceback import print_exception
+
+from auto_dlp import config_intepreter, example_file
+import auto_dlp.version
+
+_maybe_wait_after_finishing_execution = True
+_definitely_wait_after_finishing_execution = False
+CONFIG_FILENAME = "auto-dlp.json"
+
+
+def manage_music(path: Path, redownload_songs=(), test_names=(), playlist_test_names=None, verbose=False):
+    original_cur_dir = Path(os.curdir).resolve()
+    os.chdir(path)
+
+    try:
+        config_file = path / CONFIG_FILENAME
+        if not config_file.exists():
+            print(f"No {CONFIG_FILENAME} file found in directory {path.resolve()}", file=sys.stderr)
+            print(f"Use auto-dlp DIRECTORY --create-example to create an example {CONFIG_FILENAME} file")
+            return
+
+        with open(config_file) as file:
+            string = re.sub("//.*", "", file.read())
+            json_data = json.loads(string)
+
+        config_intepreter.execute(json_data, redownload_songs=redownload_songs, test_names=test_names,
+                                  playlist_test_names=playlist_test_names, verbose=verbose)
+    except KeyboardInterrupt:
+        print("Program interrupted by user", file=sys.stderr)
+
+    os.chdir(original_cur_dir)
+
+
+def command_entry_point():
+    try:
+        main()
+    except Exception as e:
+        print_exception(e)
+        if _maybe_wait_after_finishing_execution:
+            wait_to_close()
+    except SystemExit:
+        if _definitely_wait_after_finishing_execution:
+            wait_to_close()
+
+class CustomHelpFormatter(argparse.HelpFormatter):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs, max_help_position=70, width=100)
+
+def main():
+    parser = argparse.ArgumentParser(prog="auto-dlp",
+                                     description="A light-weight program for managing (and downloading) music",
+                                     allow_abbrev=True, add_help=True, exit_on_error=True, formatter_class=CustomHelpFormatter)
+    parser.add_argument("FOLDER",
+                        help=f"The folder in which to execute this program. It must contain an {CONFIG_FILENAME} file.")
+    parser.add_argument("-rd", "--redownload", metavar="SONG",
+                        help="Will delete the current version of the specified song, meaning that it will be redownloaded on the next execution of this program")
+    parser.add_argument("-tn", "--test-name", metavar="NAME",
+                        help="Allows to try out what the renaming system does to a specific string")
+    parser.add_argument("-ptn", "--playlist-test-names", metavar="PLAYLIST",
+                        help="Runs all the names of songs in PLAYLIST against -tn")
+    parser.add_argument("-w", "--wait", action="store_true", help="After finishing execution, wait for any input")
+    parser.add_argument("-v", "--verbose", action="store_true", help="Makes the script print more information")
+    parser.add_argument("--create-example", action="store_true", help=f"Creates a sample {CONFIG_FILENAME} file that will perform elementary functions")
+    parser.add_argument("--version", action="store_true", help="Show the version number and exit")
+
+    args = parser.parse_args()
+    global _maybe_wait_after_finishing_execution, _definitely_wait_after_finishing_execution
+    _maybe_wait_after_finishing_execution = args.wait
+    _definitely_wait_after_finishing_execution = args.wait
+
+    if args.version:
+        print(f"Auto-DLP version {version.program_version}")
+        if args.wait:
+            wait_to_close()
+        return
+
+    exec_dir = Path(args.FOLDER).expanduser().resolve()
+    print(f"Executing Auto-DLP in directory {exec_dir}")
+
+    if args.create_example:
+        example_file.generate_file(exec_dir, CONFIG_FILENAME)
+        if args.wait:
+            wait_to_close()
+        return
+
+    redownload = () if args.redownload is None else [args.redownload]
+    test_names = () if args.test_name is None else [args.test_name]
+    playlist = args.playlist_test_names
+
+    manage_music(exec_dir, redownload_songs=redownload, test_names=test_names, playlist_test_names=playlist,
+                 verbose=args.verbose)
+
+    if args.wait:
+        wait_to_close()
+
+def wait_to_close():
+    try:
+        input("Press ENTER to close program")
+    except KeyboardInterrupt:
+        pass

auto_dlp-2025.1.69/auto_dlp/__main__.py

@@ -0,0 +1,4 @@
+from auto_dlp import command_entry_point
+
+if __name__ == "__main__":
+    command_entry_point()