pyrekordbox 0.2.1__py3-none-any.whl → 0.2.2__py3-none-any.whl

docs/source/formats/anlz.md

@@ -247,7 +247,7 @@ Rekordbox Performance Mode, but starting with Rekordbox version 6 it also gets
 exported to external media so CDJ-3000 players can use it to control lighting looks.
 
 ```{note}
-The version that Rekordbox 6 exports is garbled with an XOR mask to make it
+The version that Rekordbox 6 **exports** is garbled with an XOR mask to make it
 more difficult to access the data. All bytes after `len_e` are XOR-masked with a
 pattern that is generated by adding the value of `len_e` to each byte of the following
 base pattern:
@@ -270,14 +270,16 @@ entries are present in the tag. Each entry represents one recognized phrase.
 
 The value `mood` specifies the overall type of phrase structure that rekordbox chose to
 represent the song, based on its analysis of the audio.
-The value 1 is a “high” mood
-where the phrase types consist of “Intro”, “Up”, “Down”, “Chorus”, and “Outro”.
-Other values in each phrase entry cause the intro, chorus, and outro phrases to have
-their labels subdivided into styes “1” or “2” (for example, “Intro 1”), and “up” is
-subdivided into style “Up 1”, “Up 2”, or “Up 3”. See the table below for an expanded
-version of this description.
+
+The value 1 is a “high” mood where the phrase types consist of “Intro”, “Up”, “Down”,
+“Chorus”, and “Outro”. Other values in each phrase entry cause the intro, chorus, and
+outro phrases to have their labels subdivided into styes “1” or “2”
+(for example, “Intro 1”), and “up” is subdivided into style “Up 1”, “Up 2”, or “Up 3”.
+See the table below for an expanded version of this description.
+
 The value 2 is a “mid” mood where the phrase types are labeled “Intro”, “Verse 1”
 through “Verse 6”, “Chorus”, “Bridge”, and “Outro”.
+
 And value 3 is a “low” mood where the phrase types are labeled “Intro”, “Verse 1”,
 “Verse 2”, “Chorus”, “Bridge”, and “Outro”. There are three different phrase type
 values for each of “Verse 1” and “Verse 2”, but rekordbox makes no distinction between
@@ -301,6 +303,175 @@ Each phrase entry has the structure shown below:
 Song structure entry.
 ```
 
+The first two bytes of each song structure entry hold `index`, which numbers each phrase,
+starting at one and incrementing with each entry. That is followed by beat,
+a two-byte value that specifies the beat at which this phrase begins in the track.
+It continues until either the beat number of the next phrase, or the beat identified
+by end in the tag header if this is the last entry.
+
+`kind` specifies what kind of phrase rekordbox has identified here.
+The interpretation depends on the value of mood in the tag header, as is detailed the
+table below. In the case of the “high” mood, there are numbered variations for some
+of the phrases displayed in rekordbox that are not reflected in kind, but depend on the
+values of three flag bytes `k1` through `k3` in a complicated way shown in its own table.
+
+We also noticed that when `mood`, `kind` and the `k` flags indicate a phrase of type
+“Up 3”, additional beat numbers (which all fall within the phrase) are present in the
+entry. These may indicate points within the phrase at which lighting changes would look good;
+more investigation is required to make sense of them.
+The number of beats that will be listed seems to depend on the value of the flag `b`:
+if this has the value 0, there will be a single beat found in `beat2`, and if `b` has
+the value 1 there will be three different beat numbers present, with increasing values,
+in `beat2`, `beat3` and `beat4`.
+
+`fill` is a flag that indicates whether there are fill (non-phrase) beats at the end of
+the phrase. If it is non-zero, then ``beat fill`` holds the beat number at which the
+fill begins. When fill-in is present, it is indicated in rekordbox by little dots on the
+full waveform. The manual says:
+
+
+    [Fill in] is a section that provides improvisational changes at the end of phrase.
+    [Fill in] is detected at the end of Intro, Up, and Chorus (up to 4 beats).
+
+
+
+```{eval-rst}
+.. list-table:: Phrase labels in each mood.
+   :header-rows: 1
+
+   * - Phrase ID
+     - Low Label
+     - Mid Label
+     - High Label
+   * - 1
+     - Intro
+     - Intro
+     - Intro n
+   * - 2
+     - Verse 1
+     - Verse 1
+     - Up n
+   * - 3
+     - Verse 1
+     - Verse 2
+     - Down
+   * - 4
+     - Verse 1
+     - Verse 3
+     -
+   * - 5
+     - Verse 2
+     - Verse 4
+     - Chorus n
+   * - 6
+     - Verse 2
+     - Verse 5
+     - Outro n
+   * - 7
+     - Verse 2
+     - Verse 6
+     -
+   * - 8
+     - Bridge
+     - Bridge
+     -
+   * - 9
+     - Chorus
+     - Chorus
+     -
+   * - 10
+     - Outro
+     - Outro
+     -
+```
+
+```{eval-rst}
+.. list-table:: High mood phrase variants.
+   :header-rows: 1
+
+   * - Phrase ID
+     - k1
+     - k2
+     - k3
+     - Expanded Label
+   * - 1
+     - 1
+     -
+     -
+     - Intro 1
+   * - 1
+     - 0
+     -
+     -
+     - Intro 2
+   * - 2
+     -
+     - 0
+     - 0
+     - Up 1
+   * - 2
+     -
+     - 0
+     - 1
+     - Up 2
+   * - 2
+     -
+     - 1
+     - 0
+     - Up 3
+   * - 3
+     -
+     -
+     -
+     - Down 1
+   * - 5
+     - 1
+     -
+     -
+     - Chorus 2
+   * - 5
+     - 0
+     -
+     -
+     - Chorus 1
+   * - 6
+     - 1
+     -
+     -
+     - Outro 1
+   * - 6
+     - 0
+     -
+     -
+     - Outro 2
+```
+
+```{eval-rst}
+.. list-table:: Track banks.
+   :header-rows: 1
+
+   * - Bank ID
+     - Label
+   * - 0
+     - Default (treated as Cool)
+   * - 1
+     - Cool
+   * - 2
+     - Natural
+   * - 3
+     - Hot
+   * - 4
+     - Subtle
+   * - 5
+     - Warm
+   * - 6
+     - Vivid
+   * - 7
+     - Club 1
+   * - 8
+     - Club 2
+```
+
 ### PWAV: Waveform Preview Tag
 
 Seen in `.DAT` analysis files. This kind of section holds a fixed-width monochrome

docs/source/formats/db6.md

@@ -489,7 +489,7 @@ around 1/75th of a second (13.333ms) per frame, i.e. about half the granularity
      - `0` if not a loop or VBR/ABR MPEG file
    * - `Kind`
      - Type of cue point
-     - Cue= `0` , Fade-In= `0` , Fade-Out= `0` , Load= `3` , Loop= `4`
+     - Cue= `0` , Fade-In= `1` , Fade-Out= `2` , Load= `3` , Loop= `4`
    * - `Color`
      - The color ID of the cue point
      - `-1` if no color

docs/source/index.md

@@ -28,11 +28,7 @@ Pioneer's Rekordbox DJ Software. It currently supports
 - Analysis files (ANLZ)
 - My-Settings files
 
-Tested Rekordbox versions: `5.8.6 | 6.5.3`
-
-Starting from version `6.6.5` Pioneer obfuscated the `app.asar` file contents, breaking
-the key extraction (see [this issue][issue] and the Rekordbox 6 database section for
-more details).
+Tested Rekordbox versions: `5.8.6 | 6.5.3 | 6.7.7`
 
 ```{warning}
 This project is still under development and might contain bugs or
@@ -59,10 +55,10 @@ maxdepth: 2
 caption: File formats
 ---
 
+formats/db6
 formats/xml
 formats/anlz
 formats/mysetting
-formats/db6
 ```
 
 ```{toctree}

docs/source/quickstart.md

@@ -25,18 +25,6 @@ from pyrekordbox import show_config
 
 show_config()
 ````
-
-which, for example, will print
-````
-Pioneer:
-   app_dir =      C:\Users\user\AppData\Roaming\Pioneer
-   install_dir =  C:\Program Files\Pioneer
-Rekordbox 5:
-   app_dir =      C:\Users\user\AppData\Roaming\Pioneer\rekordbox
-   install_dir =  C:\Program Files\Pioneer\rekordbox 5.8.6
-   ...
-````
-
 If for some reason the configuration fails the values can be updated by providing the
 paths to the directory where Pioneer applications are installed (`pioneer_install_dir`)
 and to the directory where Pioneer stores the application data  (`pioneer_app_dir`)
@@ -45,12 +33,65 @@ from pyrekordbox.config import update_config
 
 update_config("<pioneer_install_dir>", "<pioneer_app_dir>")
 ````
-
 Alternatively the two paths can be specified in a configuration file under the section
 `rekordbox`. Supported configuration files are pyproject.toml, setup.cfg, pyrekordbox.toml,
 pyrekordbox.cfg and pyrekordbox.yaml.
 
 
+## Rekordbox 6 database
+
+Rekordbox 6 now uses a SQLite database for storing the collection content.
+Unfortunatly, the new `master.db` SQLite database is encrypted using
+[SQLCipher][sqlcipher], which means it can't be used without the encryption key.
+However, since your data is stored and used locally, the key must be present on the
+machine running Rekordbox.
+
+Pyrekordbox can unlock the new Rekordbox `master.db` SQLite database and provides
+an easy interface for accessing the data stored in it:
+
+````python
+from pyrekordbox import Rekordbox6Database
+
+db = Rekordbox6Database()
+
+for content in db.get_content():
+    print(content.Title, content.Artist.Name)
+
+playlist = db.get_playlist()[0]
+for song in playlist.Songs:
+    content = song.Content
+    print(content.Title, content.Artist.Name)
+````
+Fields in the Rekordbox database that are stored without linking to other tables
+can be changed via the corresponding property of the object:
+````python
+content = db.get_content()[0]
+content.Title = "New Title"
+````
+Some fields are stored as references to other tables, for example the artist of a track.
+Check the [documentation](#db6-format) of the corresponding object for more information.
+So far only a few tables support adding or deleting entries:
+- ``DjmdPlaylist``: Playlists/Playlist Folders
+- ``DjmdSongPlaylist``: Songs in a playlist
+
+````{important}
+Starting from Rekordbox version ``6.6.5`` Pioneer obfuscated the ``app.asar`` file
+contents, breaking the key extraction (see [this discussion](https://github.com/dylanljones/pyrekordbox/discussions/97) for more details).
+If you are using a later version of Rekorbox and have no cached key from a previous
+version, the database can not be unlocked automatically.
+The command line interface of ``pyrekordbox`` provides a command for downloading
+the key from known sources and writing it to the cache file:
+```shell
+python -m pyrekordbox download-key
+```
+Once the key is cached the database can be opened without providing the key.
+The key can also be provided manually:
+```python
+db = Rekordbox6Database(key="<insert key here>")
+```
+````
+
+
 ## Rekordbox XML
 
 The Rekordbox XML database is used for importing (and exporting) Rekordbox collections
@@ -107,6 +148,17 @@ Changing and creating the Rekordbox analysis files is planned as well, but for t
 full structure of the analysis files has to be understood.
 
 
+```{note}
+Some ANLZ tags are still unsupported:
+  - PCOB
+  - PCO2
+  - PSSI
+  - PWV6
+  - PWV7
+  - PWVC
+```
+
+
 ## Rekordbox My-Settings
 
 Rekordbox stores the user settings in `*SETTING.DAT` files, which get exported to USB
@@ -126,37 +178,8 @@ sync = mysett.get("sync")
 quant = mysett.get("quantize")
 ````
 
-
-## Rekordbox 6 database
-
-Rekordbox 6 now uses a SQLite database for storing the collection content.
-Unfortunatly, the new `master.db` SQLite database is encrypted using
-[SQLCipher][sqlcipher], which means it can't be used without the encryption key.
-However, since your data is stored and used locally, the key must be present on the
-machine running Rekordbox.
-
-Pyrekordbox can unlock the new Rekordbox `master.db` SQLite database and provides
-an easy interface for accessing the data stored in it:
-
-````python
-from pyrekordbox import Rekordbox6Database
-
-db = Rekordbox6Database()
-
-for content in db.get_content():
-    print(content.Title, content.Artist.Name)
-
-playlist = db.get_playlist()[0]
-for song in playlist.Songs:
-    content = song.Content
-    print(content.Title, content.Artist.Name)
-````
-Adding new rows to the tables of the database is not supported since it is not yet known
-how Rekordbox generates the UUID/ID's. Using wrong values for new database entries
-could corrupt the library. This feature will be added after some testing.
-Changing existing entries like the title, artist or file path of a track in the database
-should work as expected.
-
-
+```{note}
+The `DEVSETTING.DAT` file is still not supported
+```
 
 [sqlcipher]: https://www.zetetic.net/sqlcipher/open-source/

docs/source/tutorial/index.md

@@ -13,8 +13,8 @@ maxdepth: 3
 ---
 
 configuration
+db6
 xml
 anlz
 mysetting
-db6
 ````

pyrekordbox/__init__.py

@@ -3,7 +3,7 @@
 # Date:   2022-04-10
 
 from .logger import logger
-from .config import show_config, get_config
+from .config import show_config, get_config, update_config
 from .xml import RekordboxXml, XmlDuplicateError, XmlAttributeKeyError
 from .anlz import get_anlz_paths, walk_anlz_paths, read_anlz_files, AnlzFile
 from .mysettings import (

pyrekordbox/_version.py

@@ -12,5 +12,5 @@ __version__: str
 __version_tuple__: VERSION_TUPLE
 version_tuple: VERSION_TUPLE
 
-__version__ = version = '0.2.1'
-__version_tuple__ = version_tuple = (0, 2, 1)
+__version__ = version = '0.2.2'
+__version_tuple__ = version_tuple = (0, 2, 2)

pyrekordbox/anlz/file.py

@@ -8,9 +8,12 @@ from pathlib import Path
 from typing import Union
 from .tags import TAGS
 from . import structs
+from construct import Int16ub
 
 logger = logging.getLogger(__name__)
 
+XOR_MASK = bytearray.fromhex("CB E1 EE FA E5 EE AD EE E9 D2 E9 EB E1 E9 F3 E8 E9 F4 E1")
+
 
 class BuildFileLengthError(Exception):
     def __init__(self, struct, len_data):
@@ -103,6 +106,31 @@ class AnlzFile(abc.Mapping):
             # Get the four byte struct type
             tag_type = tag_data[:4].decode("ascii")
 
+            if tag_type == "PSSI":
+                # The version that rekordbox 6 *exports* is garbled with an XOR mask.
+                # All bytes after byte 17 (len_e) are XOR-masked with a pattern that is
+                # generated by adding the value of len_e to each byte of XOR_MASK
+
+                # Check if the file is garbled (only on exported files)
+                # For this we check the validity of mood and bank
+                # Mood: High=1, Mid=2, Low=3
+                # Bank: 1-8
+                mood = Int16ub.parse(tag_data[18:20])
+                bank = Int16ub.parse(tag_data[28:30])
+                if 1 <= mood <= 3 and 1 <= bank <= 8:
+                    logger.debug("PSSI is not garbled!")
+                else:
+                    logger.debug("PSSI is garbled!")
+                    # deobfuscate tag_data[18:] using xor with XOR_MASK+len_entries
+                    len_entries = Int16ub.parse(tag_data[16:])
+                    tag_data = bytearray(data[i : i + len(tag_data)])
+                    for x in range(len(tag_data[18:])):
+                        mask = XOR_MASK[x % len(XOR_MASK)] + len_entries
+                        if mask > 255:
+                            mask -= 256
+                        tag_data[x + 18] ^= mask
+                    tag_data = bytes(tag_data)
+
             try:
                 # Parse the struct
                 tag = TAGS[tag_type](tag_data)
@@ -178,6 +206,17 @@ class AnlzFile(abc.Mapping):
         else:
             return [tag for tag in self.tags if tag.name == item]
 
+    def __contains__(self, item):
+        if item.isupper() and len(item) == 4:
+            for tag in self.tags:
+                if item == tag.type:
+                    return True
+        else:
+            for tag in self.tags:
+                if item == tag.name:
+                    return True
+        return False
+
     def __repr__(self):
         return f"{self.__class__.__name__}({self.tag_types})"
 

pyrekordbox/anlz/structs.py

@@ -195,8 +195,6 @@ PWV5 = Struct(
 
 # -- Song Structure Tag (PSSI) ---------------------------------------------------------
 
-# FixMe: Implement reverse XOR mask to descramble values
-
 SongStructureEntry = Struct(
     "index" / Int16ub,
     "beat" / Int16ub,
@@ -221,11 +219,11 @@ SongStructureEntry = Struct(
 PSSI = Struct(
     "len_entry_bytes" /  Const(24, Int32ub),
     "len_entries" / Int16ub,
-    "mood" / Bytes(2),
+    "mood" / Int16ub,
     "u1" / Bytes(6),
-    "end_beat" / Bytes(2),
+    "end_beat" / Int16ub,
     "u2" / Bytes(2),
-    "bank" / Bytes(1),
+    "bank" / Int8ub,
     "u3" / Bytes(1),
     "entries" / Array(this.len_entries, SongStructureEntry),
 )

pyrekordbox/config.py

@@ -15,6 +15,7 @@ import logging
 import base64
 import blowfish
 import json
+import packaging.version
 import xml.etree.cElementTree as xml
 from pathlib import Path
 from typing import Union
@@ -247,6 +248,7 @@ def _get_rb_config(
     pioneer_install_dir: Path,
     pioneer_app_dir: Path,
     major_version: int,
+    application_dirname: str = "",
 ) -> dict:
     """Get the program configuration for a given Rekordbox major version.
 
@@ -258,33 +260,55 @@ def _get_rb_config(
         The path of the Pioneer application data directory.
     major_version : int
         The major version of Rekordbox.
+    application_dirname : str, optional
+        The name of the Rekordbox application directory. If not given, the latest
+        Rekordbox installation directory for major version `major_version` is used.
 
     Returns
     -------
     config : dict
         The program configuration.
     """
-    # Get latest Rekordbox installation directory for major release `major_version`
-
-    # Find all 'V.x.x' version strings in dir names
-    versions = list()
-    for p in pioneer_install_dir.iterdir():
-        name = p.name
-        if name.startswith("rekordbox"):
-            ver_str = name.replace("rekordbox", "").strip()
-            if ver_str.startswith(str(major_version)):
-                versions.append(ver_str)
-    # Get latest 'V.x.x' version string and assure there is one
-    versions.sort(key=lambda s: list(map(int, s.split("."))))
-    try:
-        rb_version = versions[-1]
-    except IndexError:
-        raise FileNotFoundError(
-            f"No Rekordbox {major_version} folder found in installation "
-            f"directory '{pioneer_install_dir}'"
-        )
-    # Name of the Rekordbox application directory in `pioneer_install_dir`
-    rb_prog_dir = pioneer_install_dir / f"rekordbox {rb_version}"
+    if application_dirname:
+        # Applitcation dirname is given, only extract version from it
+        # `major_version` is compared to the version string
+        rb_version = application_dirname.replace("rekordbox", "").strip()
+        rb_version = rb_version.replace(".app", "")
+        if not rb_version.startswith(str(major_version)):
+            raise ValueError(
+                f"Major version is {major_version}, but the supplied application "
+                f"dirname is '{application_dirname}'"
+            )
+        rb_prog_dir = pioneer_install_dir / application_dirname
+        if not rb_prog_dir.exists():
+            raise InvalidApplicationDirname(
+                f"The supplied application dirname '{application_dirname}' does not "
+                f"exist in '{pioneer_install_dir}'"
+            )
+    else:
+        # Get latest Rekordbox installation directory for major release `major_version`
+
+        # Find all 'V.x.x' version strings in dir names
+        versions = list()
+        for p in pioneer_install_dir.iterdir():
+            name = p.name
+            if name.startswith("rekordbox"):
+                ver_str = name.replace("rekordbox", "").strip()
+                ver_str = ver_str.replace(".app", "")
+                if ver_str.startswith(str(major_version)):
+                    v = packaging.version.parse(ver_str)
+                    versions.append(v)
+        # Get latest 'V.x.x' version string and assure there is one
+        versions.sort()  # key=lambda s: list(map(int, s.split("."))))
+        try:
+            rb_version = str(versions[-1])
+        except IndexError:
+            raise FileNotFoundError(
+                f"No Rekordbox {major_version} folder found in installation "
+                f"directory '{pioneer_install_dir}'"
+            )
+        # Name of the Rekordbox application directory in `pioneer_install_dir`
+        rb_prog_dir = pioneer_install_dir / f"rekordbox {rb_version}"
 
     # Check installation directory
     if not rb_prog_dir.exists():
@@ -318,10 +342,12 @@ def _get_rb_config(
     return conf
 
 
-def _get_rb5_config(pioneer_prog_dir: Path, pioneer_app_dir: Path) -> dict:
+def _get_rb5_config(
+    pioneer_prog_dir: Path, pioneer_app_dir: Path, dirname: str = ""
+) -> dict:
     """Get the program configuration for Rekordbox v5.x.x."""
     major_version = 5
-    conf = _get_rb_config(pioneer_prog_dir, pioneer_app_dir, major_version)
+    conf = _get_rb_config(pioneer_prog_dir, pioneer_app_dir, major_version, dirname)
     return conf
 
 
@@ -377,10 +403,12 @@ def write_db6_key_cache(key: str) -> None:  # pragma: no cover
     __config__["rekordbox6"]["dp"] = key
 
 
-def _get_rb6_config(pioneer_prog_dir: Path, pioneer_app_dir: Path) -> dict:
+def _get_rb6_config(
+    pioneer_prog_dir: Path, pioneer_app_dir: Path, dirname: str = ""
+) -> dict:
     """Get the program configuration for Rekordbox v6.x.x."""
     major_version = 6
-    conf = _get_rb_config(pioneer_prog_dir, pioneer_app_dir, major_version)
+    conf = _get_rb_config(pioneer_prog_dir, pioneer_app_dir, major_version, dirname)
 
     # Read Rekordbox 6 'options.json' and check db_path
     opts = read_rekordbox6_options(pioneer_app_dir)
@@ -488,6 +516,8 @@ def read_pyrekordbox_configuration():
 def update_config(
     pioneer_install_dir: Union[str, Path] = None,
     pioneer_app_dir: Union[str, Path] = None,
+    rb5_install_dirname: str = "",
+    rb6_install_dirname: str = "",
 ):
     """Update the pyrekordbox configuration.
 
@@ -508,6 +538,12 @@ def update_config(
         The path to the Pioneer application directory. This is where the application
         user data of Pioneer programs is stored. By default, the normal location of
         the Pioneer application data is used.
+    rb5_install_dirname : str, optional
+        The name of the Rekordbox 5 installation directory. By default, the normal
+        directory name is used (Windows: 'rekordbox 5.x.x', macOS: 'rekordbox 5.app').
+    rb6_install_dirname : str, optional
+        The name of the Rekordbox 6 installation directory. By default, the normal
+        directory name is used (Windows: 'rekordbox 6.x.x', macOS: 'rekordbox 6.app').
     """
     # Read config file
     conf = read_pyrekordbox_configuration()
@@ -515,6 +551,10 @@ def update_config(
         pioneer_install_dir = conf["pioneer-install-dir"]
     if pioneer_app_dir is None and "pioneer-app-dir" in conf:
         pioneer_app_dir = conf["pioneer-app-dir"]
+    if not rb5_install_dirname and "rekordbox5-install-dirname" in conf:
+        rb5_install_dirname = conf["rekordbox5-install-dirname"]
+    if not rb6_install_dirname and "rekordbox6-install-dirname" in conf:
+        rb6_install_dirname = conf["rekordbox6-install-dirname"]
 
     # Pioneer installation directory
     try:
@@ -534,14 +574,18 @@ def update_config(
 
     # Update Rekordbox 5 config
     try:
-        conf = _get_rb5_config(pioneer_install_dir, pioneer_app_dir)
+        conf = _get_rb5_config(
+            pioneer_install_dir, pioneer_app_dir, rb5_install_dirname
+        )
         __config__["rekordbox5"].update(conf)
     except FileNotFoundError as e:
         logger.info(e)
 
     # Update Rekordbox 6 config
     try:
-        conf = _get_rb6_config(pioneer_install_dir, pioneer_app_dir)
+        conf = _get_rb6_config(
+            pioneer_install_dir, pioneer_app_dir, rb6_install_dirname
+        )
         __config__["rekordbox6"].update(conf)
     except FileNotFoundError as e:
         logger.info(e)