twitch-archiver 3.0.0.dev1__tar.gz → 3.0.0.dev5__tar.gz

{twitch-archiver-3.0.0.dev1/twitch_archiver.egg-info → twitch-archiver-3.0.0.dev5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: twitch-archiver
-Version: 3.0.0.dev1
+Version: 3.0.0.dev5
 Summary: A simple, fast, platform-independent tool for downloading Twitch streams, videos, and chat logs.
 Author-email: Brisppy <brisppy@protonmail.com>
 Project-URL: Homepage, https://github.com/Brisppy/twitch-archiver
@@ -40,7 +40,7 @@ Primarily focused on data preservation, this script can be used to archive an en
 ## Features
 * Archives both video and chat logs.
 * VODs can be downloaded as fast as your Internet connection (and storage) can handle.[^1]
-* Allows the downloading of live VODs before sections are muted or deleted.[^2]
+* Allows real-time archiving of Twitch streams.[^2]
 * Generates a readable chat log with timestamps and user badges.
 * Supports downloading streams which aren't being archived by Twitch.
 * Error notifications supported with pushbullet.
@@ -68,8 +68,9 @@ Make sure to read the [usage](#usage) section after installation.
 
 1. Either download the repository via the green code button at the top of the page, or grab the latest release [here](https://github.com/Brisppy/twitch-archiver/releases/latest).
 2. Unpack the archive and enter the directory with `cd twitch-archiver`.
-2. Install [pip](https://pip.pypa.io/en/stable/installation/) if you do not already have it.
-3. Build the package with `python -m build`, then install with `python -m pip install ./dist/twitch-archiver-*.tar.gz`.
+3. Install [pip](https://pip.pypa.io/en/stable/installation/) if you do not already have it.
+4. Install Python "Build" package with `python -m pip install --upgrade build`.
+5. Build the package with `python -m build`, then install with `python -m pip install ./dist/twitch-archiver-*.tar.gz`.
 
 #### Installing as a Docker Container
 1. Either download the repository via the green code button at the top of the page, or grab the latest release [here](https://github.com/Brisppy/twitch-archiver/releases/latest).
@@ -102,9 +103,9 @@ Would download VODs **1276315849** and **1275305106** to the directory **/mnt/tw
 #### Arguments
 Below is the output of the `--help` or `-h` command. This displays all the available arguments and a brief description of how to use them.
 ```
-usage: twitch-archiver [-h] (-c CHANNEL | -v VOD_ID) [-i CLIENT_ID] [-s CLIENT_SECRET] [-C] [-V]
-                       [-t THREADS] [-q QUALITY] [-d DIRECTORY] [-w] [-L LOG_FILE] [-I CONFIG_DIR]
-                       [-p PUSHBULLET_KEY] [-Q | -D] [--version] [--show-config]
+usage: twitch-archiver [-h] (-c CHANNEL | -v VOD_ID) [-i CLIENT_ID] [-s CLIENT_SECRET] [-C] [-V] 
+                       [-t THREADS] [-q QUALITY] [-d DIRECTORY] [-w] [-l | -a | -R] [-L LOG_FILE] 
+                       [-I CONFIG_DIR] [-p PUSHBULLET_KEY] [-Q | -D] [--version] [--show-config]
 
 requires one of:
     -c CHANNEL, --channel CHANNEL
@@ -141,9 +142,12 @@ optional arguments:
   -d DIRECTORY, --directory DIRECTORY
                         Directory to store archived VOD(s), use TWO slashes for Windows paths.
                         (default: $CURRENT_DIRECTORY)
-  -w, --watch           Continually check every 10 seconds for new streams from the specified channel.
-  -S, --stream-only     Only download streams which are currently live.
-  -N, --no-stream       Don't download streams which are currently live.
+  -w, --watch           Continually check every 10 seconds for new streams/VODs from a specified channel.
+  -l, --live-only       Only download streams / VODs which are currently live.
+  -a, --archive-only    Don't download streams / VODs which are currently live.
+  -R, --real-time-archiver
+                        Enable real-time stream archiver.
+                        Read https://github.com/Brisppy/twitch-archiver/wiki/Wiki#real-time-archiver.
   -L LOG_FILE, --log-file LOG_FILE
                         Output logs to specified file.
   -I CONFIG_DIR, --config-dir CONFIG_DIR

{twitch-archiver-3.0.0.dev1 → twitch-archiver-3.0.0.dev5}/README.md

@@ -26,7 +26,7 @@ Primarily focused on data preservation, this script can be used to archive an en
 ## Features
 * Archives both video and chat logs.
 * VODs can be downloaded as fast as your Internet connection (and storage) can handle.[^1]
-* Allows the downloading of live VODs before sections are muted or deleted.[^2]
+* Allows real-time archiving of Twitch streams.[^2]
 * Generates a readable chat log with timestamps and user badges.
 * Supports downloading streams which aren't being archived by Twitch.
 * Error notifications supported with pushbullet.
@@ -54,8 +54,9 @@ Make sure to read the [usage](#usage) section after installation.
 
 1. Either download the repository via the green code button at the top of the page, or grab the latest release [here](https://github.com/Brisppy/twitch-archiver/releases/latest).
 2. Unpack the archive and enter the directory with `cd twitch-archiver`.
-2. Install [pip](https://pip.pypa.io/en/stable/installation/) if you do not already have it.
-3. Build the package with `python -m build`, then install with `python -m pip install ./dist/twitch-archiver-*.tar.gz`.
+3. Install [pip](https://pip.pypa.io/en/stable/installation/) if you do not already have it.
+4. Install Python "Build" package with `python -m pip install --upgrade build`.
+5. Build the package with `python -m build`, then install with `python -m pip install ./dist/twitch-archiver-*.tar.gz`.
 
 #### Installing as a Docker Container
 1. Either download the repository via the green code button at the top of the page, or grab the latest release [here](https://github.com/Brisppy/twitch-archiver/releases/latest).
@@ -88,9 +89,9 @@ Would download VODs **1276315849** and **1275305106** to the directory **/mnt/tw
 #### Arguments
 Below is the output of the `--help` or `-h` command. This displays all the available arguments and a brief description of how to use them.
 ```
-usage: twitch-archiver [-h] (-c CHANNEL | -v VOD_ID) [-i CLIENT_ID] [-s CLIENT_SECRET] [-C] [-V]
-                       [-t THREADS] [-q QUALITY] [-d DIRECTORY] [-w] [-L LOG_FILE] [-I CONFIG_DIR]
-                       [-p PUSHBULLET_KEY] [-Q | -D] [--version] [--show-config]
+usage: twitch-archiver [-h] (-c CHANNEL | -v VOD_ID) [-i CLIENT_ID] [-s CLIENT_SECRET] [-C] [-V] 
+                       [-t THREADS] [-q QUALITY] [-d DIRECTORY] [-w] [-l | -a | -R] [-L LOG_FILE] 
+                       [-I CONFIG_DIR] [-p PUSHBULLET_KEY] [-Q | -D] [--version] [--show-config]
 
 requires one of:
     -c CHANNEL, --channel CHANNEL
@@ -127,9 +128,12 @@ optional arguments:
   -d DIRECTORY, --directory DIRECTORY
                         Directory to store archived VOD(s), use TWO slashes for Windows paths.
                         (default: $CURRENT_DIRECTORY)
-  -w, --watch           Continually check every 10 seconds for new streams from the specified channel.
-  -S, --stream-only     Only download streams which are currently live.
-  -N, --no-stream       Don't download streams which are currently live.
+  -w, --watch           Continually check every 10 seconds for new streams/VODs from a specified channel.
+  -l, --live-only       Only download streams / VODs which are currently live.
+  -a, --archive-only    Don't download streams / VODs which are currently live.
+  -R, --real-time-archiver
+                        Enable real-time stream archiver.
+                        Read https://github.com/Brisppy/twitch-archiver/wiki/Wiki#real-time-archiver.
   -L LOG_FILE, --log-file LOG_FILE
                         Output logs to specified file.
   -I CONFIG_DIR, --config-dir CONFIG_DIR

{twitch-archiver-3.0.0.dev1 → twitch-archiver-3.0.0.dev5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "twitch-archiver"
-version = "3.0.0.dev1"
+version = "3.0.0.dev5"
 authors = [
   { name="Brisppy", email="brisppy@protonmail.com" },
 ]

{twitch-archiver-3.0.0.dev1 → twitch-archiver-3.0.0.dev5/twitch_archiver.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: twitch-archiver
-Version: 3.0.0.dev1
+Version: 3.0.0.dev5
 Summary: A simple, fast, platform-independent tool for downloading Twitch streams, videos, and chat logs.
 Author-email: Brisppy <brisppy@protonmail.com>
 Project-URL: Homepage, https://github.com/Brisppy/twitch-archiver
@@ -40,7 +40,7 @@ Primarily focused on data preservation, this script can be used to archive an en
 ## Features
 * Archives both video and chat logs.
 * VODs can be downloaded as fast as your Internet connection (and storage) can handle.[^1]
-* Allows the downloading of live VODs before sections are muted or deleted.[^2]
+* Allows real-time archiving of Twitch streams.[^2]
 * Generates a readable chat log with timestamps and user badges.
 * Supports downloading streams which aren't being archived by Twitch.
 * Error notifications supported with pushbullet.
@@ -68,8 +68,9 @@ Make sure to read the [usage](#usage) section after installation.
 
 1. Either download the repository via the green code button at the top of the page, or grab the latest release [here](https://github.com/Brisppy/twitch-archiver/releases/latest).
 2. Unpack the archive and enter the directory with `cd twitch-archiver`.
-2. Install [pip](https://pip.pypa.io/en/stable/installation/) if you do not already have it.
-3. Build the package with `python -m build`, then install with `python -m pip install ./dist/twitch-archiver-*.tar.gz`.
+3. Install [pip](https://pip.pypa.io/en/stable/installation/) if you do not already have it.
+4. Install Python "Build" package with `python -m pip install --upgrade build`.
+5. Build the package with `python -m build`, then install with `python -m pip install ./dist/twitch-archiver-*.tar.gz`.
 
 #### Installing as a Docker Container
 1. Either download the repository via the green code button at the top of the page, or grab the latest release [here](https://github.com/Brisppy/twitch-archiver/releases/latest).
@@ -102,9 +103,9 @@ Would download VODs **1276315849** and **1275305106** to the directory **/mnt/tw
 #### Arguments
 Below is the output of the `--help` or `-h` command. This displays all the available arguments and a brief description of how to use them.
 ```
-usage: twitch-archiver [-h] (-c CHANNEL | -v VOD_ID) [-i CLIENT_ID] [-s CLIENT_SECRET] [-C] [-V]
-                       [-t THREADS] [-q QUALITY] [-d DIRECTORY] [-w] [-L LOG_FILE] [-I CONFIG_DIR]
-                       [-p PUSHBULLET_KEY] [-Q | -D] [--version] [--show-config]
+usage: twitch-archiver [-h] (-c CHANNEL | -v VOD_ID) [-i CLIENT_ID] [-s CLIENT_SECRET] [-C] [-V] 
+                       [-t THREADS] [-q QUALITY] [-d DIRECTORY] [-w] [-l | -a | -R] [-L LOG_FILE] 
+                       [-I CONFIG_DIR] [-p PUSHBULLET_KEY] [-Q | -D] [--version] [--show-config]
 
 requires one of:
     -c CHANNEL, --channel CHANNEL
@@ -141,9 +142,12 @@ optional arguments:
   -d DIRECTORY, --directory DIRECTORY
                         Directory to store archived VOD(s), use TWO slashes for Windows paths.
                         (default: $CURRENT_DIRECTORY)
-  -w, --watch           Continually check every 10 seconds for new streams from the specified channel.
-  -S, --stream-only     Only download streams which are currently live.
-  -N, --no-stream       Don't download streams which are currently live.
+  -w, --watch           Continually check every 10 seconds for new streams/VODs from a specified channel.
+  -l, --live-only       Only download streams / VODs which are currently live.
+  -a, --archive-only    Don't download streams / VODs which are currently live.
+  -R, --real-time-archiver
+                        Enable real-time stream archiver.
+                        Read https://github.com/Brisppy/twitch-archiver/wiki/Wiki#real-time-archiver.
   -L LOG_FILE, --log-file LOG_FILE
                         Output logs to specified file.
   -I CONFIG_DIR, --config-dir CONFIG_DIR

{twitch-archiver-3.0.0.dev1 → twitch-archiver-3.0.0.dev5}/twitcharchiver/__init__.py

@@ -41,9 +41,9 @@ from twitcharchiver.exceptions import TwitchAPIError
 from twitcharchiver.logger import Logger
 from twitcharchiver.processing import Processing
 from twitcharchiver.twitch import Twitch
-from twitcharchiver.utils import getenv, send_push, get_latest_version, version_tuple
+from twitcharchiver.utils import getenv, send_push, get_latest_version, version_tuple, check_update_available
 
-__version__ = '3.0.0'
+__version__ = '3.0.0.dev5'
 
 
 def main():
@@ -105,24 +105,28 @@ def main():
     parser.add_argument('-w', '--watch', action='store_true',
                         help='Continually check every 10 seconds for new streams/VODs from a specified channel.',
                         default=getenv('TWITCH_ARCHIVER_WATCH', False, True))
-    stream.add_argument('-S', '--stream-only', action='store_true',
-                        default=getenv('TWITCH_ARCHIVER_STREAM_ONLY', False, True),
-                        help='Only download streams which are currently live.')
-    stream.add_argument('-N', '--no-stream', action='store_true',
-                        help="Don't download streams which are currently live.",
-                        default=getenv("TWITCH_ARCHIVER_NO_STREAM", False, True))
+    stream.add_argument('-l', '--live-only', action='store_true',
+                        default=getenv('TWITCH_ARCHIVER_LIVE_ONLY', False, True),
+                        help='Only download streams / VODs which are currently live.')
+    stream.add_argument('-a', '--archive-only', action='store_true',
+                        help="Don't download streams / VODs which are currently live.",
+                        default=getenv("TWITCH_ARCHIVER_ARCHIVE_ONLY", False, True))
+    stream.add_argument('-R', '--real-time-archiver', action='store_true',
+                        help="Enable real-time stream archiver.\n"
+                             "Read https://github.com/Brisppy/twitch-archiver/wiki/Wiki#real-time-archiver.",
+                        default=getenv('TWITCH_ARCHIVER_REAL_TIME_ARCHIVER', False, True))
     parser.add_argument('-L', '--log-file', action='store', help='Output logs to specified file.', type=Path,
                         default=getenv("TWITCH_ARCHIVER_LOG_FILE", False))
     parser.add_argument('-I', '--config-dir', action='store', type=Path,
                         help='Directory to store configuration, VOD database and lock files.\n(default: %(default)s)',
                         default=getenv('TWITCH_ARCHIVER_CONFIG_DIR',
-                                             Path(os.path.expanduser("~"), '.config', 'twitch-archiver')))
+                                       Path(os.path.expanduser("~"), '.config', 'twitch-archiver')))
     parser.add_argument('-p', '--pushbullet-key', action='store',
                         help='Pushbullet key for sending pushes on error. Enabled by supplying key.',
                         default=getenv("TWITCH_ARCHIVER_PUSHBULLET_KEY", False))
     loglevel.add_argument('-Q', '--quiet', action='store_const', help='Disable all log output.', const=50, default=0)
     loglevel.add_argument('-D', '--debug', action='store_const', help='Enable debug logs.', const=10, default=0)
-    parser.add_argument('--version', action='version', version=f'{__name__} v{__version__}',
+    parser.add_argument('--version', action='version', version=f'Twitch Archiver v{__version__}',
                         help='Show version number and exit.')
     parser.add_argument('--show-config', action='store_true', help='Show saved config and exit.', default=False)
 
@@ -144,7 +148,7 @@ def main():
 
     # compare with current git version
     latest_version, release_notes = get_latest_version()
-    if version_tuple(__version__) < version_tuple(latest_version):
+    if check_update_available(__version__, latest_version):
         log.warning('New version of Twitch-Archiver available - Version %s:\n'
                     'https://github.com/Brisppy/twitch-archiver/releases/latest\nRelease notes:\n\n%s\n',
                     latest_version, release_notes)
@@ -177,7 +181,7 @@ def main():
             # store returned token
             config.save(Path(args.get('config_dir'), 'config.ini'))
         except TwitchAPIError as err:
-            log.error('OAuth token generation failed. Error: $s', str(err))
+            log.error('OAuth token generation failed. Error: %s', str(err))
             send_push(config.get('pushbullet_key'), 'OAuth token generation failed.', str(err))
             sys.exit(1)
 

{twitch-archiver-3.0.0.dev1 → twitch-archiver-3.0.0.dev5}/twitcharchiver/arguments.py

@@ -33,7 +33,7 @@ class Arguments:
 
         # validate mutual exclusivity of arguments passed via CLI and environment variables
         # required as values set via environment variables bypass argparse mutex handling
-        for mutex_args in (("vod_id", "channel"), ("stream_only", "no_stream")):
+        for mutex_args in (("vod_id", "channel"), ("live_only", "archive_only")):
             mutex_arg_0, mutex_arg_1 = Arguments.get(mutex_args[0]), Arguments.get(mutex_args[1])
             # check if both mutex args have a value (including empty string)
             if mutex_arg_0 is not None and mutex_arg_1 is not None:

{twitch-archiver-3.0.0.dev1 → twitch-archiver-3.0.0.dev5}/twitcharchiver/processing.py

@@ -45,8 +45,8 @@ class Processing:
         self.video = args['video']
         self.chat = args['chat']
         self.quality = args['quality']
-        self.stream_only = args['stream_only']
-        self.no_stream = args['no_stream']
+        self.live_only = args['live_only']
+        self.archive_only = args['archive_only']
         self.config_dir = args['config_dir']
         self.quiet = args['quiet']
         self.debug = args['debug']
@@ -109,10 +109,11 @@ class Processing:
                 stream_length = time_since_date(datetime.strptime(
                     channel_data[0]['started_at'], '%Y-%m-%dT%H:%M:%SZ').replace(tzinfo=timezone.utc).timestamp())
 
+                # TODO: ensure that if the stream ends, we clean up and merge segments
                 # while we wait for the api to update we must build a temporary buffer of any parts advertised in the
                 # meantime in case there is no vod and thus no way to retrieve them after the fact
-                if stream_length < 60:
-                    self.log.debug('Stream began less than 60s ago, delaying archival start until VOD API updated.')
+                if stream_length < 90:
+                    self.log.debug('Stream began less than 90s ago, delaying archival start until VOD API updated.')
                     # create temp dir for buffer
                     Path(tmp_buffer_dir).mkdir(parents=True, exist_ok=True)
 
@@ -120,7 +121,7 @@ class Processing:
                     index_uri = self.call_twitch.get_channel_hls_index(channel, self.quality)
 
                     # download new parts every 4s
-                    for i in range(int((60 - stream_length) / 4)):
+                    for i in range(int((90 - stream_length) / 4)):
                         # grab required values
                         start_timestamp = int(datetime.utcnow().timestamp())
                         incoming_segments = m3u8.loads(Api.get_request(index_uri).text).data
@@ -135,7 +136,7 @@ class Processing:
                             sleep(4 - processing_time)
 
                     # wait any remaining time
-                    sleep((60 - stream_length) % 4)
+                    sleep((90 - stream_length) % 4)
 
             # retrieve available vods
             available_vods: dict[int: tuple[int]] = {}
@@ -154,8 +155,7 @@ class Processing:
                 self.log.error('Error retrieving VODs from Twitch. Error: %s', str(e))
                 continue
 
-            self.log.info(f'Online VODs: {available_vods}' if self.debug
-                          else f'Online VODs: {len(available_vods)}')
+            self.log.debug(f'Online VODs: {available_vods}')
 
             # retrieve downloaded vods
             with Database(Path(self.config_dir, 'vods.db')) as db:
@@ -163,8 +163,7 @@ class Processing:
                 downloaded_vods = dict([(i[0], (i[1], i[2], i[3])) for i in db.execute_query(
                     'SELECT stream_id, vod_id, video_archived, chat_archived FROM vods WHERE user_id IS ?',
                     {'user_id': user_id})])
-            self.log.info(f'Downloaded vods: {downloaded_vods}' if self.debug
-                          else f'Downloaded vods: {len(downloaded_vods)}')
+            self.log.debug(f'Downloaded vods: {downloaded_vods}')
 
             # generate vod queue using downloaded and available vods
             vod_queue = {}
@@ -190,15 +189,18 @@ class Processing:
             live_vod_exists = (channel_data and int(channel_data[0]['id']) in available_vods.keys())
 
             # move on if channel offline and no vods are available
-            if not self.stream_only and not channel_live and not available_vods:
+            if not self.live_only and not channel_live and not available_vods:
                 self.log.info('No VODs were found for %s.', user_name)
                 continue
 
-            if not channel_live and self.stream_only:
+            # move on if channel offline and we are only looking for live vods
+            if not channel_live and self.live_only:
+                self.log.info('Running in stream-only mode and no stream available for %s.', user_name)
                 continue
 
-            # archive stream in non-segmented mode if no paired vod exists
-            if not self.no_stream and channel_live and not live_vod_exists and self.video:
+            # archive stream in non-segmented mode if no paired vod exists, unless we are in no_stream mode or not
+            # archiving video.
+            if not self.archive_only and channel_live and not live_vod_exists and self.video:
                 self.log.info('Channel live but not being archived to a VOD, running stream archiver.')
                 self.log.debug('Creating lock file for stream.')
 
@@ -214,7 +216,7 @@ class Processing:
 
                     # Check if stream id in database
                     if channel_data[0]['id'] in downloaded_streams:
-                        self.log.info('Stream has already been downloaded.')
+                        self.log.info('Ignoring steam as it has already been downloaded.')
 
                     else:
                         try:
@@ -271,11 +273,13 @@ class Processing:
             # begin processing each available vod
             for stream_id in vod_queue:
                 vod_id = vod_queue[stream_id][0]
-                # skip if we are only after the current stream
-                if channel_data and self.stream_only and stream_id != int(channel_data[0]['id']):
+
+                # skip if we are only after currently live streams, and stream_id is not live
+                if channel_data and self.live_only and stream_id != int(channel_data[0]['id']):
                     continue
 
-                if channel_data and self.no_stream and stream_id == int(channel_data[0]['id']):
+                # skip if we aren't after currently lives streams, and stream_id is live
+                if channel_data and self.archive_only and stream_id == int(channel_data[0]['id']):
                     self.log.info('Skipping VOD as it is live and no-stream argument provided.')
                     continue
 
@@ -432,7 +436,8 @@ class Processing:
         _r = None
 
         try:
-            if vod_live:
+            # begin real-time archiver if VOD still live and real-time archiver enabled
+            if self.real_time_archiver and vod_live:
                 stream = Stream(self.client_id, self.client_secret, self.oauth_token)
                 # concurrently grab live pieces and vod chunks
 

{twitch-archiver-3.0.0.dev1 → twitch-archiver-3.0.0.dev5}/twitcharchiver/stream.py

@@ -82,14 +82,25 @@ class Stream:
         while True:
             start_timestamp = int(datetime.utcnow().timestamp())
 
-            try:
-                self.log.debug('Fetching incoming stream segments.')
-                incoming_segments = m3u8.loads(Api.get_request(index_uri).text).data
+            # attempt to grab new segments from Twitch
+            for attempt in range(6):
+                if attempt > 4:
+                    return
 
-            # stream has ended if exception encountered
-            except TwitchAPIErrorNotFound:
-                self.get_final_segment(self.buffer, output_dir, self.segment_ids)
-                return
+                try:
+                    self.log.debug('Fetching incoming stream segments.')
+                    incoming_segments = m3u8.loads(Api.get_request(index_uri).text).data
+                    break
+
+                # stream has ended if exception encountered
+                except TwitchAPIErrorNotFound:
+                    self.get_final_segment(self.buffer, output_dir, self.segment_ids)
+                    return
+
+                except requests.exceptions.ConnectTimeout:
+                    self.log.debug('Timed out attempting to fetch new stream segments, retrying. (Attempt %s)',
+                                   attempt + 1)
+                    continue
 
             # set latest segment and timestamp if new segment found
             if incoming_segments['segments'][-1]['uri'] != latest_segment:

{twitch-archiver-3.0.0.dev1 → twitch-archiver-3.0.0.dev5}/twitcharchiver/utils.py

@@ -41,7 +41,7 @@ def generate_readable_chat_log(chat_log, stream_start):
             created_time = \
                 datetime.strptime(comment['createdAt'], '%Y-%m-%dT%H:%M:%SZ').replace(tzinfo=timezone.utc)
 
-        comment_time = '{:.3f}'.format(get_time_difference(stream_start, created_time))
+        comment_time = f'{get_time_difference(stream_start, created_time):.3f}'
 
         # catch comments without commenter informations
         if comment['commenter']:
@@ -107,7 +107,7 @@ def export_json(vod_json):
 
     :param vod_json: dict of vod parameters retrieved from twitch
     """
-    with open(Path(vod_json['store_directory'], 'vod.json'), 'w') as json_out_file:
+    with open(Path(vod_json['store_directory'], 'vod.json'), 'w', encoding='utf8') as json_out_file:
         json_out_file.write(json.dumps(vod_json))
 
 
@@ -117,10 +117,10 @@ def import_json(vod_json):
     :param vod_json: dict of vod parameters retrieved from twitch
     """
     if Path(vod_json['store_directory'], 'vod.json').exists():
-        with open(Path(vod_json['store_directory'], 'vod.json'), 'r') as json_in_file:
+        with open(Path(vod_json['store_directory'], 'vod.json'), 'r', encoding='utf8') as json_in_file:
             return json.loads(json_in_file.read())
 
-    return
+    return []
 
 
 def combine_vod_parts(vod_json, print_progress=True):
@@ -158,10 +158,10 @@ def combine_vod_parts(vod_json, print_progress=True):
         # merge all .ts files with ffmpeg concat demuxer as missing segments can cause corruption with
         # other method
 
-        log.debug(f'Discontinuity found, merging with ffmpeg.\n Discontinuity: {dicontinuity}')
+        log.debug('Discontinuity found, merging with ffmpeg.\n Discontinuity: %s', dicontinuity)
 
         # create file with list of parts for ffmpeg
-        with open(Path(vod_json['store_directory'], 'parts', 'segments.txt'), mode='w') as segment_file:
+        with open(Path(vod_json['store_directory'], 'parts', 'segments.txt'), 'w', encoding='utf8') as segment_file:
             for part in vod_parts:
                 segment_file.write(f"file '{part}'\n")
 
@@ -181,7 +181,7 @@ def combine_vod_parts(vod_json, print_progress=True):
                         progress.print_progress(int(current_time), vod_json['duration'])
 
             if p.returncode:
-                log.error(f'VOD merger exited with error. Command: {p.args}.')
+                log.error('VOD merger exited with error. Command: %s.', p.args)
                 raise VodConvertError(f'VOD merger exited with error. Command: {p.args}.')
 
 
@@ -204,9 +204,9 @@ def convert_vod(vod_json, ignore_corruptions=None, print_progress=True):
         [corrupt_part_whitelist.update(r) for r in [range(t[0] - 2, t[1] + 3) for t in ignore_corruptions]]
 
     # get dts offset of first part
-    with subprocess.Popen(
-            f'ffprobe -v quiet -print_format json -show_format -show_streams "{Path(vod_json["store_directory"], "parts", "00000.ts")}"', shell=True,
-            stdout=subprocess.PIPE, universal_newlines=True) as p:
+    with subprocess.Popen(f'ffprobe -v quiet -print_format json -show_format -show_streams '
+                          f'"{Path(vod_json["store_directory"], "parts", "00000.ts")}"', shell=True,
+                          stdout=subprocess.PIPE, universal_newlines=True) as p:
         ts_file_data = ''
         for line in p.stdout:
             ts_file_data += line
@@ -240,27 +240,26 @@ def convert_vod(vod_json, ignore_corruptions=None, print_progress=True):
                     dts_timestamp = int(re.search(r'(?<=dts = ).*(?=\).)', line).group(0))
 
                 # Catch corrupt parts without timestamp, shows up as 'NOPTS'
-                except ValueError:
+                except ValueError as e:
                     raise VodConvertError("Corrupt packet encountered at unknown timestamp while converting VOD. "
-                                          "Delete 'parts' folder and re-download VOD.")
+                                          "Delete 'parts' folder and re-download VOD.") from e
 
                 corrupt_part = floor((dts_timestamp - dts_offset) / 90000 / 10)
 
                 # ignore if corrupt packet within ignore_corruptions range
                 if corrupt_part in corrupt_part_whitelist:
-                    log.debug(f'Ignoring corrupt packet as part in whitelist. Part: {corrupt_part}')
-                    pass
+                    log.debug('Ignoring corrupt packet as part in whitelist. Part: %s', corrupt_part)
 
                 else:
                     corrupt_parts.add(int(corrupt_part))
-                    log.error(f'Corrupt packet encountered. Part: {corrupt_part}')
+                    log.error('Corrupt packet encountered. Part: %s', corrupt_part)
 
     if p.returncode:
         log.debug('FFmpeg exited with error code, output dumped to VOD directory.')
-        with open(Path(vod_json["store_directory"], 'parts', 'ffmpeg.log'), 'w') as ffout:
+        with open(Path(vod_json["store_directory"], 'parts', 'ffmpeg.log'), 'w', encoding='utf8') as ffout:
             ffout.write(ffmpeg_log)
 
-        raise VodConvertError(f"VOD converter exited with error. Delete 'parts' directory and re-download VOD.")
+        raise VodConvertError("VOD converter exited with error. Delete 'parts' directory and re-download VOD.")
 
     if corrupt_parts:
         corrupted_ranges = to_ranges(corrupt_parts)
@@ -277,6 +276,11 @@ def convert_vod(vod_json, ignore_corruptions=None, print_progress=True):
 
 # https://stackoverflow.com/a/43091576
 def to_ranges(iterable):
+    """Converts a list of integers to iterable sets of (low, high) (e.g [0, 1, 2, 5, 7, 8] -> (0, 2), (5, 5), (7, 8))
+
+    :param iterable: list of integers
+    :return: iterable generator of separate integer ranges
+    """
     iterable = sorted(set(iterable))
     for key, group in groupby(enumerate(iterable), lambda t: t[1] - t[0]):
         group = list(group)
@@ -302,25 +306,24 @@ def verify_vod_length(vod_json):
                        shell=True, capture_output=True)
 
     if p.returncode:
-        log.error(f'VOD length verification exited with error. Command: {p.args}.')
+        log.error('VOD length verification exited with error. Command: %s.', p.args)
         raise VodConvertError(f'VOD length verification exited with error. Command: {p.args}.')
 
     try:
         downloaded_length = int(float(p.stdout.decode('ascii').rstrip()))
 
     except Exception as e:
-        log.error(f'Failed to fetch downloaded VOD length. VOD may not have downloaded correctly. {e}')
-        raise VodConvertError(str(e))
+        log.error('Failed to fetch downloaded VOD length. VOD may not have downloaded correctly. %s', str(e))
+        raise VodConvertError(str(e)) from e
 
-    log.debug(f'Downloaded VOD length is {downloaded_length}. Expected length is {vod_json["duration"]}.')
+    log.debug('Downloaded VOD length is %s. Expected length is %s.', downloaded_length, vod_json["duration"])
 
     # pass verification if downloaded file is within 2s of expected length
     if 2 >= downloaded_length - vod_json['duration'] >= -2:
         log.debug('VOD passed length verification.')
         return False
 
-    else:
-        return True
+    return True
 
 
 def cleanup_vod_parts(vod_directory):
@@ -364,10 +367,10 @@ def convert_to_seconds(duration):
     if len(duration) == 1:
         return int(duration[0])
 
-    elif len(duration) == 2:
+    if len(duration) == 2:
         return (int(duration[0]) * 60) + int(duration[1])
 
-    elif len(duration) == 3:
+    if len(duration) == 3:
         return (int(duration[0]) * 3600) + (int(duration[1]) * 60) + int(duration[2])
 
 
@@ -380,7 +383,7 @@ def convert_to_hms(seconds):
     minutes = seconds // 60
     hours = minutes // 60
 
-    return "%02dh%02dm%02ds" % (hours, minutes % 60, seconds % 60)
+    return f"{hours:02d}h{minutes % 60:02d}m{seconds % 60:02d}s"
 
 
 def create_lock(ini_path, vod_id):
@@ -391,8 +394,9 @@ def create_lock(ini_path, vod_id):
     :return: true if lock file creation fails
     """
     try:
-        with open(Path(ini_path, f'.lock.{vod_id}'), 'x') as _:
+        with open(Path(ini_path, f'.lock.{vod_id}'), 'x', encoding='utf8') as _:
             pass
+        return
 
     except FileExistsError:
         return True
@@ -407,6 +411,7 @@ def remove_lock(config_dir, vod_id):
     """
     try:
         Path(config_dir, f'.lock.{vod_id}').unlink()
+        return
 
     except Exception as e:
         return e
@@ -459,9 +464,38 @@ def get_latest_version():
 # reference:
 #   https://stackoverflow.com/a/11887825
 def version_tuple(v):
+    """Convert
+
+    :param v:
+    :return:
+    """
     return tuple(map(int, (v.split("."))))
 
 
+def check_update_available(local_version, remote_version):
+    """Compares two software versions.
+
+    :param local_version: local version in use
+    :param remote_version: remote version to compare against
+    :return: True if remote version has a higher version number, otherwise False
+    """
+    # check if local version is 'special', as in a development build or release candidate
+    local_version_parts = local_version.split('.')
+    if len(local_version_parts) > 3:
+        log.warning(
+            'Currently using a development or release candidate build. These may be unfinished or contain serious '
+            'bugs. Report any issues you encounter to https://github.com/Brisppy/twitch-archiver/issues.')
+        # update is available if we are using a dev or release candidate build equal to or prior to
+        # the latest stable release
+        if version_tuple('.'.join(local_version_parts[:-1])) <= version_tuple(remote_version):
+            return True
+
+    elif version_tuple(local_version) < version_tuple(remote_version):
+        return True
+
+    return False
+
+
 def get_quality_index(desired_quality, available_qualities):
     """Finds the index of a user defined quality from a list of available stream qualities.
 
@@ -509,13 +543,13 @@ def send_push(pushbullet_key, title, body=''):
             if _r.status_code != 200:
                 if _r.json()['error']['code'] == 'pushbullet_pro_required':
                     log.error('Error sending push. Likely rate limited (500/month). '
-                              f'Error {_r.status_code}: {_r.text}')
+                              'Error %s: %s', _r.status_code, _r.text)
 
                 else:
-                    log.error(f'Error sending push. Error {_r.status_code}: {_r.text}')
+                    log.error('Error sending push. Error %s: %s', _r.status_code, _r.text)
 
         except Exception as e:
-            log.error(f'Error sending push. Error: {e}')
+            log.error('Error sending push. Error: %s', e)
 
 
 # reference:
@@ -586,13 +620,14 @@ def getenv(name, default_val=None, is_bool=False):
     if is_bool and isinstance(val, str):
         if val.upper() == "TRUE":
             return True
-        elif val.upper() == "FALSE":
+
+        if val.upper() == "FALSE":
             return False
-        else:
-            raise ValueError(f"Invalid boolean value (true or false) received for environment variable: {name}={val}")
-    else:
-        # return empty strings '' as None type
-        return val if val else None
+
+        raise ValueError(f"Invalid boolean value (true or false) received for environment variable: {name}={val}")
+
+    # return empty strings '' as None type
+    return val if val else None
 
 
 def format_vod_chapters(chapters):
@@ -611,7 +646,7 @@ def format_vod_chapters(chapters):
     
     """)
 
-    if type(chapters) == tuple:
+    if isinstance(chapters, tuple):
         formatted_chapters += chapter_base.format(
             start=chapters[1],
             end=chapters[2],
@@ -652,7 +687,7 @@ class Progress:
         """
         m, s = divmod(s, 60)
         h, m = divmod(m, 60)
-        return '{:0>2}:{:0>2}:{:0>2}'.format(h, m, s)
+        return f'{h:0>2}:{m:0>2}:{s:0>2}'
 
     def print_progress(self, cur, total, last_frame=False):
         """Prints and updates a nice progress bar.