tumblrbot 1.9.7__tar.gz → 1.10.1__tar.gz

{tumblrbot-1.9.7 → tumblrbot-1.10.1}/PKG-INFO

@@ -1,9 +1,12 @@
 Metadata-Version: 2.4
 Name: tumblrbot
-Version: 1.9.7
+Version: 1.10.1
 Summary: An updated bot that posts to Tumblr, based on your very own blog!
-Requires-Python: >= 3.14
+Project-URL: Funding, https://ko-fi.com/maidscientistizutsumimarin
+Project-URL: Source, https://github.com/MaidScientistIzutsumiMarin/tumblrbot
+Requires-Python: >=3.14
 Description-Content-Type: text/markdown
+Requires-Dist: CurrencyConverter
 Requires-Dist: openai
 Requires-Dist: pydantic
 Requires-Dist: requests
@@ -12,15 +15,9 @@ Requires-Dist: rich
 Requires-Dist: tenacity
 Requires-Dist: tiktoken
 Requires-Dist: tomlkit
-Requires-Dist: pyinstaller ; extra == "dev"
-Project-URL: Funding, https://ko-fi.com/maidscientistizutsumimarin
-Project-URL: Source, https://github.com/MaidScientistIzutsumiMarin/tumblrbot
-Provides-Extra: dev
 
 # tumblrbot
 
-[tumblrbot.exe]: https://github.com/MaidScientistIzutsumiMarin/tumblrbot/releases/latest/download/tumblrbot.exe
-
 [OAuth]: https://oauth.net/1
 [Python]: https://python.org/download
 
@@ -30,11 +27,11 @@ Provides-Extra: dev
 [pip]: https://pypi.org
 [Rich]: https://pypi.org/project/rich
 
-[OpenAI]: https://pypi.org/project/openai
+[OpenAI]: https://openai.com
 [OpenAI Pricing]: https://platform.openai.com/docs/pricing#fine-tuning
 [OpenAI Tokens]: https://platform.openai.com/settings/organization/api-keys
 [OpenAI Moderation API]: https://platform.openai.com/docs/guides/moderation
-[Flags]: https://platform.openai.com/docs/guides/moderation/over#content-classifications
+[Flags]: https://platform.openai.com/docs/guides/moderation#content-classifications
 [Fine-Tuning Portal]: https://platform.openai.com/finetune
 
 [Tumblr]: https://tumblr.com
@@ -53,6 +50,19 @@ Provides-Extra: dev
 [Fine-Tuning]: #manual-fine-tuning
 [![PyPI - Version](https://img.shields.io/pypi/v/tumblrbot)](https://python.org/pypi/tumblrbot)
 
+## Installation & Usage
+
+1. Install the latest version of [Python]:
+   - Windows: `winget install python3`
+   - Linux (apt): `apt install python-pip`
+   - Linux (pacman): `pacman install python-pip`
+1. Install the [pip] package: `pip install tumblrbot`
+   - Alternatively, you can install from this repository: `pip install git+https://github.com/MaidScientistIzutsumiMarin/tumblrbot.git`
+   - On Linux, you will have to make a virtual environment or use the flag to install packages system-wide.
+1. Run `tumblrbot` from anywhere. Run `tumblrbot --help` for command-line options. Every command-line option corresponds to a value from the [config][configurable].
+
+---
+
 Description of original project:
 > 4tv-tumblrbot was a collaborative project I embarked on with my close friend Dima, who goes by @smoqueen on Tumblr. The aim of this endeavor was straightforward yet silly: to develop a Tumblr bot powered by a machine-learning model. This bot would be specifically trained on the content from a particular Tumblr blog or a selected set of blogs, allowing it to mimic the style, tone, and thematic essence of the original posts.
 
@@ -93,39 +103,17 @@ Features:
   - You can use regular expressions to filter out training data in the [config][configurable]. This is more of a brute-force solution, but it can work if the other solutions do not.
   - You can try limiting your dataset by specifying fewer blogs to download from or limiting the number of posts taken from each one in the [config][configurable].
   - If all else fails, you can manually remove data from the examples file until it passes. It is unfortunately not a definitive resource, but it can help to read about what the [OpenAI moderation API flags][Flags].
-- Sometimes, you will get an error about the training file not being found when starting fine-tuning. We do not currently have a fix or workaround for this. You should instead use the online portal for fine-tuning if this continues to happen. Read more in [fine-tuning]
-  - *We are unsure if this is still happening.*
-- Post counts are incorrect when downloading posts. Our tests suggest this is a [Tumblr] API problem that is giving inaccurate numbers, so treat them as estimates.
+- Post counts can be incorrect when downloading posts. Our tests suggest this is a [Tumblr] API problem that is giving inaccurate numbers, so treat them as estimates.
 
-**Please submit an issue or contact us for features you want added/reimplemented.**
-
-## Installation & Usage
-
-### Downloadable Binary
+**To-Do:**
 
-| Pros | Cons |
-| --- | --- |
-| Easier to install | Harder to update |
-| No risk of dependencies breaking | Dependencies may be older |
+- Add selection menus to make it easier to pick a particular operation.
+- Add retrying to the fine-tuning process (if necessary).
+- Add an option to filter newest posts by a specific date.
 
-1. Download the latest release's [tumblrbot.exe].
-1. Launch `tumblrbot.exe` in the install location.
-
-### PyPi
-
-| Pros | Cons |
-| --- | --- |
-| Easier to update | Harder to install |
-| Dependencies may be newer | Dependencies may break |
+**Please submit an issue or contact us for features you want added/reimplemented.**
 
-1. Install the latest version of [Python]:
-   - Windows: `winget install python3`
-   - Linux (apt): `apt install python-pip`
-   - Linux (pacman): `pacman install python-pip`
-1. Install the [pip] package: `pip install tumblrbot`
-   - Alternatively, you can install from this repository: `pip install git+https://github.com/MaidScientistIzutsumiMarin/tumblrbot.git`
-   - On Linux, you will have to make a virtual environment or use the flag to install packages system-wide.
-1. Run `tumblrbot` from anywhere. Run `tumblrbot --help` for command-line options. Every command-line option corresponds to a value from the [config][configurable].
+---
 
 ## Obtaining Tokens
 
@@ -207,4 +195,3 @@ You can manually upload the examples file to [OpenAI] and start the fine-tuning
 1. Press `Create`.
 1. (Optional) Copy the value next to `Job ID` and paste it into the [config][configurable] under `job_id`. You can then run the program and monitor its progress as usual.
 1. If you do not do the above, you will have to copy the value next to `Output model` once the job is complete and paste it into the [config][configurable] under `fine_tuned_model`.
-

tumblrbot-1.10.1/README.md

@@ -0,0 +1,179 @@
+# tumblrbot
+
+[OAuth]: https://oauth.net/1
+[Python]: https://python.org/download
+
+[JSON Lines]: https://jsonlines.org
+[JSON Lines Validator]: https://jsonlines.org/validator
+
+[pip]: https://pypi.org
+[Rich]: https://pypi.org/project/rich
+
+[OpenAI]: https://openai.com
+[OpenAI Pricing]: https://platform.openai.com/docs/pricing#fine-tuning
+[OpenAI Tokens]: https://platform.openai.com/settings/organization/api-keys
+[OpenAI Moderation API]: https://platform.openai.com/docs/guides/moderation
+[Flags]: https://platform.openai.com/docs/guides/moderation#content-classifications
+[Fine-Tuning Portal]: https://platform.openai.com/finetune
+
+[Tumblr]: https://tumblr.com
+[Tumblr Tokens]: https://tumblr.com/oauth/apps
+[Tumblr API Documentation on Blog Identifiers]: https://tumblr.com/docs/en/api/v2#blog-identifiers
+
+[Format String]: https://docs.python.org/3/library/string.html#format-string-syntax
+
+[Download]: src/tumblrbot/flow/download.py
+[Examples]: src/tumblrbot/flow/examples.py
+[Fine-Tune]: src/tumblrbot/flow/fine_tune.py
+[Generate]: src/tumblrbot/flow/generate.py
+[Main]: src/tumblrbot/__main__.py
+
+[Configurable]: #configuration
+[Fine-Tuning]: #manual-fine-tuning
+[![PyPI - Version](https://img.shields.io/pypi/v/tumblrbot)](https://python.org/pypi/tumblrbot)
+
+## Installation & Usage
+
+1. Install the latest version of [Python]:
+   - Windows: `winget install python3`
+   - Linux (apt): `apt install python-pip`
+   - Linux (pacman): `pacman install python-pip`
+1. Install the [pip] package: `pip install tumblrbot`
+   - Alternatively, you can install from this repository: `pip install git+https://github.com/MaidScientistIzutsumiMarin/tumblrbot.git`
+   - On Linux, you will have to make a virtual environment or use the flag to install packages system-wide.
+1. Run `tumblrbot` from anywhere. Run `tumblrbot --help` for command-line options. Every command-line option corresponds to a value from the [config][configurable].
+
+---
+
+Description of original project:
+> 4tv-tumblrbot was a collaborative project I embarked on with my close friend Dima, who goes by @smoqueen on Tumblr. The aim of this endeavor was straightforward yet silly: to develop a Tumblr bot powered by a machine-learning model. This bot would be specifically trained on the content from a particular Tumblr blog or a selected set of blogs, allowing it to mimic the style, tone, and thematic essence of the original posts.
+
+This fork is largely a rewrite of the source code with similarities in its structure and process.
+
+Features:
+
+- An [interactive console][Main] for all steps of generating posts for the blog:
+   1. Asks for [OpenAI] and [Tumblr] tokens.
+   1. Retrieves [Tumblr] [OAuth] tokens.
+   1. [Downloads posts][Download] from specified blogs ([configurable]).
+      - Skips redownloading already downloaded posts.
+      - Shows progress and previews the current post.
+   1. [Creates examples][Examples] to fine-tune the model from the downloaded posts.
+      - Filters out posts that contain more than just text data.
+      - Filters out posts that contain regular expressions ([configurable]).
+      - Only uses the most recent posts from each blog ([configurable]).
+      - Adds custom user messages and assistant responses to the dataset ([configurable]).
+   1. Filters out any posts flagged by the [OpenAI Moderation API].
+   1. [Uploads examples][Fine-Tune] to [OpenAI] and begins the fine-tuning process.
+      - Provides cost estimates if the currently saved examples are used to fine-tune a base model ([configurable]).
+      - Resumes monitoring the same fine-tuning process when restarted.
+      - Deletes the uploaded examples file if fine-tuning does not succeed (optional).
+      - Stores the output model automatically when fine-tuning is completed.
+   1. [Generates and uploads posts][Generate] to a blog using the fine-tuned model ([configurable]).
+      - Creates tags by extracting keywords using the base model ([configurable]).
+      - Uploads posts as drafts.
+      - Reblogs posts from allowed blogs ([configurable]).
+      - Shows progress and previews the current post.
+- Colorful output, progress bars, and post previews using [rich].
+- Automatically keeps the [config][configurable] file up-to-date and recreates it if missing (without overriding user settings).
+
+**Known Issues:**
+
+- Fine-tuning can fail after the validation phase due to the examples file not passing [OpenAI] moderation checks. There are a few workarounds for this that can be tried in combination:
+  - You can retry with the same examples file. This has, on rare occasions, worked.
+  - You can submit the examples file to the [OpenAI] moderation API with this program's guided prompts. This has worked consistently for our dataset, but others have reported it not being thorough enough.
+  - You can use regular expressions to filter out training data in the [config][configurable]. This is more of a brute-force solution, but it can work if the other solutions do not.
+  - You can try limiting your dataset by specifying fewer blogs to download from or limiting the number of posts taken from each one in the [config][configurable].
+  - If all else fails, you can manually remove data from the examples file until it passes. It is unfortunately not a definitive resource, but it can help to read about what the [OpenAI moderation API flags][Flags].
+- Post counts can be incorrect when downloading posts. Our tests suggest this is a [Tumblr] API problem that is giving inaccurate numbers, so treat them as estimates.
+
+**To-Do:**
+
+- Add selection menus to make it easier to pick a particular operation.
+- Add retrying to the fine-tuning process (if necessary).
+- Add an option to filter newest posts by a specific date.
+
+**Please submit an issue or contact us for features you want added/reimplemented.**
+
+---
+
+## Obtaining Tokens
+
+### OpenAI
+
+API token can be created here: [OpenAI Tokens].
+
+   1. Leave everything at the defaults and set `Project` to `Default Project`.
+   1. Press `Create secret key`.
+   1. Press `Copy` to copy the API token to your clipboard.
+
+### Tumblr
+
+API tokens can be created here: [Tumblr Tokens].
+
+   1. Press `+ Register Application`.
+   1. Enter anything for `Application Name` and `Application Description`.
+   1. Enter any URL for `Application Website` and `Default callback URL`, like `https://example.com`.
+   1. Enter any email address for `Administrative contact email`. It probably doesn't need to be one you have access to.
+   1. Press the checkbox next to `I'm not a robot` and complete the CAPTCHA.
+   1. Press `Register`.
+   1. You now have access to your `consumer key` next to `Oauth Consumer Key`.
+   1. Press `Show secret key` to see your `Consumer Secret`.
+
+When running this program, you will be prompted to enter all of these tokens. If something goes wrong while entering the tokens, you can always reset them by running the program again and answering `y` to the relevant prompt.
+
+After inputting the [Tumblr] tokens, you will be given a URL that you need to open in your browser. Press `Allow`, then copy and paste the URL of the page you are redirected to into the console.
+
+## Configuration
+
+All config options can be found in `config.toml` after running the program once. This will be kept up-to-date if there are changes to the config's format in a future update. This also means it may be worthwhile to double-check the config file after an update. Any changes to the config should be in the changelog for a given version.
+
+All file options can include directories that will be created when the program is run.
+
+All config options that involve *blog identifiers* expect any version of a blog URL, which is explained in more detail in the [Tumblr API documentation on blog identifiers].
+
+A valid post:
+
+- Contains any content.
+- Only has text.
+- Is not an ask.
+- Is not a reblog.
+
+Specific Options:
+
+- `custom_prompts_file` This file should follow the following file format:
+
+   ```json
+   {"user message 1": "assistant response 1"}
+   {"user message 1": "assistant response 1"}
+   {"user message 2": "assistant response 2", "user message 3": "assistant response 3"}
+   ```
+
+   To be specific, it should follow the [JSON Lines] file format with one collection of name/value pairs (a dictionary) per line. You can validate your file using the [JSON Lines Validator].
+
+- **`post_limit`** - At most, this many valid posts will be included in the training data. This effectively is a filter to select the `N` most recent valid posts from each blog. `0` will use every available valid post.
+- **`moderation_batch_size`** - This controls the batch size when submitting posts to the OpenAI moderation. There is no limit, but higher numbers will cause you to be rate-limited more, which can overall be slower. Low numbers reduce rate-limiting, but can sometimes take longer due to needing more requests. The best value will depend on your computer, internet connection, and any number of factors on OpenAI's side. The default value is just what worked best for our computer.
+- **`filtered_words`** - During training data generation, any posts with the specified words will be removed. Word boundaries are not checked by default, so “the” will also filter out posts with “them” or “thematic”. This setting supports regular expressions, so you can explicitly look for word boundaries by surrounding an entry with “\\\b”, i.e., “\\\bthe\\\b”. Regular expressions have to be escaped like so due to how JSON data is read in. If you are familiar with regular expressions, it could be useful for you to know that every entry is joined with a “|” which is then used to search the post content for any matches.
+- **`developer_message`** - This message is used in for fine-tuning the AI as well as generating prompts. If you change this, you will need to run the fine-tuning again with the new value before generating posts.
+- **`user_message`** - This setting is used and works in the same way as `developer_message`.
+- **`expected_epochs`** - The default value here is the default number of epochs for `base_model`. You may have to change this value if you change `base_model`. After running fine-tuning once, you will see the number of epochs used in the [fine-tuning portal] under *Hyperparameters*. This value will also be updated automatically if you run fine-tuning through this program.
+- **`token_price`** - The default value here is the default token price for `base_model`. You can find the up-to-date value in [OpenAI Pricing], in the *Training* column.
+- **`job_id`** - If there is any value here, this program will resume monitoring the corresponding job, instead of starting a new one. This gets set when starting the fine-tuning and is cleared when it is completed. You can read more in [fine-tuning].
+- **`base_model`** - This value is used to choose the tokenizer for estimating fine-tuning costs. It is also the base model that will be fine-tuned and the model that is used to generate tags. You can find a list of options in the [fine-tuning portal] by pressing `+ Create` and opening the drop-down list for `Base Model`. Be sure to update `token_price` if you change this value.
+- **`fine_tuned_model`** - Set automatically after monitoring fine-tuning if the job has succeeded. You can read more in [fine-tuning].
+- **`tags_chance`** - This should be between 0 and 1. Setting it to 0 corresponds to a 0% chance (never) to add tags to a post. 1 corresponds to a 100% chance (always) to add tags to a post. Adding tags incurs a very small token cost.
+- **`reblog_blog_identifiers`** - Whenever a reblog is attempted, a random blog from this list will be chosen to be reblogged from.
+- **`reblog_chance`** - This setting works the same way as `tags_chance`.
+- **`reblog_user_message`** - This setting is a [format string]. The only argument it is formatted with is the content of the post being reblogged. In simple terms, the `{}` will be replaced with said content. Alternatively, you can leave out the `{}` so that the reblogged post is appended to the end.
+  - *Note: The bot is only given the latest message in a reblog chain due to the required complexity and added costs of including the entire chain.*
+
+## Manual Fine-Tuning
+
+You can manually upload the examples file to [OpenAI] and start the fine-tuning here: [fine-tuning portal].
+
+1. Press `+ Create`.
+1. Select the desired `Base Model` from the dropdown. This should ideally match the model set in the [config][configurable].
+1. Upload the generated examples file to the section under `Training data`. You can find the path for this in the [config][configurable].
+1. Press `Create`.
+1. (Optional) Copy the value next to `Job ID` and paste it into the [config][configurable] under `job_id`. You can then run the program and monitor its progress as usual.
+1. If you do not do the above, you will have to copy the value next to `Output model` once the job is complete and paste it into the [config][configurable] under `fine_tuned_model`.

{tumblrbot-1.9.7 → tumblrbot-1.10.1}/pyproject.toml

@@ -1,30 +1,27 @@
-[project]
-name = "tumblrbot"
-version = "1.9.7"
-description = "An updated bot that posts to Tumblr, based on your very own blog!"
-readme = "README.md"
-requires-python = ">= 3.14"
-dependencies = [
-  "openai",
-  "pydantic",
-  "requests",
-  "requests-oauthlib",
-  "rich",
-  "tenacity",
-  "tiktoken",
-  "tomlkit"
-]
-
-[project.urls]
-Funding = "https://ko-fi.com/maidscientistizutsumimarin"
-Source = "https://github.com/MaidScientistIzutsumiMarin/tumblrbot"
-
-[project.scripts]
-tumblrbot = "tumblrbot.__main__:main"
-
-[project.optional-dependencies]
-dev = ["pyinstaller"]
-
-[build-system]
-requires = ["flit_core"]
-build-backend = "flit_core.buildapi"
+[project]
+name = "tumblrbot"
+version = "1.10.1"
+description = "An updated bot that posts to Tumblr, based on your very own blog!"
+readme = "README.md"
+requires-python = ">= 3.14"
+dependencies = [
+  "CurrencyConverter",
+  "openai",
+  "pydantic",
+  "requests",
+  "requests-oauthlib",
+  "rich",
+  "tenacity",
+  "tiktoken",
+  "tomlkit"
+]
+
+[project.urls]
+Funding = "https://ko-fi.com/maidscientistizutsumimarin"
+Source = "https://github.com/MaidScientistIzutsumiMarin/tumblrbot"
+
+[project.scripts]
+tumblrbot = "tumblrbot.__main__:main"
+
+[tool.uv]
+package = true

tumblrbot-1.10.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

{tumblrbot-1.9.7 → tumblrbot-1.10.1}/src/tumblrbot/__main__.py

@@ -1,43 +1,47 @@
-from sys import exit as sys_exit
-
-from openai import OpenAI
-from rich.prompt import Confirm
-from rich.traceback import install
-
-from tumblrbot.flow.download import PostDownloader
-from tumblrbot.flow.examples import ExamplesWriter
-from tumblrbot.flow.fine_tune import FineTuner
-from tumblrbot.flow.generate import DraftGenerator
-from tumblrbot.utils.common import FlowClass
-from tumblrbot.utils.models import Tokens
-from tumblrbot.utils.tumblr import TumblrSession
-
-
-def main() -> None:
-    install()
-
-    tokens = Tokens.load()
-    with OpenAI(api_key=tokens.openai_api_key) as openai, TumblrSession(tokens) as tumblr:
-        if Confirm.ask("Download latest posts?", default=False):
-            PostDownloader(openai, tumblr).main()
-
-        examples_writer = ExamplesWriter(openai, tumblr)
-        if Confirm.ask("Create training data?", default=False):
-            examples_writer.main()
-
-        if Confirm.ask("Remove training data flagged by the OpenAI moderation? [bold]This can sometimes resolve errors with fine-tuning validation, but is slow.", default=False):
-            examples_writer.filter_examples()
-
-        fine_tuner = FineTuner(openai, tumblr)
-        fine_tuner.print_estimates()
-
-        message = "Resume monitoring the previous fine-tuning process?" if FlowClass.config.job_id else "Upload data to OpenAI for fine-tuning?"
-        if Confirm.ask(f"{message} [bold]You must do this to set the model to generate drafts from. Alternatively, manually enter a model into the config", default=False):
-            fine_tuner.main()
-
-        if Confirm.ask("Generate drafts?", default=False):
-            DraftGenerator(openai, tumblr).main()
-
-
-if __name__ == "__main__":
-    sys_exit(main())
+from locale import LC_ALL, setlocale
+from sys import exit as sys_exit
+from sys import maxsize
+
+from openai import OpenAI
+from rich.prompt import Confirm
+from rich.traceback import install
+
+from tumblrbot.flow.download import PostDownloader
+from tumblrbot.flow.examples import ExamplesWriter
+from tumblrbot.flow.fine_tune import FineTuner
+from tumblrbot.flow.generate import DraftGenerator
+from tumblrbot.utils.common import FlowClass
+from tumblrbot.utils.models import Tokens
+from tumblrbot.utils.tumblr import TumblrSession
+
+
+def main() -> None:
+    setlocale(LC_ALL, "")
+
+    install()
+
+    tokens = Tokens.load()
+    with OpenAI(api_key=tokens.openai_api_key, max_retries=maxsize) as openai, TumblrSession(tokens) as tumblr:
+        if Confirm.ask("Download latest posts?", default=False):
+            PostDownloader(openai, tumblr).main()
+
+        examples_writer = ExamplesWriter(openai, tumblr)
+        if Confirm.ask("Create training data?", default=False):
+            examples_writer.main()
+
+        if Confirm.ask("Remove training data flagged by the OpenAI moderation? [bold]This can sometimes resolve errors with fine-tuning validation, but is slow.", default=False):
+            examples_writer.filter_examples()
+
+        fine_tuner = FineTuner(openai, tumblr)
+        fine_tuner.print_estimates()
+
+        message = "Resume monitoring the previous fine-tuning process?" if FlowClass.config.job_id else "Upload data to OpenAI for fine-tuning?"
+        if Confirm.ask(f"{message} [bold]You must do this to set the model to generate drafts from. Alternatively, manually enter a model into the config", default=False):
+            fine_tuner.main()
+
+        if Confirm.ask("Generate drafts?", default=False):
+            DraftGenerator(openai, tumblr).main()
+
+
+if __name__ == "__main__":
+    sys_exit(main())

{tumblrbot-1.9.7 → tumblrbot-1.10.1}/src/tumblrbot/flow/download.py

@@ -1,55 +1,55 @@
-from json import dump
-from typing import TYPE_CHECKING, override
-
-from tumblrbot.utils.common import FlowClass, PreviewLive
-from tumblrbot.utils.models import Post
-
-if TYPE_CHECKING:
-    from io import TextIOBase
-
-
-class PostDownloader(FlowClass):
-    @override
-    def main(self) -> None:
-        self.config.data_directory.mkdir(parents=True, exist_ok=True)
-
-        with PreviewLive() as live:
-            for blog_identifier in self.config.download_blog_identifiers:
-                data_path = self.get_data_path(blog_identifier)
-
-                completed = 0
-                after = 0
-                if data_path.exists():
-                    lines = data_path.read_bytes().splitlines() if data_path.exists() else []
-                    completed = len(lines)
-                    if lines:
-                        after = Post.model_validate_json(lines[-1]).timestamp
-
-                with data_path.open("a", encoding="utf_8") as fp:
-                    self.paginate_posts(
-                        blog_identifier,
-                        completed,
-                        after,
-                        fp,
-                        live,
-                    )
-
-    def paginate_posts(self, blog_identifier: str, completed: int, after: int, fp: TextIOBase, live: PreviewLive) -> None:
-        task_id = live.progress.add_task(f"Downloading posts from '{blog_identifier}'...", total=None, completed=completed)
-
-        while True:
-            response = self.tumblr.retrieve_published_posts(blog_identifier, after=after)
-            live.progress.update(task_id, total=response.response.blog.posts, completed=completed)
-
-            if not response.response.posts:
-                return
-
-            for post in response.response.posts:
-                dump(post, fp)
-                fp.write("\n")
-
-                model = Post.model_validate(post)
-                after = model.timestamp
-                live.custom_update(model)
-
-            completed += len(response.response.posts)
+from json import dump
+from typing import TYPE_CHECKING, override
+
+from tumblrbot.utils.common import FlowClass, PreviewLive
+from tumblrbot.utils.models import Post
+
+if TYPE_CHECKING:
+    from io import TextIOBase
+
+
+class PostDownloader(FlowClass):
+    @override
+    def main(self) -> None:
+        self.config.data_directory.mkdir(parents=True, exist_ok=True)
+
+        with PreviewLive() as live:
+            for blog_identifier in self.config.download_blog_identifiers:
+                data_path = self.get_data_path(blog_identifier)
+
+                completed = 0
+                after = 0
+                if data_path.exists():
+                    lines = data_path.read_bytes().splitlines() if data_path.exists() else []
+                    completed = len(lines)
+                    if lines:
+                        after = Post.model_validate_json(lines[-1]).timestamp
+
+                with data_path.open("a", encoding="utf_8") as fp:
+                    self.paginate_posts(
+                        blog_identifier,
+                        completed,
+                        after,
+                        fp,
+                        live,
+                    )
+
+    def paginate_posts(self, blog_identifier: str, completed: int, after: int, fp: TextIOBase, live: PreviewLive) -> None:
+        task_id = live.progress.add_task(f"Downloading posts from '{blog_identifier}'...", total=None, completed=completed)
+
+        while True:
+            response = self.tumblr.retrieve_published_posts(blog_identifier, after=after)
+            live.progress.update(task_id, total=response.response.blog.posts, completed=completed)
+
+            if not response.response.posts:
+                return
+
+            for post in response.response.posts:
+                dump(post, fp)
+                fp.write("\n")
+
+                model = Post.model_validate(post)
+                after = model.timestamp
+                live.custom_update(model)
+
+            completed += len(response.response.posts)