tumblrbot 1.1.1__tar.gz → 1.1.3__tar.gz

tumblrbot-1.1.3/PKG-INFO

@@ -0,0 +1,108 @@
+Metadata-Version: 2.4
+Name: tumblrbot
+Version: 1.1.3
+Summary: An updated bot that posts to Tumblr, based on your very own blog!
+Requires-Python: >= 3.13
+Description-Content-Type: text/markdown
+Requires-Dist: more-itertools
+Requires-Dist: openai
+Requires-Dist: pydantic
+Requires-Dist: pydantic-settings
+Requires-Dist: requests
+Requires-Dist: requests-oauthlib
+Requires-Dist: rich
+Requires-Dist: tiktoken
+Requires-Dist: tomlkit
+Project-URL: Source, https://github.com/MaidThatPrograms/tumblrbot
+
+[OpenAI]: https://pypi.org/project/openai
+[Python]: https://python.org/download
+[Rich]: https://pypi.org/project/rich
+
+[gpt-4.1-nano-2025-04-14]: https://platform.openai.com/docs/models/gpt-4.1-nano
+[Moderation API]: https://platform.openai.com/docs/api-reference/moderations
+[New Post Format]: https://tumblr.com/docs/npf
+[OAuth 2.0]: https://www.tumblr.com/docs/en/api/v2#oauth2-authorization
+[pip]: https://pypi.org
+
+[Download]: tumblrbot/flow/download.py
+[Examples]: tumblrbot/flow/examples.py
+[Fine-Tune]: tumblrbot/flow/fine_tune.py
+[Generate]: tumblrbot/flow/generate.py
+[Settings]: tumblrbot/utils/settings.py
+[Main]: __main__.py
+[README.md]: README.md
+
+# tumblrbot
+[![PyPI - Version](https://img.shields.io/pypi/v/tumblrbot)](https://python.org/pypi/tumblrbot)
+
+Description of original project:
+> 4tv-tumblrbot was a collaborative project I embarked on with my close friend Dima, who goes by @smoqueen on Tumblr. The aim of this endeavor was straightforward yet silly: to develop a Tumblr bot powered by a machine-learning model. This bot would be specifically trained on the content from a particular Tumblr blog or a selected set of blogs, allowing it to mimic the style, tone, and thematic essence of the original posts.
+
+This fork is largely a rewrite of the source code with similarities in its structure and process:
+- Updates:
+   - Updated to [OAuth 2.0].
+   - Updated to the [New Post Format].
+   - Updated to the latest version of [OpenAI].
+   - Updated the [base model version][Settings] to [gpt-4.1-nano-2025-04-14].
+- Removed features:
+   - [Generation][Generate]:
+      - Removed clearing drafts behavior.
+   - [Training][Examples]:
+      - Removed exports that had HTML or reblogs.
+      - Removed special word-replacement behavior.
+      - Removed filtering by year.
+   - Removed setup and related files.
+- Changed/Added features:
+   - [Generation][Generate]:
+      - Added a link to the blog's draft page.
+      - Added error checking for uploading drafts.
+   - [Training][Examples]:
+      - Added the option to [Download] the latest posts from the [specified blogs][Settings].
+      - Added the option to remove posts flagged by the [Moderation API].
+      - Added the option to automatically [Fine-Tune] the examples on the [specified base model][Settings].
+      - Changed to now escape examples automatically.
+      - Set encoding for reading post data to `UTF-8` to fix decoding errors.
+      - Added newlines between paragraphs.
+      - Removed "ALT", submission, ask, and poll text from posts.
+      - Improved the estimated token counts and costs.
+   - Changed to [Rich] for output.
+      - Added progress bars.
+      - Added post previews.
+      - Added color, formatting, and more information to output.
+   - Created a [guided utility][Main] for every step of building your bot blog.
+   - Maid scripts wait for user input before the console closes.
+   - Added comand-line options to override [Settings] options.
+   - Added behavior to regenerate the default [config.toml][Settings] and [env.toml][Settings] if missing.
+   - Renamed several files.
+   - Renamed several [Settings] options.
+   - Changed the value of several [Settings] options.
+   - Added full type-checking coverage (fully importable from third-party scripts).
+
+To-Do:
+- Add documentation.
+- Finish updating [README.md].
+- Change the differences list to instead just be a list of features.
+- Allow adding arbitrary data to examples.
+
+
+**Please submit an issue or contact us for features you want to added/reimplemented.**
+
+## Installation
+1. Install the latest version of [Python]:
+   - Windows: `winget install python3`
+   - Linux (apt): `apt install python-pip`
+   - Linux (pacman): `pacman install python-pip`
+1. Install the [pip] package: `pip install tumblrbot`
+   - Alternatively, you can install from this repository: `pip install git+https://github.com/MaidThatPrograms/tumblrbot.git`
+   - On Linux, you will have to make a virtual environment.
+
+## Usage
+Run `tumblrbot` from anywhere. Run `tumblrbot --help` for command-line options.
+
+## Obtaining Tokens
+> WIP
+
+## Configuration
+> WIP
+

{tumblrbot-1.1.1 → tumblrbot-1.1.3}/README.md

@@ -65,8 +65,8 @@ This fork is largely a rewrite of the source code with similarities in its struc
 To-Do:
 - Add documentation.
 - Finish updating [README.md].
-- Look into places more-itertools can help.
 - Change the differences list to instead just be a list of features.
+- Allow adding arbitrary data to examples.
 
 
 **Please submit an issue or contact us for features you want to added/reimplemented.**

{tumblrbot-1.1.1 → tumblrbot-1.1.3}/pyproject.toml

@@ -1,7 +1,8 @@
 [project]
 name = "tumblrbot"
-version = "1.1.1"
+version = "1.1.3"
 description = "An updated bot that posts to Tumblr, based on your very own blog!"
+readme = "README.md"
 requires-python = ">= 3.13"
 dependencies = [
   "more-itertools",

{tumblrbot-1.1.1 → tumblrbot-1.1.3}/src/tumblrbot/flow/download.py

@@ -7,12 +7,12 @@ from tumblrbot.utils.models import Post
 
 
 class PostDownloader(UtilClass):
-    def paginate_posts(self, blog_identifier: str, offset: int, fp: TextIOBase, live: PreviewLive) -> None:
-        task_id = live.progress.add_task(f"Downloading posts from '{blog_identifier}'...", total=None, completed=offset)
+    def paginate_posts(self, blog_identifier: str, completed: int, after: int, fp: TextIOBase, live: PreviewLive) -> None:
+        task_id = live.progress.add_task(f"Downloading posts from '{blog_identifier}'...", total=None, completed=completed)
 
         while True:
-            response = self.tumblr.retrieve_published_posts(blog_identifier, offset).json()["response"]
-            live.progress.update(task_id, total=response["blog"]["posts"], completed=offset)
+            response = self.tumblr.retrieve_published_posts(blog_identifier, after=after).json()["response"]
+            live.progress.update(task_id, total=response["blog"]["posts"], completed=completed)
 
             if posts := response["posts"]:
                 for post in posts:
@@ -20,11 +20,12 @@ class PostDownloader(UtilClass):
                     fp.write("\n")
 
                     model = Post.model_validate(post)
+                    after = model.timestamp
                     live.custom_update(model)
 
-                offset += len(posts)
+                completed += len(posts)
             else:
-                break
+                return
 
     def get_data_path(self, blog_identifier: str) -> Path:
         return (self.config.data_directory / blog_identifier).with_suffix(".jsonl")
@@ -39,10 +40,19 @@ class PostDownloader(UtilClass):
             for blog_identifier in self.config.download_blog_identifiers:
                 data_path = self.get_data_path(blog_identifier)
 
+                completed = 0
+                after = 0
+                if data_path.exists():
+                    lines = data_path.read_text("utf_8").splitlines() if data_path.exists() else []
+                    completed = len(lines)
+                    if lines:
+                        after = Post.model_validate_json(lines[-1]).timestamp
+
                 with data_path.open("a", encoding="utf_8") as fp:
                     self.paginate_posts(
                         blog_identifier,
-                        len(data_path.read_text("utf_8").splitlines()) if data_path.exists() else 0,
+                        completed,
+                        after,
                         fp,
                         live,
                     )

{tumblrbot-1.1.1 → tumblrbot-1.1.3}/src/tumblrbot/utils/models.py

@@ -1,7 +1,7 @@
-from typing import Annotated, Any, Literal, override
+from typing import Any, Literal, override
 
 from openai import BaseModel
-from pydantic import ConfigDict, PlainSerializer
+from pydantic import ConfigDict
 from pydantic.json_schema import SkipJsonSchema
 from rich.panel import Panel
 
@@ -22,11 +22,14 @@ class Post(FullyValidatedModel):
         text: str = ""
         blocks: list[int] = []  # noqa: RUF012
 
-    tags: Annotated[list[str], PlainSerializer(",".join)] = []  # noqa: RUF012
+    timestamp: SkipJsonSchema[int] = 0
+    tags: list[str] = []  # noqa: RUF012
+    state: SkipJsonSchema[Literal["published", "queued", "draft", "private", "unapproved"]] = "published"
+
     content: SkipJsonSchema[list[Block]] = []  # noqa: RUF012
     layout: SkipJsonSchema[list[Block]] = []  # noqa: RUF012
     trail: SkipJsonSchema[list[Any]] = []  # noqa: RUF012
-    state: SkipJsonSchema[Literal["published", "queued", "draft", "private", "unapproved"]] = "published"
+
     is_submission: SkipJsonSchema[bool] = False
 
     def __rich__(self) -> Panel:

{tumblrbot-1.1.1 → tumblrbot-1.1.3}/src/tumblrbot/utils/tumblr.py

@@ -39,7 +39,6 @@ class TumblrClient(OAuth2Session):
 
             rich.print(f"Please go to {authorization_url} and authorize access.")
             authorization_response = Prompt.ask("Enter the full callback URL")
-            rich.print("\n")
 
             self.token_saver(
                 self.fetch_token(  # pyright: ignore[reportUnknownMemberType]
@@ -58,21 +57,14 @@ class TumblrClient(OAuth2Session):
         try:
             response.raise_for_status()
         except HTTPError as error:
-            json = response.json()
-            if error_description := json.get("error_description", None):
-                error.add_note(error_description)
-            elif errors := json.get("errors", None):
-                for suberror in errors:
-                    error.add_note(f"{suberror['title']} ({suberror['code']}): {suberror['detail']}")
-            else:
-                error.add_note(str(json))
+            error.add_note(response.text)
             raise
 
-    def retrieve_published_posts(self, blog_identifier: str, offset: int) -> Response:
+    def retrieve_published_posts(self, blog_identifier: str, after: int) -> Response:
         return self.get(
             f"https://api.tumblr.com/v2/blog/{blog_identifier}/posts",
             params={
-                "offset": offset,
+                "after": after,
                 "sort": "asc",
                 "npf": True,
             },

tumblrbot-1.1.1/PKG-INFO

@@ -1,15 +0,0 @@
-Metadata-Version: 2.4
-Name: tumblrbot
-Version: 1.1.1
-Summary: An updated bot that posts to Tumblr, based on your very own blog!
-Requires-Python: >= 3.13
-Requires-Dist: more-itertools
-Requires-Dist: openai
-Requires-Dist: pydantic
-Requires-Dist: pydantic-settings
-Requires-Dist: requests
-Requires-Dist: requests-oauthlib
-Requires-Dist: rich
-Requires-Dist: tiktoken
-Requires-Dist: tomlkit
-Project-URL: Source, https://github.com/MaidThatPrograms/tumblrbot