fap-dl 0.1.0__tar.gz

fap_dl-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Reveren
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

fap_dl-0.1.0/PKG-INFO

@@ -0,0 +1,183 @@
+Metadata-Version: 2.4
+Name: fap-dl
+Version: 0.1.0
+Summary: A command-line downloader for Fapello profiles.
+Author: Michael Horton
+License-Expression: MIT
+Project-URL: Repository, https://github.com/Reveren/fap-dl
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: playwright>=1.55
+Requires-Dist: requests>=2.32
+Dynamic: license-file
+
+# fap-dl
+
+A command-line downloader for images and videos from Fapello profiles.
+
+`fap-dl` uses Chromium to discover the contents of a profile, then closes the
+browser and downloads the media directly.
+
+## Features
+
+- Downloads images and videos
+- Automatically discovers lazy-loaded posts
+- Detects video posts
+- Streams downloads directly to disk
+- Skips media that has already been downloaded
+- Uses `.part` files for incomplete downloads
+- Separates images and videos by default
+- Supports combined output
+- Supports image-only and video-only downloads
+- Automatically installs the required Playwright Chromium browser when needed
+
+## Requirements
+
+- Python 3.10 or newer
+- Internet connection
+
+### Platform support
+
+`fap-dl` is currently developed and tested on macOS.
+
+The underlying Python, Playwright, Chromium, and Requests components are
+cross-platform, so the program is expected to work on Linux and Windows as
+well. Linux and Windows have not yet been tested with this project.
+
+## Usage
+
+Download a profile:
+
+```bash
+fap-dl PROFILE
+```
+
+For example:
+
+```bash
+fap-dl diamondnips-1
+```
+
+You can also provide the full profile URL:
+
+```bash
+fap-dl https://fapello.com/diamondnips-1/
+```
+
+By default, files are stored in:
+
+```text
+~/Downloads/fap-dl/PROFILE/
+├── images/
+└── videos/
+```
+
+### Combined folder
+
+```bash
+fap-dl PROFILE --combined
+```
+
+### Images only
+
+```bash
+fap-dl PROFILE --images-only
+```
+
+### Videos only
+
+```bash
+fap-dl PROFILE --videos-only
+```
+
+### Custom output directory
+
+```bash
+fap-dl PROFILE --output ~/Desktop/media
+```
+
+This creates:
+
+```text
+~/Desktop/media/PROFILE/
+```
+
+## Why does Chromium open?
+
+Fapello is protected by Cloudflare and may challenge or block ordinary
+automated HTTP requests. During development, headless Chromium was also unable
+to access profile content reliably even when using a previously established
+browser session.
+
+For that reason, `fap-dl` briefly launches a normal, visible Chromium window.
+This allows the profile to load in a regular browser environment and also
+allows you to complete a Cloudflare verification challenge if one appears.
+
+Chromium is used only for the discovery stage:
+
+1. Chromium opens the requested profile.
+2. `fap-dl` scrolls through the profile to discover all available posts.
+3. Images and video posts are identified.
+4. Browser session information is transferred to the downloader.
+5. Chromium closes.
+6. Images and videos are streamed directly to disk without keeping the
+   browser open.
+
+Most runs should therefore require no interaction with the Chromium window.
+If Fapello presents a verification challenge, complete it in Chromium and
+follow the prompt in the terminal.
+
+Browser data is retained in:
+
+```text
+~/.fap-dl/browser/
+```
+
+This allows browser session information to persist between runs and can reduce
+the need for repeated verification.
+
+## Downloads and existing files
+
+Existing non-empty media files are skipped. This makes it possible to run
+`fap-dl` against the same profile later and download newly discovered media
+without downloading the entire collection again.
+
+Downloads are streamed directly to disk rather than being held entirely in
+memory.
+
+While a file is downloading, it uses the `.part` extension:
+
+```text
+example.mp4.part
+```
+
+After the download completes successfully, it becomes:
+
+```text
+example.mp4
+```
+
+This prevents an interrupted or incomplete download from being mistaken for a
+completed media file.
+
+## Video fallback
+
+Video URLs can normally be determined directly from the profile gallery.
+
+If a predicted video URL fails, `fap-dl` can reopen Chromium and inspect the
+individual post to locate its actual MP4 source. Multiple failed videos are
+handled in the same fallback browser session rather than opening a separate
+browser for each one.
+
+## Disclaimer
+
+This project is not affiliated with or endorsed by Fapello.
+
+Users are responsible for ensuring that their use of this software complies
+with applicable laws, copyright restrictions, website terms, and the rights
+of content creators.
+
+## License
+
+MIT

fap_dl-0.1.0/README.md

@@ -0,0 +1,169 @@
+# fap-dl
+
+A command-line downloader for images and videos from Fapello profiles.
+
+`fap-dl` uses Chromium to discover the contents of a profile, then closes the
+browser and downloads the media directly.
+
+## Features
+
+- Downloads images and videos
+- Automatically discovers lazy-loaded posts
+- Detects video posts
+- Streams downloads directly to disk
+- Skips media that has already been downloaded
+- Uses `.part` files for incomplete downloads
+- Separates images and videos by default
+- Supports combined output
+- Supports image-only and video-only downloads
+- Automatically installs the required Playwright Chromium browser when needed
+
+## Requirements
+
+- Python 3.10 or newer
+- Internet connection
+
+### Platform support
+
+`fap-dl` is currently developed and tested on macOS.
+
+The underlying Python, Playwright, Chromium, and Requests components are
+cross-platform, so the program is expected to work on Linux and Windows as
+well. Linux and Windows have not yet been tested with this project.
+
+## Usage
+
+Download a profile:
+
+```bash
+fap-dl PROFILE
+```
+
+For example:
+
+```bash
+fap-dl diamondnips-1
+```
+
+You can also provide the full profile URL:
+
+```bash
+fap-dl https://fapello.com/diamondnips-1/
+```
+
+By default, files are stored in:
+
+```text
+~/Downloads/fap-dl/PROFILE/
+├── images/
+└── videos/
+```
+
+### Combined folder
+
+```bash
+fap-dl PROFILE --combined
+```
+
+### Images only
+
+```bash
+fap-dl PROFILE --images-only
+```
+
+### Videos only
+
+```bash
+fap-dl PROFILE --videos-only
+```
+
+### Custom output directory
+
+```bash
+fap-dl PROFILE --output ~/Desktop/media
+```
+
+This creates:
+
+```text
+~/Desktop/media/PROFILE/
+```
+
+## Why does Chromium open?
+
+Fapello is protected by Cloudflare and may challenge or block ordinary
+automated HTTP requests. During development, headless Chromium was also unable
+to access profile content reliably even when using a previously established
+browser session.
+
+For that reason, `fap-dl` briefly launches a normal, visible Chromium window.
+This allows the profile to load in a regular browser environment and also
+allows you to complete a Cloudflare verification challenge if one appears.
+
+Chromium is used only for the discovery stage:
+
+1. Chromium opens the requested profile.
+2. `fap-dl` scrolls through the profile to discover all available posts.
+3. Images and video posts are identified.
+4. Browser session information is transferred to the downloader.
+5. Chromium closes.
+6. Images and videos are streamed directly to disk without keeping the
+   browser open.
+
+Most runs should therefore require no interaction with the Chromium window.
+If Fapello presents a verification challenge, complete it in Chromium and
+follow the prompt in the terminal.
+
+Browser data is retained in:
+
+```text
+~/.fap-dl/browser/
+```
+
+This allows browser session information to persist between runs and can reduce
+the need for repeated verification.
+
+## Downloads and existing files
+
+Existing non-empty media files are skipped. This makes it possible to run
+`fap-dl` against the same profile later and download newly discovered media
+without downloading the entire collection again.
+
+Downloads are streamed directly to disk rather than being held entirely in
+memory.
+
+While a file is downloading, it uses the `.part` extension:
+
+```text
+example.mp4.part
+```
+
+After the download completes successfully, it becomes:
+
+```text
+example.mp4
+```
+
+This prevents an interrupted or incomplete download from being mistaken for a
+completed media file.
+
+## Video fallback
+
+Video URLs can normally be determined directly from the profile gallery.
+
+If a predicted video URL fails, `fap-dl` can reopen Chromium and inspect the
+individual post to locate its actual MP4 source. Multiple failed videos are
+handled in the same fallback browser session rather than opening a separate
+browser for each one.
+
+## Disclaimer
+
+This project is not affiliated with or endorsed by Fapello.
+
+Users are responsible for ensuring that their use of this software complies
+with applicable laws, copyright restrictions, website terms, and the rights
+of content creators.
+
+## License
+
+MIT

fap_dl-0.1.0/pyproject.toml

@@ -0,0 +1,27 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "fap-dl"
+version = "0.1.0"
+description = "A command-line downloader for Fapello profiles."
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+authors = [
+	{ name = "Michael Horton" }
+]
+dependencies = [
+	"playwright>=1.55",
+	"requests>=2.32"
+]
+
+[project.scripts]
+fap-dl = "fap_dl.cli:main"
+
+[project.urls]
+Repository = "https://github.com/Reveren/fap-dl"
+
+[tool.setuptools.packages.find]
+where = ["src"]

fap_dl-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

fap_dl-0.1.0/src/fap_dl/browser.py

@@ -0,0 +1,194 @@
+from pathlib import Path
+import subprocess
+import sys
+
+from playwright.sync_api import (
+	Error as PlaywrightError,
+	TimeoutError as PlaywrightTimeoutError,
+)
+
+from .gallery import get_video_fallback_url
+
+
+def get_browser_data_dir():
+	return Path.home() / ".fap-dl" / "browser"
+
+
+def install_chromium():
+	print()
+	print("Chromium is required for browser discovery.")
+	print("Installing Chromium for fap-dl...")
+	print()
+
+	try:
+		subprocess.run(
+			[
+				sys.executable,
+				"-m",
+				"playwright",
+				"install",
+				"chromium",
+			],
+			check=True,
+		)
+	except subprocess.CalledProcessError as exc:
+		raise RuntimeError(
+			"Unable to install Chromium automatically. "
+			"Run: python -m playwright install chromium"
+		) from exc
+
+	print()
+	print("Chromium installation complete.")
+
+
+def launch_browser(p):
+	browser_data_dir = get_browser_data_dir()
+	browser_data_dir.mkdir(
+		parents=True,
+		exist_ok=True,
+	)
+
+	try:
+		return p.chromium.launch_persistent_context(
+			user_data_dir=str(browser_data_dir),
+			headless=False,
+		)
+
+	except PlaywrightError as exc:
+		message = str(exc)
+
+		# Playwright gives an "Executable doesn't exist"
+		# error when its Chromium build has not yet been installed.
+		if "Executable doesn't exist" not in message:
+			raise
+
+		install_chromium()
+
+		return p.chromium.launch_persistent_context(
+			user_data_dir=str(browser_data_dir),
+			headless=False,
+		)
+
+
+def get_page(context):
+	if context.pages:
+		return context.pages[0]
+
+	return context.new_page()
+
+
+def open_profile(
+	p,
+	profile_url,
+	model,
+	post_pattern,
+	get_post_count,
+):
+	context = launch_browser(p)
+	page = get_page(context)
+
+	print(f"\nOpening {profile_url}")
+
+	try:
+		page.goto(
+			profile_url,
+			wait_until="domcontentloaded",
+			timeout=60000,
+		)
+	except PlaywrightTimeoutError:
+		print(
+			"Initial page load timed out; "
+			"checking page anyway."
+		)
+
+	page.wait_for_timeout(2000)
+
+	initial_count = get_post_count(
+		page,
+		model,
+		post_pattern,
+	)
+
+	if initial_count == 0:
+		print()
+		print("No profile posts detected yet.")
+		print(
+			"If browser verification is displayed, "
+			"complete it in the browser."
+		)
+		print()
+
+		input(
+			"Press Enter here when the profile is visible..."
+		)
+
+		page.wait_for_timeout(1000)
+
+		initial_count = get_post_count(
+			page,
+			model,
+			post_pattern,
+		)
+
+		if initial_count == 0:
+			context.close()
+
+			raise RuntimeError(
+				"Still unable to detect profile posts."
+			)
+
+	return context, page
+
+
+def resolve_video_fallbacks(
+	p,
+	failed_videos,
+):
+	if not failed_videos:
+		return {}
+
+	print()
+	print("Resolving failed video URLs...")
+	print("Opening browser for fallback lookup...")
+
+	context = launch_browser(p)
+	page = get_page(context)
+
+	resolved = {}
+
+	try:
+		for index, item in enumerate(
+			failed_videos,
+			start=1,
+		):
+			post_number = item["post_number"]
+			post_url = item["post_url"]
+
+			print(
+				f"[{index}/{len(failed_videos)}] "
+				f"Post {post_number}",
+				end=" ",
+				flush=True,
+			)
+
+			try:
+				video_url = get_video_fallback_url(
+					page,
+					post_url,
+				)
+
+				if video_url:
+					resolved[post_number] = video_url
+					print("✓ found")
+				else:
+					print("✗ no MP4 source")
+
+			except Exception as exc:
+				print(f"✗ {exc}")
+
+	finally:
+		context.close()
+
+	print("Fallback browser closed.")
+
+	return resolved