eaf-base-api 4.1.1__tar.gz → 4.2.1__tar.gz

{eaf_base_api-4.1.1 → eaf_base_api-4.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: eaf_base_api
-Version: 4.1.1
+Version: 4.2.1
 Summary: A base API for EchterAlsFake's Porn APIs
 Author: Johannes Habel
 Author-email: Johannes Habel <EchterAlsFake@proton.me>
@@ -11,7 +11,6 @@ Requires-Dist: cachetools>=7.0.0
 Requires-Dist: curl-cffi
 Requires-Dist: tenacity>=9.1.2
 Requires-Dist: m3u8 ; extra == 'hls'
-Requires-Dist: av ; python_full_version >= '3.12' and extra == 'hls'
 Requires-Python: >=3.12
 Project-URL: Homepage, https://github.com/EchterAlsFake/eaf_base_api
 Project-URL: Repository, https://github.com/EchterAlsFake/eaf_base_api
@@ -34,7 +33,92 @@ support, and there was a lot of code that I would have rewritten in every API ag
 package. The `BaseCore` class does all the necessary stuff like m3u8 parsing, a great caching system, network request
 fetching with retry attempts and proxy support.
 
-# Documentation (IMPORTANT!) 
+## Consistent errors and logging (4.2)
+
+Configure logging once in your application to capture all 15 provider APIs and
+the base library:
+
+```python
+import logging
+from base_api import configure_app_logging, DownloadFailed
+
+configure_app_logging(level=logging.WARNING, log_file="api-errors.log")
+
+# Inside your async application:
+try:
+    await video.download(configuration)
+except DownloadFailed as error:
+    print(error.api, error.class_name, error.url)
+    # The failure and its full traceback have already been logged.
+```
+
+Logs use Python module names, such as `xvideos_api.api` and `base_api.base`.
+Request, source-loading, and download logs include the qualified class name and
+URL in the message and in `LogRecord.class_name` / `LogRecord.url`. For example:
+
+```text
+ERROR xvideos_api.api [class=xvideos_api.api.Video url=https://example.test/video/123] Download failed: disk full
+Traceback (most recent call last):
+  ...
+OSError: disk full
+```
+
+Network logs identify the actual request URL, including CDN/segment URLs; media
+errors also identify the original video page. Context stays separate for
+concurrent downloads and follows parsing work into worker threads. Creating a
+client/core does not install handlers or override your application's log level.
+Without configuration, Python's fallback logging handler still prints warnings
+and errors to stderr.
+
+Catch common failures from `base_api`: `DownloadFailed`, `NetworkError`,
+`NotFound`, `ProxyError`, `BotDetection`, `VideoUnavailable`, `LoginFailed`, and
+`RegionBlocked`. Existing provider exception imports remain compatible,
+including Pornhub's `PornhubAPIError` hierarchy. Translated exceptions preserve
+the original exception as `__cause__`.
+
+Ordinary download failures now raise `DownloadFailed` instead of returning
+`False`. A stop signal raises `base_api.modules.errors.DownloadCancelled`;
+async task cancellation propagates unchanged. Neither is logged as an error.
+With `DownloadConfigHLS(return_report=True)`, incomplete/cancelled downloads
+still return their explicit `DownloadReport`; unexpected exceptions still raise.
+
+Shared request translation, download error handling, HLS preparation, and output
+configuration live in `base_api.modules.provider`. The identical Tube8 and
+Thumbzilla result-grid extraction lives in `base_api.modules.static_functions`.
+Site-specific parsing, quality selection, and fallback behavior stay in each
+provider. Updated providers require `eaf-base-api>=4.2.0`; update the base library
+alongside them.
+
+## FFmpeg remuxing and single-pass metadata tagging (4.3)
+
+HLS TS→MP4 remuxing (`DownloadConfigHLS(remux=True)`) now runs directly via an
+`ffmpeg` binary (discovered on `PATH` or configured via `RuntimeConfig.ffmpeg_path`)
+instead of `PyAV`. You can also pass `metadata` (a provider `Video` object or `dict`)
+and `thumbnail` (`bytes` or path) on `DownloadConfigHLS` so `eaf_base_api` embeds
+MP4 container tags and cover art in the **same single FFmpeg remux pass** without
+rewriting the file a second time:
+
+```python
+from base_api import DownloadConfigHLS, write_metadata_tags
+
+config = DownloadConfigHLS(
+    quality="best",
+    path="./downloads",
+    remux=True,
+    metadata=video,           # Writes title, author, genre, date, description, url, comment
+    thumbnail=cover_bytes,    # Optional JPEG/PNG cover art embedded in the same pass
+    return_report=True,
+)
+report = await video.download(configuration=config)
+print(report.metadata_written)  # True when single-pass metadata/cover tagging succeeded
+```
+
+For direct MP4 downloads (`DownloadConfigRAW`) or existing MP4 files, use
+`write_metadata_tags(path, metadata=video, thumbnail=cover_bytes)` from
+`base_api.modules.ffmpeg` (or `base_api`). If `ffmpeg` is not installed when
+`remux=True`, `eaf_base_api` logs a warning and preserves the raw MPEG-TS stream.
+
+# Documentation (IMPORTANT!)
 > [!IMPORTANT]
 > Configuring eaf_base_api is necessary if you use any of my Porn APIs, because they all depend on this project.
 > Please read through the documentation to learn how `PROXIES`, `CACHING` and `LOGGING` etc... work!
@@ -154,7 +238,10 @@ stream = helper.iterator(
 async with stream:
     async for result in stream:
         if not result.succeeded:
-            logger.error("%s failed: %s", result.stage, result.error)
+            logger.error(
+                "%s failed for %s: %s", result.stage, result.url, result.error,
+                exc_info=(type(result.error), result.error, result.error.__traceback__),
+            )
             continue
         video = result.unwrap()
 ```
@@ -165,6 +252,31 @@ or `ErrorMode.RAISE`. `RetryPolicy` provides a strict maximum attempt count and
 optional exponential delay; the independent page and item handlers return an
 `ErrorAction` and cannot create an unbounded retry loop.
 
+## Error logging
+
+Configure logging once in the application to capture all provider and base API logs:
+
+```python
+import logging
+from base_api.modules.logger import configure_app_logging
+
+configure_app_logging(log_file="api.log", level=logging.INFO)
+```
+
+Failures include the operation, video/page URL, and original traceback. The default
+formatter also shows the file, line, and function. Media source failures are logged
+even when a caller catches the exception; iterator failures are logged even when
+the configured policy skips or yields them. Provider CLIs configure console logging
+automatically.
+
+Provider request and download exceptions use the shared types in
+`base_api.modules.errors`, preserving the original exception as `__cause__`.
+Pornhub's existing exception classes remain catchable as `PornhubAPIError` and as
+their shared equivalents. `ScraperException` also inherits `BaseScraperError`.
+Download preparation errors now carry the video URL in `DownloadFailed`, and
+explicit cancellation remains `DownloadCancelled` or `asyncio.CancelledError`.
+Existing boolean/download-report results from the base downloader remain supported.
+
 # Can I use this for myself?
 Yes, you can, but I may change stuff here and there from time to time, and it would maybe break your project.
 I would not recommend you to install and use it as a package, but just copy the code you need.

{eaf_base_api-4.1.1 → eaf_base_api-4.2.1}/README.md

@@ -14,7 +14,92 @@ support, and there was a lot of code that I would have rewritten in every API ag
 package. The `BaseCore` class does all the necessary stuff like m3u8 parsing, a great caching system, network request
 fetching with retry attempts and proxy support.
 
-# Documentation (IMPORTANT!) 
+## Consistent errors and logging (4.2)
+
+Configure logging once in your application to capture all 15 provider APIs and
+the base library:
+
+```python
+import logging
+from base_api import configure_app_logging, DownloadFailed
+
+configure_app_logging(level=logging.WARNING, log_file="api-errors.log")
+
+# Inside your async application:
+try:
+    await video.download(configuration)
+except DownloadFailed as error:
+    print(error.api, error.class_name, error.url)
+    # The failure and its full traceback have already been logged.
+```
+
+Logs use Python module names, such as `xvideos_api.api` and `base_api.base`.
+Request, source-loading, and download logs include the qualified class name and
+URL in the message and in `LogRecord.class_name` / `LogRecord.url`. For example:
+
+```text
+ERROR xvideos_api.api [class=xvideos_api.api.Video url=https://example.test/video/123] Download failed: disk full
+Traceback (most recent call last):
+  ...
+OSError: disk full
+```
+
+Network logs identify the actual request URL, including CDN/segment URLs; media
+errors also identify the original video page. Context stays separate for
+concurrent downloads and follows parsing work into worker threads. Creating a
+client/core does not install handlers or override your application's log level.
+Without configuration, Python's fallback logging handler still prints warnings
+and errors to stderr.
+
+Catch common failures from `base_api`: `DownloadFailed`, `NetworkError`,
+`NotFound`, `ProxyError`, `BotDetection`, `VideoUnavailable`, `LoginFailed`, and
+`RegionBlocked`. Existing provider exception imports remain compatible,
+including Pornhub's `PornhubAPIError` hierarchy. Translated exceptions preserve
+the original exception as `__cause__`.
+
+Ordinary download failures now raise `DownloadFailed` instead of returning
+`False`. A stop signal raises `base_api.modules.errors.DownloadCancelled`;
+async task cancellation propagates unchanged. Neither is logged as an error.
+With `DownloadConfigHLS(return_report=True)`, incomplete/cancelled downloads
+still return their explicit `DownloadReport`; unexpected exceptions still raise.
+
+Shared request translation, download error handling, HLS preparation, and output
+configuration live in `base_api.modules.provider`. The identical Tube8 and
+Thumbzilla result-grid extraction lives in `base_api.modules.static_functions`.
+Site-specific parsing, quality selection, and fallback behavior stay in each
+provider. Updated providers require `eaf-base-api>=4.2.0`; update the base library
+alongside them.
+
+## FFmpeg remuxing and single-pass metadata tagging (4.3)
+
+HLS TS→MP4 remuxing (`DownloadConfigHLS(remux=True)`) now runs directly via an
+`ffmpeg` binary (discovered on `PATH` or configured via `RuntimeConfig.ffmpeg_path`)
+instead of `PyAV`. You can also pass `metadata` (a provider `Video` object or `dict`)
+and `thumbnail` (`bytes` or path) on `DownloadConfigHLS` so `eaf_base_api` embeds
+MP4 container tags and cover art in the **same single FFmpeg remux pass** without
+rewriting the file a second time:
+
+```python
+from base_api import DownloadConfigHLS, write_metadata_tags
+
+config = DownloadConfigHLS(
+    quality="best",
+    path="./downloads",
+    remux=True,
+    metadata=video,           # Writes title, author, genre, date, description, url, comment
+    thumbnail=cover_bytes,    # Optional JPEG/PNG cover art embedded in the same pass
+    return_report=True,
+)
+report = await video.download(configuration=config)
+print(report.metadata_written)  # True when single-pass metadata/cover tagging succeeded
+```
+
+For direct MP4 downloads (`DownloadConfigRAW`) or existing MP4 files, use
+`write_metadata_tags(path, metadata=video, thumbnail=cover_bytes)` from
+`base_api.modules.ffmpeg` (or `base_api`). If `ffmpeg` is not installed when
+`remux=True`, `eaf_base_api` logs a warning and preserves the raw MPEG-TS stream.
+
+# Documentation (IMPORTANT!)
 > [!IMPORTANT]
 > Configuring eaf_base_api is necessary if you use any of my Porn APIs, because they all depend on this project.
 > Please read through the documentation to learn how `PROXIES`, `CACHING` and `LOGGING` etc... work!
@@ -134,7 +219,10 @@ stream = helper.iterator(
 async with stream:
     async for result in stream:
         if not result.succeeded:
-            logger.error("%s failed: %s", result.stage, result.error)
+            logger.error(
+                "%s failed for %s: %s", result.stage, result.url, result.error,
+                exc_info=(type(result.error), result.error, result.error.__traceback__),
+            )
             continue
         video = result.unwrap()
 ```
@@ -145,6 +233,31 @@ or `ErrorMode.RAISE`. `RetryPolicy` provides a strict maximum attempt count and
 optional exponential delay; the independent page and item handlers return an
 `ErrorAction` and cannot create an unbounded retry loop.
 
+## Error logging
+
+Configure logging once in the application to capture all provider and base API logs:
+
+```python
+import logging
+from base_api.modules.logger import configure_app_logging
+
+configure_app_logging(log_file="api.log", level=logging.INFO)
+```
+
+Failures include the operation, video/page URL, and original traceback. The default
+formatter also shows the file, line, and function. Media source failures are logged
+even when a caller catches the exception; iterator failures are logged even when
+the configured policy skips or yields them. Provider CLIs configure console logging
+automatically.
+
+Provider request and download exceptions use the shared types in
+`base_api.modules.errors`, preserving the original exception as `__cause__`.
+Pornhub's existing exception classes remain catchable as `PornhubAPIError` and as
+their shared equivalents. `ScraperException` also inherits `BaseScraperError`.
+Download preparation errors now carry the video URL in `DownloadFailed`, and
+explicit cancellation remains `DownloadCancelled` or `asyncio.CancelledError`.
+Existing boolean/download-report results from the base downloader remain supported.
+
 # Can I use this for myself?
 Yes, you can, but I may change stuff here and there from time to time, and it would maybe break your project.
 I would not recommend you to install and use it as a package, but just copy the code you need.

{eaf_base_api-4.1.1 → eaf_base_api-4.2.1}/base_api/__init__.py

@@ -15,6 +15,7 @@ __all__ = [
     "FieldNotLoadableError",
     "Helper",
     "ItemFetchError",
+    "IteratorConfig",
     "LoadState",
     "LoaderConfigurationError",
     "LoaderContractError",
@@ -35,6 +36,35 @@ __all__ = [
     "errors",
     "media_field",
     "UnknownMediaFieldError",
+    "make_iterator_config",
+    "default_on_error",
+    "scrape_stream",
+    "stream_results",
+    "is_resource_gone",
+    "contains_resource_gone",
+    "parse_duration",
+    "parse_count",
+    "get_text_safe",
+    "get_attr_safe",
+    "build_m3u8_master",
+    "str_to_bool",
+    "ScraperException",
+    "NotFound",
+    "NetworkError",
+    "BotDetection",
+    "ProxyError",
+    "UnknownNetworkError",
+    "DownloadFailed",
+    "MetadataWriteError",
+    "VideoUnavailable",
+    "ResourceGone",
+    "LoginFailed",
+    "RegionBlocked",
+    "configure_app_logging",
+    "find_ffmpeg",
+    "has_ffmpeg",
+    "remux_ts_to_mp4",
+    "write_metadata_tags",
 ]
 
 
@@ -53,7 +83,28 @@ from base_api.modules.errors import (
     RequestRetriesExhausted,
     ScrapeOperationError,
     UnknownMediaFieldError,
+    is_resource_gone,
+    contains_resource_gone,
+    ScraperException,
+    NotFound,
+    NetworkError,
+    BotDetection,
+    ProxyError,
+    UnknownNetworkError,
+    DownloadFailed,
+    MetadataWriteError,
+    VideoUnavailable,
+    ResourceGone,
+    LoginFailed,
+    RegionBlocked,
 )
+from base_api.modules.ffmpeg import (
+    find_ffmpeg,
+    has_ffmpeg,
+    remux_ts_to_mp4,
+    write_metadata_tags,
+)
+from base_api.modules.logger import configure_app_logging
 from base_api.base import (
     BaseCore,
     BaseMedia,
@@ -74,5 +125,22 @@ from base_api.base import (
     ScrapeStream,
     SegmentCacheKey,
     media_field,
+    scrape_stream,
+    stream_results,
+)
+from base_api.modules.config import (
+    config,
+    DownloadConfigHLS,
+    DownloadConfigRAW,
+    IteratorConfig,
+    make_iterator_config,
+    default_on_error,
+)
+from base_api.modules.static_functions import (
+    str_to_bool,
+    parse_duration,
+    parse_count,
+    get_text_safe,
+    get_attr_safe,
+    build_m3u8_master,
 )
-from base_api.modules.config import config, DownloadConfigHLS, DownloadConfigRAW