yt-meta 0.1.0__tar.gz

yt_meta-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Shane Isley
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE. 

yt_meta-0.1.0/PKG-INFO

@@ -0,0 +1,222 @@
+Metadata-Version: 2.4
+Name: yt-meta
+Version: 0.1.0
+Summary: A lightweight YouTube metadata library.
+Author-email: Shane <shane.isley@gmail.com>
+License: MIT License
+        
+        Copyright (c) 2025 Shane Isley
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE. 
+Project-URL: Homepage, https://github.com/shaneisley/yt-meta
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Natural Language :: English
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Multimedia :: Video
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: youtube-comment-downloader>=0.1.76
+Requires-Dist: beautifulsoup4>=4.12.3
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: python-dateutil>=2.9.0
+Requires-Dist: loguru>=0.7.2
+Provides-Extra: dev
+Requires-Dist: pytest>=8.3.2; extra == "dev"
+Requires-Dist: pytest-cov>=6.2.1; extra == "dev"
+Requires-Dist: pytest-mock>=3.14.1; extra == "dev"
+Requires-Dist: freezegun>=1.5.1; extra == "dev"
+Requires-Dist: ruff>=0.5.5; extra == "dev"
+Dynamic: license-file
+
+# yt-meta
+
+A Python library for finding video and channel metadata from YouTube.
+
+## Purpose
+
+This library is designed to provide a simple and efficient way to collect metadata for YouTube videos and channels, such as titles, view counts, likes, and descriptions. It is built to support data analysis, research, or any application that needs structured information from YouTube.
+
+## Installation
+
+You can install `yt-meta` from PyPI:
+
+```bash
+pip install yt-meta
+```
+
+To install the latest development version directly from GitHub:
+```bash
+pip install git+https://github.com/sw-yx/yt-meta.git
+```
+
+## Inspiration
+
+This project extends the great `youtube-comment-downloader` library, inheriting its session management while adding additional metadata capabilities.
+
+## Core Features
+
+The library offers several ways to fetch metadata.
+
+### 1. Get Video Metadata
+
+Fetches comprehensive metadata for a specific YouTube video.
+
+**Example:**
+
+```python
+from yt_meta import YtMetaClient
+
+client = YtMetaClient()
+video_url = "https://www.youtube.com/watch?v=B68agR-OeJM"
+metadata = client.get_video_metadata(video_url)
+print(f"Title: {metadata['title']}")
+```
+
+### 2. Get Channel Metadata
+
+Fetches metadata for a specific YouTube channel.
+
+**Example:**
+
+```python
+from yt_meta import YtMetaClient
+
+client = YtMetaClient()
+channel_url = "https://www.youtube.com/@samwitteveenai"
+channel_metadata = client.get_channel_metadata(channel_url)
+print(f"Channel Name: {channel_metadata['title']}")
+```
+
+### 3. Get All Videos from a Channel
+
+Returns a generator that yields metadata for all videos on a channel's "Videos" tab, handling pagination automatically.
+
+**Example:**
+```python
+import itertools
+from yt_meta import YtMetaClient
+
+client = YtMetaClient()
+channel_url = "https://www.youtube.com/@AI-Makerspace/videos"
+videos_generator = client.get_channel_videos(channel_url)
+
+# Print the first 5 videos
+for video in itertools.islice(videos_generator, 5):
+    print(f"- {video['title']} (ID: {video['videoId']})")
+```
+
+### 4. Filtering Channel Videos by Date
+
+You can efficiently filter videos by a date range using the `start_date` and `end_date` arguments. The library automatically stops fetching older videos once it passes the `start_date`, saving time and network requests.
+
+The date arguments can be a `datetime.date` object or a string in two formats:
+*   **Shorthand:** `"1d"`, `"2w"`, `"3m"`, `"4y"` (days, weeks, months, years)
+*   **Human-readable:** `"1 day ago"`, `"2 weeks ago"`
+
+**Example:**
+
+```python
+from datetime import date, timedelta
+from yt_meta import YtMetaClient
+import itertools
+
+client = YtMetaClient()
+channel_url = "https://www.youtube.com/@samwitteveenai/videos"
+
+# Get videos from the last 30 days
+print("\n--- Videos from the last 30 days ---")
+recent_videos = client.get_channel_videos(channel_url, start_date="30d")
+for video in itertools.islice(recent_videos, 5):
+    print(f"- {video.get('title')}")
+
+# Get videos from a specific window (90 to 60 days ago)
+print("\n--- Videos from a specific 30-day window in the past ---")
+start = date.today() - timedelta(days=90)
+end = date.today() - timedelta(days=60)
+window_videos = client.get_channel_videos(channel_url, start_date=start, end_date=end)
+for video in itertools.islice(window_videos, 5):
+    print(f"- {video.get('title')}")
+```
+
+## Logging
+
+`yt-meta` uses Python's `logging` module to provide insights into its operations. To see the log output, you can configure a basic logger.
+
+**Example:**
+```python
+import logging
+
+# Configure logging to print INFO-level messages
+logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+
+# Now, when you use the client, you will see logs
+# ...
+```
+
+## API Reference
+
+### `YtMetaClient()`
+
+The main client for interacting with the library. It inherits from `youtube-comment-downloader` and handles session management and caching.
+
+#### `get_video_metadata(youtube_url: str) -> dict`
+Fetches comprehensive metadata for a single YouTube video.
+-   **`youtube_url`**: The full URL of the YouTube video.
+-   **Returns**: A dictionary containing metadata such as `title`, `description`, `view_count`, `like_count`, `publish_date`, `category`, and more.
+-   **Raises**: `VideoUnavailableError` if the video page cannot be fetched or the video is private/deleted.
+
+#### `get_channel_metadata(channel_url: str, force_refresh: bool = False) -> dict`
+Fetches metadata for a YouTube channel.
+-   **`channel_url`**: The URL of the channel's main page or "Videos" tab.
+-   **`force_refresh`**: If `True`, bypasses the internal cache and fetches fresh data.
+-   **Returns**: A dictionary with channel metadata like `title`, `description`, `subscriber_count`, `vanity_url`, etc.
+-   **Raises**: `VideoUnavailableError`, `MetadataParsingError`.
+
+#### `get_channel_videos(channel_url: str, fetch_full_metadata: bool = False, start_date: Optional[Union[str, date]] = None, end_date: Optional[Union[str, date]] = None) -> Generator[dict, None, None]`
+Returns a generator that yields metadata for all videos on a channel's "Videos" tab. It handles pagination automatically.
+-   **`channel_url`**: URL of the channel's "Videos" tab.
+-   **`fetch_full_metadata`**: If `True`, fetches the complete, detailed metadata for each video. This is slower as it requires an additional request per video. If `False` (default), returns basic metadata available directly from the channel page.
+-   **`start_date`**: The earliest date for videos to include. Can be a `datetime.date` object or a string (e.g., `"30d"`, `"2 months ago"`). The generator will efficiently stop once it encounters videos older than this date.
+-   **`end_date`**: The latest date for videos to include. Can be a `datetime.date` object or a string.
+-   **Yields**: Dictionaries of video metadata. The contents depend on the `fetch_full_metadata` flag.
+
+#### `clear_cache(channel_url: str = None)`
+Clears the internal in-memory cache.
+-   **`channel_url`**: If provided, clears the cache for only that specific channel. If `None` (default), the entire cache is cleared.
+
+## Error Handling
+
+The library uses custom exceptions to signal specific error conditions.
+
+### `YtMetaError`
+The base exception for all errors in this library.
+
+### `MetadataParsingError`
+Raised when the necessary metadata (e.g., the `ytInitialData` JSON object) cannot be found or parsed from the YouTube page. This can happen if YouTube changes its page structure.
+
+### `VideoUnavailableError`
+Raised when a video or channel page cannot be fetched. This could be due to a network error, a deleted/private video, or an invalid URL.

yt_meta-0.1.0/README.md

@@ -0,0 +1,168 @@
+# yt-meta
+
+A Python library for finding video and channel metadata from YouTube.
+
+## Purpose
+
+This library is designed to provide a simple and efficient way to collect metadata for YouTube videos and channels, such as titles, view counts, likes, and descriptions. It is built to support data analysis, research, or any application that needs structured information from YouTube.
+
+## Installation
+
+You can install `yt-meta` from PyPI:
+
+```bash
+pip install yt-meta
+```
+
+To install the latest development version directly from GitHub:
+```bash
+pip install git+https://github.com/sw-yx/yt-meta.git
+```
+
+## Inspiration
+
+This project extends the great `youtube-comment-downloader` library, inheriting its session management while adding additional metadata capabilities.
+
+## Core Features
+
+The library offers several ways to fetch metadata.
+
+### 1. Get Video Metadata
+
+Fetches comprehensive metadata for a specific YouTube video.
+
+**Example:**
+
+```python
+from yt_meta import YtMetaClient
+
+client = YtMetaClient()
+video_url = "https://www.youtube.com/watch?v=B68agR-OeJM"
+metadata = client.get_video_metadata(video_url)
+print(f"Title: {metadata['title']}")
+```
+
+### 2. Get Channel Metadata
+
+Fetches metadata for a specific YouTube channel.
+
+**Example:**
+
+```python
+from yt_meta import YtMetaClient
+
+client = YtMetaClient()
+channel_url = "https://www.youtube.com/@samwitteveenai"
+channel_metadata = client.get_channel_metadata(channel_url)
+print(f"Channel Name: {channel_metadata['title']}")
+```
+
+### 3. Get All Videos from a Channel
+
+Returns a generator that yields metadata for all videos on a channel's "Videos" tab, handling pagination automatically.
+
+**Example:**
+```python
+import itertools
+from yt_meta import YtMetaClient
+
+client = YtMetaClient()
+channel_url = "https://www.youtube.com/@AI-Makerspace/videos"
+videos_generator = client.get_channel_videos(channel_url)
+
+# Print the first 5 videos
+for video in itertools.islice(videos_generator, 5):
+    print(f"- {video['title']} (ID: {video['videoId']})")
+```
+
+### 4. Filtering Channel Videos by Date
+
+You can efficiently filter videos by a date range using the `start_date` and `end_date` arguments. The library automatically stops fetching older videos once it passes the `start_date`, saving time and network requests.
+
+The date arguments can be a `datetime.date` object or a string in two formats:
+*   **Shorthand:** `"1d"`, `"2w"`, `"3m"`, `"4y"` (days, weeks, months, years)
+*   **Human-readable:** `"1 day ago"`, `"2 weeks ago"`
+
+**Example:**
+
+```python
+from datetime import date, timedelta
+from yt_meta import YtMetaClient
+import itertools
+
+client = YtMetaClient()
+channel_url = "https://www.youtube.com/@samwitteveenai/videos"
+
+# Get videos from the last 30 days
+print("\n--- Videos from the last 30 days ---")
+recent_videos = client.get_channel_videos(channel_url, start_date="30d")
+for video in itertools.islice(recent_videos, 5):
+    print(f"- {video.get('title')}")
+
+# Get videos from a specific window (90 to 60 days ago)
+print("\n--- Videos from a specific 30-day window in the past ---")
+start = date.today() - timedelta(days=90)
+end = date.today() - timedelta(days=60)
+window_videos = client.get_channel_videos(channel_url, start_date=start, end_date=end)
+for video in itertools.islice(window_videos, 5):
+    print(f"- {video.get('title')}")
+```
+
+## Logging
+
+`yt-meta` uses Python's `logging` module to provide insights into its operations. To see the log output, you can configure a basic logger.
+
+**Example:**
+```python
+import logging
+
+# Configure logging to print INFO-level messages
+logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')
+
+# Now, when you use the client, you will see logs
+# ...
+```
+
+## API Reference
+
+### `YtMetaClient()`
+
+The main client for interacting with the library. It inherits from `youtube-comment-downloader` and handles session management and caching.
+
+#### `get_video_metadata(youtube_url: str) -> dict`
+Fetches comprehensive metadata for a single YouTube video.
+-   **`youtube_url`**: The full URL of the YouTube video.
+-   **Returns**: A dictionary containing metadata such as `title`, `description`, `view_count`, `like_count`, `publish_date`, `category`, and more.
+-   **Raises**: `VideoUnavailableError` if the video page cannot be fetched or the video is private/deleted.
+
+#### `get_channel_metadata(channel_url: str, force_refresh: bool = False) -> dict`
+Fetches metadata for a YouTube channel.
+-   **`channel_url`**: The URL of the channel's main page or "Videos" tab.
+-   **`force_refresh`**: If `True`, bypasses the internal cache and fetches fresh data.
+-   **Returns**: A dictionary with channel metadata like `title`, `description`, `subscriber_count`, `vanity_url`, etc.
+-   **Raises**: `VideoUnavailableError`, `MetadataParsingError`.
+
+#### `get_channel_videos(channel_url: str, fetch_full_metadata: bool = False, start_date: Optional[Union[str, date]] = None, end_date: Optional[Union[str, date]] = None) -> Generator[dict, None, None]`
+Returns a generator that yields metadata for all videos on a channel's "Videos" tab. It handles pagination automatically.
+-   **`channel_url`**: URL of the channel's "Videos" tab.
+-   **`fetch_full_metadata`**: If `True`, fetches the complete, detailed metadata for each video. This is slower as it requires an additional request per video. If `False` (default), returns basic metadata available directly from the channel page.
+-   **`start_date`**: The earliest date for videos to include. Can be a `datetime.date` object or a string (e.g., `"30d"`, `"2 months ago"`). The generator will efficiently stop once it encounters videos older than this date.
+-   **`end_date`**: The latest date for videos to include. Can be a `datetime.date` object or a string.
+-   **Yields**: Dictionaries of video metadata. The contents depend on the `fetch_full_metadata` flag.
+
+#### `clear_cache(channel_url: str = None)`
+Clears the internal in-memory cache.
+-   **`channel_url`**: If provided, clears the cache for only that specific channel. If `None` (default), the entire cache is cleared.
+
+## Error Handling
+
+The library uses custom exceptions to signal specific error conditions.
+
+### `YtMetaError`
+The base exception for all errors in this library.
+
+### `MetadataParsingError`
+Raised when the necessary metadata (e.g., the `ytInitialData` JSON object) cannot be found or parsed from the YouTube page. This can happen if YouTube changes its page structure.
+
+### `VideoUnavailableError`
+Raised when a video or channel page cannot be fetched. This could be due to a network error, a deleted/private video, or an invalid URL.

yt_meta-0.1.0/pyproject.toml

@@ -0,0 +1,60 @@
+[project]
+name = "yt-meta"
+version = "0.1.0"
+description = "A lightweight YouTube metadata library."
+readme = "README.md"
+authors = [{ name = "Shane", email = "shane.isley@gmail.com" }]
+license = { file = "LICENSE" }
+classifiers = [
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3",
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "Natural Language :: English",
+    "Operating System :: OS Independent",
+    "Topic :: Internet :: WWW/HTTP",
+    "Topic :: Multimedia :: Video",
+    "Topic :: Software Development :: Libraries :: Python Modules",
+]
+dependencies = [
+    "youtube-comment-downloader>=0.1.76",
+    "beautifulsoup4>=4.12.3",
+    "httpx>=0.27.0",
+    "python-dateutil>=2.9.0",
+    "loguru>=0.7.2",
+]
+requires-python = ">=3.9"
+
+[project.urls]
+Homepage = "https://github.com/shaneisley/yt-meta"
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.3.2",
+    "pytest-cov>=6.2.1",
+    "pytest-mock>=3.14.1",
+    "freezegun>=1.5.1",
+    "ruff>=0.5.5",
+]
+
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[tool.pytest.ini_options]
+markers = [
+    "integration: marks tests as integration tests (slow, requires network)",
+]
+addopts = "--durations=10"
+
+[tool.ruff]
+line-length = 120
+
+[tool.ruff.lint]
+# Enable Pyflakes, pycodestyle, and isort rules.
+select = ["E", "F", "I"]
+
+[tool.ruff.format]
+quote-style = "double"

yt_meta-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

yt_meta-0.1.0/tests/test_caching.py

@@ -0,0 +1,111 @@
+import time
+
+import pytest
+
+from yt_meta import YtMetaClient
+
+# A list of diverse YouTube channels for live testing
+LIVE_CHANNEL_URLS = [
+    "https://www.youtube.com/@samwitteveenai/videos",
+    "https://www.youtube.com/@bulwarkmedia/videos",
+    "https://www.youtube.com/@AI-Makerspace/videos",
+]
+
+
+@pytest.fixture
+def client():
+    """Provides a fresh instance of the client for each test, ensuring no state is shared."""
+    return YtMetaClient()
+
+
+@pytest.mark.integration
+@pytest.mark.parametrize("channel_url", LIVE_CHANNEL_URLS)
+def test_live_caching_is_faster(client, channel_url):
+    """
+    Tests that a cached call is significantly faster than the initial network call
+    by fetching from a live channel.
+    """
+    # First call - makes a network request
+    start_time_first = time.time()
+    client.get_channel_metadata(channel_url)
+    duration_first = time.time() - start_time_first
+
+    # Second call - should hit the cache
+    start_time_second = time.time()
+    client.get_channel_metadata(channel_url)
+    duration_second = time.time() - start_time_second
+
+    print(
+        f"\n[{channel_url}] First call: {duration_first:.4f}s, Cached call: {duration_second:.4f}s"
+    )
+    # The cached call should be at least 100x faster (realistically more)
+    assert duration_second < duration_first / 100
+
+
+@pytest.mark.integration
+@pytest.mark.parametrize("channel_url", LIVE_CHANNEL_URLS)
+def test_live_force_refresh_bypasses_cache(client, channel_url):
+    """
+    Tests that `force_refresh=True` correctly re-fetches data by making two
+    network calls of similar duration.
+    """
+    # Initial call to populate the cache
+    start_time_initial = time.time()
+    client.get_channel_metadata(channel_url)
+    duration_initial = time.time() - start_time_initial
+
+    # Second call with force_refresh should also make a network request
+    start_time_refresh = time.time()
+    client.get_channel_metadata(channel_url, force_refresh=True)
+    duration_refresh = time.time() - start_time_refresh
+
+    print(
+        f"\n[{channel_url}] Initial call: {duration_initial:.4f}s, Refresh call: {duration_refresh:.4f}s"
+    )
+    # Both calls should take a similar amount of time; we'll check if the refresh
+    # call took at least 20% of the time of the initial call.
+    assert duration_refresh > duration_initial * 0.2
+
+
+def test_clear_cache_all(client, mocker):
+    """Tests that clear_cache() without arguments clears the entire cache."""
+    # Mock the internal cache to control its state
+    client._channel_page_cache = {"url1": "data1", "url2": "data2"}
+
+    # Act
+    client.clear_cache()
+
+    # Assert
+    assert not client._channel_page_cache  # Cache should be empty
+
+
+def test_clear_cache_specific_url(client, mocker):
+    """Tests that clear_cache() with a URL clears only that entry."""
+    # Arrange
+    client._channel_page_cache = {
+        "https://www.youtube.com/@channel1/videos": "data1",
+        "https://www.youtube.com/@channel2/videos": "data2",
+    }
+
+    # Act
+    client.clear_cache("https://www.youtube.com/@channel1/videos")
+
+    # Assert
+    assert "https://www.youtube.com/@channel1/videos" not in client._channel_page_cache
+    assert "https://www.youtube.com/@channel2/videos" in client._channel_page_cache
+
+
+def test_clear_cache_handles_trailing_slash(client, mocker):
+    """
+    Tests that clear_cache() correctly handles URLs with or without a trailing slash.
+    """
+    # Arrange
+    url_with_slash = "https://www.youtube.com/@channel1/videos/"
+    url_without_slash = "https://www.youtube.com/@channel1/videos"
+    client._channel_page_cache = {url_without_slash: "data1"}
+
+    # Act
+    client.clear_cache(url_with_slash)
+
+    # Assert
+    assert not client._channel_page_cache