webscout 8.2.2__py3-none-any.whl → 2026.1.19__py3-none-any.whl

webscout/Extra/YTToolkit/ytapi/README.md

@@ -0,0 +1,230 @@
+# 🎬 YTAPI: YouTube Data Extraction Module
+
+## Overview
+
+YTAPI is a powerful, lightweight YouTube data extraction module within the Webscout Python package. It provides comprehensive tools for retrieving YouTube video, channel, playlist, and search data without requiring an API key.
+
+## ✨ Features
+
+### Core Classes
+
+- **Video** - Video metadata, comments, chapters, related videos
+- **Channel** - Channel info, uploads, streams, playlists
+- **Playlist** - Playlist metadata and videos
+- **Search** - Search videos, channels, playlists, Shorts, live streams
+
+### New Classes
+
+- **Suggestions** - YouTube search autocomplete
+- **Shorts** - YouTube Shorts detection and search
+- **Hashtag** - Videos by hashtag
+- **Captions** - Video transcripts and subtitles
+- **Extras** - Trending content by category
+
+## 📦 Installation
+
+```bash
+pip install webscout
+```
+
+## 💡 Quick Examples
+
+### Video Operations
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Video
+
+video = Video("dQw4w9WgXcQ")
+
+# Get metadata
+meta = video.metadata
+print(f"Title: {meta['title']}")
+print(f"Views: {meta['views']}")
+
+# New features
+print(f"Is Live: {video.is_live}")
+print(f"Is Short: {video.is_short}")
+print(f"Hashtags: {video.hashtags}")
+
+# Get related videos
+related = video.get_related_videos(5)
+print(f"Related: {related}")
+
+# Get chapters
+chapters = video.get_chapters()
+if chapters:
+    for ch in chapters:
+        print(f"{ch['start_time']} - {ch['title']}")
+
+# Stream comments
+for comment in video.stream_comments(10):
+    print(f"{comment['author']}: {comment['text'][:50]}")
+```
+
+### Search Operations
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Search
+
+# Basic search
+videos = Search.videos("python tutorial", limit=10)
+channels = Search.channels("tech", limit=5)
+
+# New search features
+shorts = Search.shorts("funny cats", limit=10)
+live = Search.live_streams("music", limit=5)
+
+# Filtered search
+recent = Search.videos_by_upload_date("news", when="today", limit=10)
+short_vids = Search.videos_by_duration("tutorial", duration="short", limit=10)
+```
+
+### Suggestions (NEW)
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Suggestions
+
+# Get autocomplete suggestions
+suggestions = Suggestions.autocomplete("how to")
+print(suggestions)
+# ['how to make money', 'how to tie a tie', ...]
+
+# Get trending searches
+trending = Suggestions.trending_searches()
+print(trending)
+```
+
+### Shorts (NEW)
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Shorts
+
+# Check if a video is a Short
+is_short = Shorts.is_short("video_id")
+
+# Get trending Shorts
+trending = Shorts.get_trending(10)
+
+# Search for Shorts
+results = Shorts.search("dance", limit=20)
+```
+
+### Hashtag (NEW)
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Hashtag
+
+# Get videos by hashtag
+videos = Hashtag.get_videos("python", limit=20)
+
+# Get hashtag metadata
+meta = Hashtag.get_metadata("coding")
+print(f"Tag: {meta['tag']}")
+print(f"Sample videos: {meta['sample_videos']}")
+
+# Extract hashtags from text
+tags = Hashtag.extract_from_text("Check out my #python #tutorial!")
+print(tags)  # ['python', 'tutorial']
+```
+
+### Captions (NEW)
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Captions
+
+# Get available languages
+langs = Captions.get_available_languages("dQw4w9WgXcQ")
+for lang in langs:
+    print(f"{lang['code']}: {lang['name']}")
+
+# Get transcript
+transcript = Captions.get_transcript("dQw4w9WgXcQ", "en")
+print(transcript[:500])
+
+# Get timed transcript
+timed = Captions.get_timed_transcript("dQw4w9WgXcQ")
+for entry in timed[:5]:
+    print(f"{entry['start']:.1f}s: {entry['text']}")
+
+# Search within transcript
+matches = Captions.search_transcript("dQw4w9WgXcQ", "never gonna")
+for match in matches:
+    print(f"{match['start']:.1f}s: {match['text']}")
+```
+
+### Extras (Trending)
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Extras
+
+# Get trending content
+trending = Extras.trending_videos(10)
+music = Extras.music_videos(10)
+gaming = Extras.gaming_videos(10)
+news = Extras.news_videos(10)
+live = Extras.live_videos(10)
+sports = Extras.sport_videos(10)
+educational = Extras.educational_videos(10)
+
+# New categories
+shorts = Extras.shorts_videos(10)
+movies = Extras.movies(10)
+podcasts = Extras.podcasts(10)
+```
+
+### Channel Operations
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Channel
+
+channel = Channel("@MrBeast")
+
+# Get metadata
+meta = channel.metadata
+print(f"Name: {meta['name']}")
+print(f"Subscribers: {meta['subscribers']}")
+
+# Get uploads
+uploads = channel.uploads(20)
+
+# Check if live
+if channel.live:
+    print(f"Currently streaming: {channel.streaming_now}")
+
+# Get playlists
+playlists = channel.playlists
+```
+
+### Playlist Operations
+
+```python
+from webscout.Extra.YTToolkit.ytapi import Playlist
+
+playlist = Playlist("PLrAXtmErZgOeiKm4sgNOknGvNjby9efdf")
+
+# Get metadata
+meta = playlist.metadata
+print(f"Name: {meta['name']}")
+print(f"Videos: {meta['video_count']}")
+print(f"Video IDs: {meta['videos']}")
+```
+
+## 🔧 Available Classes
+
+| Class | Methods |
+|-------|---------|
+| `Video` | `metadata`, `embed_html`, `embed_url`, `thumbnail_url`, `thumbnail_urls`, `is_live`, `is_short`, `hashtags`, `get_related_videos`, `get_chapters`, `stream_comments` |
+| `Channel` | `metadata`, `live`, `streaming_now`, `current_streams`, `old_streams`, `uploads`, `last_uploaded`, `upcoming`, `playlists` |
+| `Playlist` | `metadata` |
+| `Search` | `video`, `videos`, `channel`, `channels`, `playlist`, `playlists`, `shorts`, `live_streams`, `videos_by_duration`, `videos_by_upload_date` |
+| `Suggestions` | `autocomplete`, `trending_searches` |
+| `Shorts` | `is_short`, `get_trending`, `search` |
+| `Hashtag` | `get_videos`, `get_metadata`, `extract_from_text` |
+| `Captions` | `get_available_languages`, `get_transcript`, `get_timed_transcript`, `search_transcript` |
+| `Extras` | `trending_videos`, `music_videos`, `gaming_videos`, `news_videos`, `live_videos`, `sport_videos`, `educational_videos`, `shorts_videos`, `movies`, `podcasts` |
+
+## ⚠️ Notes
+
+- All features work without a YouTube API key
+- Some features may break if YouTube changes their HTML structure
+- Rate limiting may apply for excessive requests

webscout/Extra/YTToolkit/ytapi/__init__.py

@@ -1,6 +1,22 @@
-from .errors import *
-from .video import Video
-from .query import Search
-from .extras import Extras
-from .channel import Channel
-from .playlist import Playlist
+from .captions import Captions
+from .channel import Channel
+from .errors import *
+from .extras import Extras
+from .hashtag import Hashtag
+from .playlist import Playlist
+from .query import Search
+from .shorts import Shorts
+from .suggestions import Suggestions
+from .video import Video
+
+__all__ = [
+    'Video',
+    'Search',
+    'Extras',
+    'Channel',
+    'Playlist',
+    'Suggestions',
+    'Shorts',
+    'Hashtag',
+    'Captions'
+]

webscout/Extra/YTToolkit/ytapi/captions.py

@@ -0,0 +1,190 @@
+"""YouTube video captions and transcripts.
+
+This module wraps the YTTranscriber for a simplified interface.
+"""
+import re
+from typing import Any, Dict, List, Optional
+
+from curl_cffi.requests import Session
+
+# Use the existing robust YTTranscriber
+from webscout.Extra.YTToolkit.transcriber import TranscriptListFetcher, YTTranscriber
+
+
+class Captions:
+    """Class for YouTube captions and transcripts.
+
+    Uses YTTranscriber internally for reliable transcript fetching.
+
+    Example:
+        >>> from webscout.Extra.YTToolkit.ytapi import Captions
+        >>> transcript = Captions.get_transcript("dQw4w9WgXcQ")
+        >>> print(transcript[:100])
+    """
+
+    @staticmethod
+    def _extract_video_id(video_id: str) -> str:
+        """Extract clean video ID from URL or ID."""
+        if not video_id:
+            return ""
+
+        patterns = [
+            r'(?:v=|youtu\.be/|shorts/)([a-zA-Z0-9_-]{11})',
+            r'^([a-zA-Z0-9_-]{11})$'
+        ]
+
+        for pattern in patterns:
+            match = re.search(pattern, video_id)
+            if match:
+                return match.group(1)
+
+        return video_id
+
+    @staticmethod
+    def get_available_languages(video_id: str) -> List[Dict[str, str]]:
+        """
+        Get available caption languages for a video.
+
+        Args:
+            video_id: YouTube video ID or URL
+
+        Returns:
+            List of dicts with 'code', 'name', 'is_auto' for each language
+        """
+        if not video_id:
+            return []
+
+        video_id = Captions._extract_video_id(video_id)
+
+        try:
+            session = Session()
+            session.headers.update({
+                'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'
+            })
+
+            fetcher = TranscriptListFetcher(session)
+            transcript_list = fetcher.fetch(video_id)
+
+            languages = []
+
+            # Add manually created transcripts
+            for transcript in transcript_list._manually_created_transcripts.values():
+                languages.append({
+                    'code': transcript.language_code,
+                    'name': transcript.language,
+                    'is_auto': False
+                })
+
+            # Add generated transcripts
+            for transcript in transcript_list._generated_transcripts.values():
+                languages.append({
+                    'code': transcript.language_code,
+                    'name': transcript.language,
+                    'is_auto': True
+                })
+
+            return languages
+        except Exception:
+            return []
+
+    @staticmethod
+    def get_transcript(video_id: str, language: str = "en") -> Optional[str]:
+        """
+        Get plain text transcript for a video.
+
+        Args:
+            video_id: YouTube video ID or URL
+            language: Language code (e.g., 'en', 'es')
+
+        Returns:
+            Transcript text or None
+        """
+        timed = Captions.get_timed_transcript(video_id, language)
+        if not timed:
+            return None
+
+        return " ".join([entry['text'] for entry in timed])
+
+    @staticmethod
+    def get_timed_transcript(video_id: str, language: str = "en") -> Optional[List[Dict[str, Any]]]:
+        """
+        Get transcript with timestamps.
+
+        Args:
+            video_id: YouTube video ID or URL
+            language: Language code (e.g., 'en', 'es'). Use 'any' for first available.
+
+        Returns:
+            List of dicts with 'text', 'start', 'duration' or None
+        """
+        if not video_id:
+            return None
+
+        video_id = Captions._extract_video_id(video_id)
+
+        try:
+            # Use YTTranscriber for reliable fetching
+            transcript = YTTranscriber.get_transcript(
+                video_id,
+                languages=language if language != 'any' else None
+            )
+            return transcript
+        except Exception:
+            # If requested language fails, try any available
+            if language != 'any':
+                try:
+                    transcript = YTTranscriber.get_transcript(video_id, languages=None)
+                    return transcript
+                except Exception:
+                    pass
+            return None
+
+    @staticmethod
+    def search_transcript(video_id: str, query: str, language: str = "en") -> List[Dict[str, Any]]:
+        """
+        Search within a video's transcript.
+
+        Args:
+            video_id: YouTube video ID or URL
+            query: Text to search for
+            language: Language code
+
+        Returns:
+            List of matching segments with timestamps
+        """
+        if not query:
+            return []
+
+        transcript = Captions.get_timed_transcript(video_id, language)
+        if not transcript:
+            return []
+
+        query_lower = query.lower()
+        results = []
+
+        for entry in transcript:
+            if query_lower in entry['text'].lower():
+                results.append(entry)
+
+        return results
+
+
+if __name__ == "__main__":
+    # Test with a video that has captions
+    video_id = "dQw4w9WgXcQ"
+
+    print("Available languages:")
+    langs = Captions.get_available_languages(video_id)
+    for lang in langs[:5]:
+        print(f"  - {lang['code']}: {lang['name']} (auto: {lang.get('is_auto', False)})")
+
+    print("\nGetting transcript:")
+    transcript = Captions.get_timed_transcript(video_id)
+    if transcript:
+        print(f"  Found {len(transcript)} segments")
+        for entry in transcript[:3]:
+            print(f"  {entry['start']:.1f}s: {entry['text'][:50]}")
+    else:
+        print("  No transcript available")
+
+