get-class-material 0.2.7__tar.gz → 0.2.8__tar.gz

{get_class_material-0.2.7 → get_class_material-0.2.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: get-class-material
-Version: 0.2.7
+Version: 0.2.8
 Summary: One-command bulk downloader for NTU COOL (Canvas) course materials — PDFs, lecture videos, and Pages.
 Author-email: jabir <jabir95tsai@gmail.com>
 License-Expression: MIT
@@ -188,7 +188,7 @@ ntu-cool-gcm
   上課影片:   新增 4、跳過 0、失敗 0
 
 檔案存放位置:
-  C:\Users\你\ntu-cool-gcm_material\音樂、演化與大腦 Music, Evolution and the Brain (57544)
+  C:\Users\你\Documents\ntu-cool-gcm_material\音樂、演化與大腦 Music, Evolution and the Brain (57544)
 ```
 
 **最後那行就是你的檔案放在哪。** 用檔案總管打開就能看到。
@@ -209,7 +209,7 @@ ntu-cool-gcm
 下載到的目錄結構長這樣:
 
 ```
-C:\Users\你\ntu-cool-gcm_material\
+C:\Users\你\Documents\ntu-cool-gcm_material\
 └── 音樂、演化與大腦 Music, Evolution and the Brain (57544)\
     ├── course_overview.md        ← 這份課程的目錄索引(可以餵給 AI)
     ├── week1\
@@ -283,7 +283,19 @@ ntu-cool-gcm --skip-youtube --skip-cool-videos
 ntu-cool-gcm --out D:\我的課程
 ```
 
-預設是當前目錄底下的 `ntu-cool-gcm_material/`。
+預設是在你的「文件 / Documents」資料夾底下的 `ntu-cool-gcm_material/`。
+
+如果你的目前資料夾已經有舊版建立的 `ntu-cool-gcm_material/`,工具會沿用那個資料夾,避免把舊檔案拆散。
+
+### 下載 Word / PowerPoint / Excel / Zip 等非 PDF 檔
+
+預設只下載 PDF,避免把 Word、PowerPoint、Zip 之類的檔案誤存成 `.pdf`。
+
+如果老師有放這類檔案,請加:
+
+```powershell
+ntu-cool-gcm --all-file-types
+```
 
 ### 已經知道課程 ID(網址裡的數字)
 
@@ -380,9 +392,15 @@ ntu-cool-gcm --refresh-session
 
 ### Q: 我的檔案到底放在哪
 
-預設在你**執行 `ntu-cool-gcm` 時所在的目錄**底下,叫 `ntu-cool-gcm_material/`。
+預設在你的「文件 / Documents」資料夾底下,叫 `ntu-cool-gcm_material/`。
+
+常見位置像這樣:
+
+```text
+C:\Users\你\Documents\ntu-cool-gcm_material\
+```
 
-如果你在 PowerShell 看到提示是 `PS C:\Users\你>`,那就是 `C:\Users\你\ntu-cool-gcm_material\`。
+如果你以前已經在目前資料夾建立過 `ntu-cool-gcm_material/`,工具會優先沿用那個舊資料夾。
 
 每次跑完工具最後一行也會印「檔案存放位置:」+ 完整路徑。
 

{get_class_material-0.2.7 → get_class_material-0.2.8}/README.md

@@ -156,7 +156,7 @@ ntu-cool-gcm
   上課影片:   新增 4、跳過 0、失敗 0
 
 檔案存放位置:
-  C:\Users\你\ntu-cool-gcm_material\音樂、演化與大腦 Music, Evolution and the Brain (57544)
+  C:\Users\你\Documents\ntu-cool-gcm_material\音樂、演化與大腦 Music, Evolution and the Brain (57544)
 ```
 
 **最後那行就是你的檔案放在哪。** 用檔案總管打開就能看到。
@@ -177,7 +177,7 @@ ntu-cool-gcm
 下載到的目錄結構長這樣:
 
 ```
-C:\Users\你\ntu-cool-gcm_material\
+C:\Users\你\Documents\ntu-cool-gcm_material\
 └── 音樂、演化與大腦 Music, Evolution and the Brain (57544)\
     ├── course_overview.md        ← 這份課程的目錄索引(可以餵給 AI)
     ├── week1\
@@ -251,7 +251,19 @@ ntu-cool-gcm --skip-youtube --skip-cool-videos
 ntu-cool-gcm --out D:\我的課程
 ```
 
-預設是當前目錄底下的 `ntu-cool-gcm_material/`。
+預設是在你的「文件 / Documents」資料夾底下的 `ntu-cool-gcm_material/`。
+
+如果你的目前資料夾已經有舊版建立的 `ntu-cool-gcm_material/`,工具會沿用那個資料夾,避免把舊檔案拆散。
+
+### 下載 Word / PowerPoint / Excel / Zip 等非 PDF 檔
+
+預設只下載 PDF,避免把 Word、PowerPoint、Zip 之類的檔案誤存成 `.pdf`。
+
+如果老師有放這類檔案,請加:
+
+```powershell
+ntu-cool-gcm --all-file-types
+```
 
 ### 已經知道課程 ID(網址裡的數字)
 
@@ -348,9 +360,15 @@ ntu-cool-gcm --refresh-session
 
 ### Q: 我的檔案到底放在哪
 
-預設在你**執行 `ntu-cool-gcm` 時所在的目錄**底下,叫 `ntu-cool-gcm_material/`。
+預設在你的「文件 / Documents」資料夾底下,叫 `ntu-cool-gcm_material/`。
+
+常見位置像這樣:
+
+```text
+C:\Users\你\Documents\ntu-cool-gcm_material\
+```
 
-如果你在 PowerShell 看到提示是 `PS C:\Users\你>`,那就是 `C:\Users\你\ntu-cool-gcm_material\`。
+如果你以前已經在目前資料夾建立過 `ntu-cool-gcm_material/`,工具會優先沿用那個舊資料夾。
 
 每次跑完工具最後一行也會印「檔案存放位置:」+ 完整路徑。
 

{get_class_material-0.2.7 → get_class_material-0.2.8}/get_class_material.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: get-class-material
-Version: 0.2.7
+Version: 0.2.8
 Summary: One-command bulk downloader for NTU COOL (Canvas) course materials — PDFs, lecture videos, and Pages.
 Author-email: jabir <jabir95tsai@gmail.com>
 License-Expression: MIT
@@ -188,7 +188,7 @@ ntu-cool-gcm
   上課影片:   新增 4、跳過 0、失敗 0
 
 檔案存放位置:
-  C:\Users\你\ntu-cool-gcm_material\音樂、演化與大腦 Music, Evolution and the Brain (57544)
+  C:\Users\你\Documents\ntu-cool-gcm_material\音樂、演化與大腦 Music, Evolution and the Brain (57544)
 ```
 
 **最後那行就是你的檔案放在哪。** 用檔案總管打開就能看到。
@@ -209,7 +209,7 @@ ntu-cool-gcm
 下載到的目錄結構長這樣:
 
 ```
-C:\Users\你\ntu-cool-gcm_material\
+C:\Users\你\Documents\ntu-cool-gcm_material\
 └── 音樂、演化與大腦 Music, Evolution and the Brain (57544)\
     ├── course_overview.md        ← 這份課程的目錄索引(可以餵給 AI)
     ├── week1\
@@ -283,7 +283,19 @@ ntu-cool-gcm --skip-youtube --skip-cool-videos
 ntu-cool-gcm --out D:\我的課程
 ```
 
-預設是當前目錄底下的 `ntu-cool-gcm_material/`。
+預設是在你的「文件 / Documents」資料夾底下的 `ntu-cool-gcm_material/`。
+
+如果你的目前資料夾已經有舊版建立的 `ntu-cool-gcm_material/`,工具會沿用那個資料夾,避免把舊檔案拆散。
+
+### 下載 Word / PowerPoint / Excel / Zip 等非 PDF 檔
+
+預設只下載 PDF,避免把 Word、PowerPoint、Zip 之類的檔案誤存成 `.pdf`。
+
+如果老師有放這類檔案,請加:
+
+```powershell
+ntu-cool-gcm --all-file-types
+```
 
 ### 已經知道課程 ID(網址裡的數字)
 
@@ -380,9 +392,15 @@ ntu-cool-gcm --refresh-session
 
 ### Q: 我的檔案到底放在哪
 
-預設在你**執行 `ntu-cool-gcm` 時所在的目錄**底下,叫 `ntu-cool-gcm_material/`。
+預設在你的「文件 / Documents」資料夾底下,叫 `ntu-cool-gcm_material/`。
+
+常見位置像這樣:
+
+```text
+C:\Users\你\Documents\ntu-cool-gcm_material\
+```
 
-如果你在 PowerShell 看到提示是 `PS C:\Users\你>`,那就是 `C:\Users\你\ntu-cool-gcm_material\`。
+如果你以前已經在目前資料夾建立過 `ntu-cool-gcm_material/`,工具會優先沿用那個舊資料夾。
 
 每次跑完工具最後一行也會印「檔案存放位置:」+ 完整路徑。
 

{get_class_material-0.2.7 → get_class_material-0.2.8}/ntu_cool_materials/course_pipeline.py

@@ -304,6 +304,41 @@ _KNOWN_FILE_EXTS = {
 }
 
 
+def _file_item_real_ext(item: dict[str, Any]) -> str:
+    """Best-effort original extension for a Canvas File module item."""
+    title = str(item.get("title") or "").strip()
+    content_details = item.get("content_details") or {}
+    display_name = str(content_details.get("display_name") or "")
+    real_ext = Path(display_name).suffix.lower() if display_name else ""
+    if not real_ext:
+        real_ext = Path(title).suffix.lower()
+    return real_ext
+
+
+def _file_item_target_name(item: dict[str, Any], *, all_file_types: bool) -> str:
+    """Filename for a Canvas File item.
+
+    Default mode downloads every file but writes it to disk with a `.pdf`
+    extension regardless of the original type — most NTU course material IS
+    PDF, this keeps the layout uniform for downstream AI tools, and most
+    apps still open files whose extension is wrong (Word/Excel sniff the
+    binary signature). The trade-off: if a tool relies purely on extension
+    (e.g. zip extractors), the file won't open. Users who hit that pass
+    `--all-file-types` to keep real extensions like .pptx / .xlsx / .zip.
+
+    Always returns a name (never None). Default-mode behavior changed
+    from "skip non-PDF" → "force .pdf"; reverting required because the
+    skip-by-default surprised users who expected every file to land.
+    """
+    title = str(item.get("title") or "").strip() or f"item-{item.get('id')}"
+    real_ext = _file_item_real_ext(item)
+    use_ext = real_ext if all_file_types and real_ext else ".pdf"
+    title_ext = Path(title).suffix.lower()
+    stem = title[:-len(title_ext)] if title_ext in _KNOWN_FILE_EXTS else title
+    safe_title = sanitize_teacher_title(stem)
+    return f"{safe_title}{use_ext}"
+
+
 def download_files(
     plan: CoursePlan, client: CanvasSessionClient, *, all_file_types: bool = False,
 ) -> StageStats:
@@ -329,31 +364,11 @@ def download_files(
                 continue
             file_id = str(item.get("content_id") or "")
             title = str(item.get("title") or "").strip() or f"item-{item.get('id')}"
-
-            # Real extension for this file: prefer Canvas's display_name in
-            # content_details, fall back to the title's suffix.
-            content_details = item.get("content_details") or {}
-            display_name = str(content_details.get("display_name") or "")
-            real_ext = Path(display_name).suffix.lower() if display_name else ""
-            if not real_ext:
-                real_ext = Path(title).suffix.lower()
-
-            # Decide what extension to use ON DISK.
-            if all_file_types:
-                use_ext = real_ext or ".pdf"
-            else:
-                use_ext = ".pdf"
-                if real_ext and real_ext != ".pdf":
-                    forced_pdf_count += 1
-
-            # Strip any known extension from the title (case-insensitive) so we
-            # don't double up: "syllabus.pdf" + ".pdf" = "syllabus.pdf", not
-            # "syllabus.pdf.pdf"; "MS-02_C03-Ex.xlsx" + ".pdf" = "MS-02_C03-Ex.pdf",
-            # not "MS-02_C03-Ex.xlsx.pdf".
-            title_ext = Path(title).suffix.lower()
-            stem = title[:-len(title_ext)] if title_ext in _KNOWN_FILE_EXTS else title
-            safe_title = sanitize_teacher_title(stem)
-            target = week.week_dir / f"{safe_title}{use_ext}"
+            real_ext = _file_item_real_ext(item)
+            target_name = _file_item_target_name(item, all_file_types=all_file_types)
+            target = week.week_dir / target_name
+            if not all_file_types and real_ext and real_ext != ".pdf":
+                forced_pdf_count += 1
             if target.exists():
                 stats.skipped += 1
                 continue
@@ -451,6 +466,91 @@ def _delete_orphan_format_fragments(videos_dir: Path) -> int:
     return n
 
 
+def _count_youtube_urls_in_plan(plan: CoursePlan) -> int:
+    """Distinct YouTube video IDs across the whole plan.
+
+    Used to size the "found N YouTube videos, want to log in?" prompt.
+    Distinct (not raw item count) because the same video may appear in
+    multiple modules and we don't want to overstate the work."""
+    seen: set[str] = set()
+    for week in plan.weeks:
+        for item in week.items:
+            if item.get("type") not in {"ExternalUrl", "ExternalTool"}:
+                continue
+            raw = str(item.get("external_url") or item.get("url") or "")
+            for vid in extract_youtube_ids(raw):
+                seen.add(vid)
+    return len(seen)
+
+
+def maybe_capture_youtube_cookies(
+    plan: CoursePlan, cookies_path: Path,
+    *, interactive: bool = True, input_fn=None,
+) -> bool:
+    """If the plan has YouTube items but no cookies file, offer a one-shot
+    browser login that writes cookies into `cookies_path`.
+
+    Returns True if cookies are now in place (either already-existed or just
+    captured), False if user declined or capture failed. Caller can use this
+    to decide whether to print the "unlisted videos may fail" warning.
+
+    No-op when:
+      - cookies file already exists (don't re-prompt every run)
+      - stdin isn't a TTY (piped scripts, CI, etc.)
+      - the plan has no YouTube items at all
+      - interactive=False (callers that want to suppress the prompt entirely)
+
+    The browser flow is delegated to `youtube_cookies.export_youtube_cookies`,
+    which is also exposed as the standalone `youtube-cookies` CLI. Same code
+    path, same default profile dir — so cookies captured here are reused by
+    the standalone command and vice-versa.
+    """
+    if cookies_path.exists():
+        return True
+    if not interactive:
+        return False
+    if input_fn is None:
+        if not sys.stdin.isatty():
+            return False
+        input_fn = input
+    url_count = _count_youtube_urls_in_plan(plan)
+    if url_count == 0:
+        return False
+
+    prompt = t(
+        f"  ❓ 找到 {url_count} 個 YouTube 影片連結。\n"
+        f"     公開影片不用登入。要不要先登入 YouTube,以下載私人 / 限齡 / 會員影片? [y/N]: ",
+        f"  ❓ Found {url_count} YouTube link(s).\n"
+        f"     Public videos don't need login. Log in to YouTube to enable "
+        f"private/age-restricted/member videos? [y/N]: ",
+    )
+    try:
+        answer = input_fn(prompt).strip().lower()
+    except (EOFError, KeyboardInterrupt):
+        return False
+    if answer not in {"y", "yes"}:
+        return False
+
+    try:
+        from .youtube_cookies import export_youtube_cookies
+        print(t(
+            "  → 開啟瀏覽器登入 YouTube(登入後會自動匯出 cookies)...",
+            "  → opening browser for YouTube login (cookies export on detection)...",
+        ))
+        result = export_youtube_cookies(cookies_path=cookies_path, wait_for_login=True)
+        print(t(
+            f"  ✓ 匯出 {result.cookie_count} 個 cookies → {cookies_path}",
+            f"  ✓ exported {result.cookie_count} cookies → {cookies_path}",
+        ))
+        return True
+    except Exception as exc:
+        print(t(
+            f"  ⚠ 抓 YouTube cookies 失敗 ({exc})。改用公開模式繼續。",
+            f"  ⚠ YouTube cookie capture failed ({exc}). Proceeding in public mode.",
+        ))
+        return False
+
+
 def download_youtube(plan: CoursePlan, *, cookies_path: Path, yt_dlp: str = "yt-dlp") -> StageStats:
     """For each week with YouTube items, run yt-dlp into the week dir then rename."""
     import platform
@@ -524,13 +624,14 @@ def download_youtube(plan: CoursePlan, *, cookies_path: Path, yt_dlp: str = "yt-
         print(f"  [{week.label}/youtube] 開始下載 {len(urls)} 個 URL (缺少 {len(missing)} 個)")
         cmd = [
             yt_dlp, *YT_DLP_BASE_ARGS,
-            "--cookies", str(cookies_path),
             # No --newline: let yt-dlp use \r so the progress bar refreshes
             # in place on a single line instead of spamming the terminal.
             "-P", str(videos_dir),
             "-o", "%(id)s_%(title).120s.%(ext)s",
             "-a", str(urls_file),
         ]
+        if cookies_path.exists():
+            cmd[1:1] = ["--cookies", str(cookies_path)]
         try:
             result = subprocess.run(cmd)
         finally:
@@ -804,7 +905,7 @@ def _build_session_client_from_file(headers_path: Path, base_url: str) -> Canvas
     return CanvasSessionClient(base_url=base_url, headers=read_headers_file(headers_path))
 
 
-def _write_course_overview(plan: CoursePlan) -> Path:
+def _write_course_overview(plan: CoursePlan, *, all_file_types: bool = False) -> Path:
     """Write a Markdown index at the course root listing every downloaded artifact per week.
     Designed to be readable by both humans and AI tools."""
     from datetime import datetime, timezone
@@ -829,10 +930,15 @@ def _write_course_overview(plan: CoursePlan) -> Path:
             kind = item.get("type")
             title = str(item.get("title") or "").strip()
             if kind == "File":
-                stem = title[:-4] if title.lower().endswith(".pdf") else title
-                fname = f"{sanitize_teacher_title(stem)}.pdf"
+                fname = _file_item_target_name(item, all_file_types=all_file_types)
                 if (week.week_dir / fname).exists():
-                    per_type["pdf"].append(f"- 📄 [{title}]({week.label}/{urllib.parse.quote(fname)})")
+                    # In default mode every file is .pdf on disk, so the dual
+                    # icon only matters under --all-file-types where extensions
+                    # are preserved. Either way we route by what's actually
+                    # on disk, not by what the source extension said.
+                    icon = "📄" if Path(fname).suffix.lower() == ".pdf" else "📎"
+                    bucket = "pdf" if Path(fname).suffix.lower() == ".pdf" else "other"
+                    per_type[bucket].append(f"- {icon} [{title}]({week.label}/{urllib.parse.quote(fname)})")
             elif kind == "Page":
                 fname = f"{sanitize_teacher_title(title)}.md"
                 if (week.week_dir / fname).exists():
@@ -966,13 +1072,19 @@ def download_course(
             ))
         if not skip_youtube:
             print(t("\n[3/4] YouTube 影片", "\n[3/4] YouTube videos"))
-            if yt_cookies is None or not yt_cookies.exists():
+            yt_cookies_path = yt_cookies or Path(".secrets/youtube_cookies.txt")
+            # Offer in-pipeline cookie capture if there are YouTube items but
+            # no cookies on disk yet. User can decline; we proceed in public
+            # mode either way. Skipped when stdin isn't a TTY (so this never
+            # blocks scripted / CI invocations).
+            have_cookies = maybe_capture_youtube_cookies(plan, yt_cookies_path)
+            if not have_cookies:
                 print(t(
-                    f"  注意: 找不到 YouTube cookies ({yt_cookies}),不公開影片可能下載失敗",
-                    f"  WARNING: youtube cookies not found at {yt_cookies} — unlisted videos may fail",
+                    f"  注意: 沒有 YouTube cookies,不公開 / 限齡 / 會員影片可能下載失敗",
+                    f"  Note: no YouTube cookies — private / age-restricted / member videos may fail",
                 ))
             course_stats.youtube = download_youtube(
-                plan, cookies_path=yt_cookies or Path(".secrets/youtube_cookies.txt"), yt_dlp=yt_dlp
+                plan, cookies_path=yt_cookies_path, yt_dlp=yt_dlp
             )
 
         if not skip_cool_videos:
@@ -990,7 +1102,7 @@ def download_course(
 
         # Per-course overview at the course root.
         try:
-            overview_path = _write_course_overview(plan)
+            overview_path = _write_course_overview(plan, all_file_types=all_file_types)
             print(t(f"\n  目錄: {overview_path.name}", f"\n  overview: {overview_path.name}"))
         except Exception as exc:
             print(t(f"\n  (無法產生目錄: {exc})", f"\n  (could not write overview: {exc})"))

{get_class_material-0.2.7 → get_class_material-0.2.8}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "get-class-material"
-version = "0.2.7"
+version = "0.2.8"
 description = "One-command bulk downloader for NTU COOL (Canvas) course materials — PDFs, lecture videos, and Pages."
 readme = "README.md"
 license = "MIT"

get_class_material-0.2.8/tests/test_course_pipeline.py

@@ -0,0 +1,484 @@
+"""Tests for course_pipeline._ensure_logged_in.
+
+These mock the Playwright Page object entirely — no actual browser is
+launched. The function under test is a thin policy layer over Playwright,
+so what matters is that we route correctly for each combination of
+(goto outcome, current URL) we might see during a real SSO redirect.
+
+Regression covered: an earlier version used `wait_until="domcontentloaded"`
+with a 120s timeout; NTU's SAML chain plus a slow user typing credentials
+sometimes never fires DOM-ready inside 120s, and the function crashed
+instead of falling through to wait_for_url's 600s budget.
+"""
+from __future__ import annotations
+
+import contextlib
+import io
+import tempfile
+import unittest
+from pathlib import Path
+from unittest import mock
+
+from playwright.sync_api import TimeoutError as PWTimeout
+
+from ntu_cool_materials import course_pipeline
+
+
+def _make_page(*, url: str, goto_raises: Exception | None = None,
+               wait_for_url_raises: Exception | None = None,
+               url_after_wait: str | None = None):
+    """Build a mock Page with the URL state at each step of the flow.
+
+    `url` is what page.url reports when first read (after goto returns).
+    `url_after_wait` is what page.url reports after wait_for_url returns —
+    simulates the user finishing SSO. Defaults to keeping the same URL.
+    """
+    page = mock.MagicMock()
+
+    # page.url is a property that can change over time. Use PropertyMock so
+    # we can swap the underlying value mid-test.
+    url_state = {"value": url}
+
+    def get_url():
+        return url_state["value"]
+
+    type(page).url = mock.PropertyMock(side_effect=get_url)
+
+    if goto_raises is not None:
+        page.goto.side_effect = goto_raises
+    else:
+        page.goto.return_value = None
+
+    def _wait_for_url(*args, **kwargs):
+        if wait_for_url_raises is not None:
+            raise wait_for_url_raises
+        if url_after_wait is not None:
+            url_state["value"] = url_after_wait
+
+    page.wait_for_url.side_effect = _wait_for_url
+    return page
+
+
+class EnsureLoggedInTests(unittest.TestCase):
+    def setUp(self) -> None:
+        # `_ensure_logged_in` print()s status updates in Traditional Chinese.
+        # The CLI normally calls _force_utf8_streams() to reconfigure stdout
+        # before any such print, but unit tests bypass that path — and the
+        # GitHub Windows runner ships with cp1252 stdout, which can't encode
+        # CJK characters. Redirect stdout to a StringIO so the prints happen
+        # against an in-memory UTF-8-capable stream. (No assertion looks at
+        # captured output; we only care about the function's return value
+        # and which Playwright methods it called.)
+        self._stdout_ctx = contextlib.redirect_stdout(io.StringIO())
+        self._stdout_ctx.__enter__()
+        self.addCleanup(self._stdout_ctx.__exit__, None, None, None)
+
+    def test_already_logged_in_returns_true(self) -> None:
+        """Happy path: cookies are fresh, goto lands directly on /courses."""
+        page = _make_page(url="https://cool.ntu.edu.tw/courses")
+        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
+        self.assertTrue(ok)
+        page.goto.assert_called_once()
+        page.wait_for_url.assert_not_called()
+
+    def test_redirects_to_sso_then_user_logs_in(self) -> None:
+        """Stale cookies: goto lands on SSO, wait_for_url returns once the user
+        finishes typing creds. Function returns True."""
+        page = _make_page(
+            url="https://web2.cc.ntu.edu.tw/saml/sso?...",
+            url_after_wait="https://cool.ntu.edu.tw/courses",
+        )
+        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
+        self.assertTrue(ok)
+        page.wait_for_url.assert_called_once()
+
+    def test_goto_timeout_but_already_on_sso_still_proceeds(self) -> None:
+        """THE REGRESSION GUARD: goto raised TimeoutError because DOM never
+        fired inside its short budget, but the navigation DID commit to the
+        SSO page. wait_for_url should still run and succeed."""
+        page = _make_page(
+            url="https://web2.cc.ntu.edu.tw/saml/sso?...",
+            goto_raises=PWTimeout("Page.goto: Timeout 60000ms exceeded."),
+            url_after_wait="https://cool.ntu.edu.tw/courses",
+        )
+        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
+        self.assertTrue(ok, "must fall through to wait_for_url instead of bubbling TimeoutError")
+        page.wait_for_url.assert_called_once()
+
+    def test_goto_timeout_and_page_never_committed_returns_false(self) -> None:
+        """Real network failure: goto timed out AND the page is still on
+        about:blank (didn't commit any navigation). Bail with a clear
+        error rather than waiting 10 minutes on a dead URL."""
+        page = _make_page(
+            url="about:blank",
+            goto_raises=PWTimeout("Page.goto: Timeout 60000ms exceeded."),
+        )
+        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
+        self.assertFalse(ok)
+
+    def test_sso_login_actually_times_out(self) -> None:
+        """User opens the browser but never finishes SSO. wait_for_url
+        timeout should propagate as a False return, not crash."""
+        page = _make_page(
+            url="https://web2.cc.ntu.edu.tw/saml/sso?...",
+            wait_for_url_raises=PWTimeout("Page.wait_for_url: Timeout exceeded."),
+        )
+        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
+        self.assertFalse(ok)
+
+    def test_uses_course_specific_url_when_course_id_given(self) -> None:
+        """When called from the bulk download pipeline with a specific course,
+        we should navigate to /courses/<id> rather than /courses."""
+        page = _make_page(url="https://cool.ntu.edu.tw/courses/60804")
+        course_pipeline._ensure_logged_in(page, course_id="60804", sso_timeout_sec=600)
+        called_url = page.goto.call_args.args[0]
+        self.assertIn("/courses/60804", called_url)
+
+    def test_uses_commit_wait_until_not_domcontentloaded(self) -> None:
+        """Explicit regression guard against reverting to domcontentloaded.
+        The whole point of this fix is that we DON'T wait for DOM ready,
+        because SAML's redirect chain can take longer than any reasonable
+        timeout to settle into a stable DOM."""
+        page = _make_page(url="https://cool.ntu.edu.tw/courses")
+        course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
+        self.assertEqual(page.goto.call_args.kwargs.get("wait_until"), "commit")
+
+
+class CourseFileHandlingTests(unittest.TestCase):
+    def setUp(self) -> None:
+        self._stdout_ctx = contextlib.redirect_stdout(io.StringIO())
+        self._stdout_ctx.__enter__()
+        self.addCleanup(self._stdout_ctx.__exit__, None, None, None)
+
+    def test_non_pdf_file_keeps_pdf_extension_by_default(self) -> None:
+        """Default mode forces every download to land as .pdf on disk so the
+        output folder looks uniform for downstream AI tools. --all-file-types
+        opts into the real extension (.pptx in this case)."""
+        item = {
+            "id": 1,
+            "type": "File",
+            "title": "week1 slides.pptx",
+            "content_details": {"display_name": "week1 slides.pptx"},
+        }
+
+        self.assertEqual(
+            course_pipeline._file_item_target_name(item, all_file_types=False),
+            "week1 slides.pdf",
+        )
+        self.assertEqual(
+            course_pipeline._file_item_target_name(item, all_file_types=True),
+            "week1 slides.pptx",
+        )
+
+    def test_pdf_file_unchanged_in_both_modes(self) -> None:
+        """A real PDF should produce the same filename regardless of mode —
+        guards against the .pdf.pdf double-extension regression."""
+        item = {
+            "id": 2,
+            "type": "File",
+            "title": "syllabus.pdf",
+            "content_details": {"display_name": "syllabus.pdf"},
+        }
+        self.assertEqual(
+            course_pipeline._file_item_target_name(item, all_file_types=False),
+            "syllabus.pdf",
+        )
+        self.assertEqual(
+            course_pipeline._file_item_target_name(item, all_file_types=True),
+            "syllabus.pdf",
+        )
+
+    def test_youtube_command_omits_missing_cookie_file(self) -> None:
+        with tempfile.TemporaryDirectory() as temp:
+            root = Path(temp)
+            plan = course_pipeline.CoursePlan(
+                course={"id": "1", "name": "Course"},
+                course_id="1",
+                course_dir=root,
+                weeks=[
+                    course_pipeline.WeekPlan(
+                        label="week1",
+                        module={
+                            "items": [
+                                {
+                                    "id": 10,
+                                    "type": "ExternalUrl",
+                                    "title": "1-1 video",
+                                    "external_url": "https://www.youtube.com/watch?v=BT4w_3QsrQ8",
+                                }
+                            ]
+                        },
+                        week_dir=root / "week1",
+                    )
+                ],
+            )
+
+            run_result = mock.Mock(returncode=0)
+            with (
+                mock.patch.object(course_pipeline.shutil, "which", return_value="tool"),
+                mock.patch.object(course_pipeline.subprocess, "run", return_value=run_result) as run,
+                mock.patch.object(course_pipeline, "rename_downloaded_videos", return_value=[]),
+            ):
+                course_pipeline.download_youtube(
+                    plan,
+                    cookies_path=root / "missing-youtube-cookies.txt",
+                    yt_dlp="yt-dlp",
+                )
+
+            cmd = run.call_args.args[0]
+            self.assertNotIn("--cookies", cmd)
+
+    def test_youtube_command_uses_cookie_file_when_present(self) -> None:
+        with tempfile.TemporaryDirectory() as temp:
+            root = Path(temp)
+            cookies = root / "youtube-cookies.txt"
+            cookies.write_text("# Netscape HTTP Cookie File\n", encoding="utf-8")
+            plan = course_pipeline.CoursePlan(
+                course={"id": "1", "name": "Course"},
+                course_id="1",
+                course_dir=root,
+                weeks=[
+                    course_pipeline.WeekPlan(
+                        label="week1",
+                        module={
+                            "items": [
+                                {
+                                    "id": 10,
+                                    "type": "ExternalUrl",
+                                    "title": "1-1 video",
+                                    "external_url": "https://www.youtube.com/watch?v=BT4w_3QsrQ8",
+                                }
+                            ]
+                        },
+                        week_dir=root / "week1",
+                    )
+                ],
+            )
+
+            run_result = mock.Mock(returncode=0)
+            with (
+                mock.patch.object(course_pipeline.shutil, "which", return_value="tool"),
+                mock.patch.object(course_pipeline.subprocess, "run", return_value=run_result) as run,
+                mock.patch.object(course_pipeline, "rename_downloaded_videos", return_value=[]),
+            ):
+                course_pipeline.download_youtube(plan, cookies_path=cookies, yt_dlp="yt-dlp")
+
+            cmd = run.call_args.args[0]
+            self.assertIn("--cookies", cmd)
+            self.assertEqual(cmd[cmd.index("--cookies") + 1], str(cookies))
+
+    def test_course_overview_uses_real_extension_for_all_file_types(self) -> None:
+        with tempfile.TemporaryDirectory() as temp:
+            root = Path(temp)
+            week_dir = root / "week1"
+            week_dir.mkdir()
+            (week_dir / "week1 slides.pptx").write_bytes(b"pptx")
+            plan = course_pipeline.CoursePlan(
+                course={"id": "1", "name": "Course"},
+                course_id="1",
+                course_dir=root,
+                weeks=[
+                    course_pipeline.WeekPlan(
+                        label="week1",
+                        module={
+                            "name": "Week 1",
+                            "items": [
+                                {
+                                    "id": 1,
+                                    "type": "File",
+                                    "title": "week1 slides.pptx",
+                                    "content_details": {"display_name": "week1 slides.pptx"},
+                                }
+                            ],
+                        },
+                        week_dir=week_dir,
+                    )
+                ],
+            )
+
+            overview = course_pipeline._write_course_overview(plan, all_file_types=True)
+            text = overview.read_text(encoding="utf-8")
+            self.assertIn("week1%20slides.pptx", text)
+            self.assertNotIn("week1%20slides.pdf", text)
+
+
+class MaybeCaptureYoutubeCookiesTests(unittest.TestCase):
+    """Tests for the in-pipeline YouTube cookie capture prompt.
+
+    The function should be conservative: never prompt when cookies already
+    exist, never prompt for non-YouTube courses, never block CI/piped runs.
+    """
+
+    def setUp(self) -> None:
+        self._stdout_ctx = contextlib.redirect_stdout(io.StringIO())
+        self._stdout_ctx.__enter__()
+        self.addCleanup(self._stdout_ctx.__exit__, None, None, None)
+
+    def _make_plan_with_youtube(self, urls: list[str]):
+        return course_pipeline.CoursePlan(
+            course={"id": "1", "name": "Course"},
+            course_id="1",
+            course_dir=Path("."),
+            weeks=[
+                course_pipeline.WeekPlan(
+                    label="week1",
+                    module={
+                        "items": [
+                            {"id": i, "type": "ExternalUrl", "title": f"vid{i}", "external_url": u}
+                            for i, u in enumerate(urls, start=1)
+                        ]
+                    },
+                    week_dir=Path("./week1"),
+                )
+            ],
+        )
+
+    def test_returns_true_when_cookies_already_exist(self) -> None:
+        """Existing cookies = no prompt, no capture. Caller knows it's ok."""
+        with tempfile.TemporaryDirectory() as temp:
+            cookies = Path(temp) / "yt.txt"
+            cookies.write_text("# Netscape HTTP Cookie File\n", encoding="utf-8")
+            plan = self._make_plan_with_youtube(["https://youtu.be/dQw4w9WgXcQ"])
+            calls = []
+            result = course_pipeline.maybe_capture_youtube_cookies(
+                plan, cookies, input_fn=lambda p: calls.append(p) or "y",
+            )
+        self.assertTrue(result)
+        self.assertEqual(calls, [], "must not prompt when cookies file already present")
+
+    def test_no_prompt_when_plan_has_no_youtube(self) -> None:
+        """Course with only PDFs / pages / cool-videos should not get bothered."""
+        plan = course_pipeline.CoursePlan(
+            course={"id": "1", "name": "Course"}, course_id="1", course_dir=Path("."),
+            weeks=[course_pipeline.WeekPlan(label="w1", module={"items": []}, week_dir=Path("./w1"))],
+        )
+        with tempfile.TemporaryDirectory() as temp:
+            cookies = Path(temp) / "yt.txt"  # doesn't exist
+            calls = []
+            result = course_pipeline.maybe_capture_youtube_cookies(
+                plan, cookies, input_fn=lambda p: calls.append(p) or "y",
+            )
+        self.assertFalse(result)
+        self.assertEqual(calls, [])
+
+    def test_skipped_when_interactive_false(self) -> None:
+        """interactive=False is the escape hatch for batch / CI callers."""
+        plan = self._make_plan_with_youtube(["https://youtu.be/dQw4w9WgXcQ"])
+        with tempfile.TemporaryDirectory() as temp:
+            cookies = Path(temp) / "yt.txt"
+            calls = []
+            result = course_pipeline.maybe_capture_youtube_cookies(
+                plan, cookies, interactive=False, input_fn=lambda p: calls.append(p) or "y",
+            )
+        self.assertFalse(result)
+        self.assertEqual(calls, [])
+
+    def test_user_declines_returns_false(self) -> None:
+        """User presses Enter / N: no capture, function returns False so
+        caller prints the 'no cookies' warning."""
+        plan = self._make_plan_with_youtube(["https://youtu.be/dQw4w9WgXcQ"])
+        with tempfile.TemporaryDirectory() as temp:
+            cookies = Path(temp) / "yt.txt"
+            result = course_pipeline.maybe_capture_youtube_cookies(
+                plan, cookies, input_fn=lambda p: "n",
+            )
+        self.assertFalse(result)
+        self.assertFalse(cookies.exists())
+
+    def test_user_accepts_runs_export(self) -> None:
+        """User says 'y': we delegate to youtube_cookies.export_youtube_cookies
+        with the right cookies_path. We mock that function to avoid spawning
+        a real browser; what we verify is the wiring."""
+        plan = self._make_plan_with_youtube(["https://youtu.be/dQw4w9WgXcQ", "https://youtu.be/oHg5SJYRHA0"])
+        with tempfile.TemporaryDirectory() as temp:
+            cookies = Path(temp) / "yt.txt"
+
+            captured_kwargs = {}
+
+            def fake_export(**kwargs):
+                captured_kwargs.update(kwargs)
+                # Simulate the real function's side effect: write the cookies file.
+                kwargs["cookies_path"].write_text("# fake cookies\n", encoding="utf-8")
+                return mock.Mock(cookies_path=kwargs["cookies_path"], cookie_count=42, current_url="x")
+
+            with mock.patch("ntu_cool_materials.youtube_cookies.export_youtube_cookies",
+                            side_effect=fake_export):
+                result = course_pipeline.maybe_capture_youtube_cookies(
+                    plan, cookies, input_fn=lambda p: "y",
+                )
+        self.assertTrue(result)
+        self.assertEqual(captured_kwargs.get("cookies_path"), cookies)
+        self.assertTrue(captured_kwargs.get("wait_for_login"))
+
+    def test_eof_during_prompt_returns_false(self) -> None:
+        """User Ctrl-D'd through the prompt. Don't crash."""
+        plan = self._make_plan_with_youtube(["https://youtu.be/dQw4w9WgXcQ"])
+        def raising_input(_prompt):
+            raise EOFError
+        with tempfile.TemporaryDirectory() as temp:
+            cookies = Path(temp) / "yt.txt"
+            result = course_pipeline.maybe_capture_youtube_cookies(
+                plan, cookies, input_fn=raising_input,
+            )
+        self.assertFalse(result)
+
+    def test_export_failure_swallowed_returns_false(self) -> None:
+        """If the browser flow blows up, we warn and proceed in public mode
+        instead of crashing the whole download_course run."""
+        plan = self._make_plan_with_youtube(["https://youtu.be/dQw4w9WgXcQ"])
+        with tempfile.TemporaryDirectory() as temp:
+            cookies = Path(temp) / "yt.txt"
+            with mock.patch("ntu_cool_materials.youtube_cookies.export_youtube_cookies",
+                            side_effect=RuntimeError("playwright went boom")):
+                result = course_pipeline.maybe_capture_youtube_cookies(
+                    plan, cookies, input_fn=lambda p: "y",
+                )
+        self.assertFalse(result)
+        self.assertFalse(cookies.exists())
+
+
+class CountYoutubeUrlsInPlanTests(unittest.TestCase):
+    def test_counts_distinct_video_ids(self) -> None:
+        """Same video appearing twice should count once.
+
+        Note: YouTube IDs are exactly 11 chars of [A-Za-z0-9_-] — short fake
+        IDs like 'abc' won't match extract_youtube_ids' regex. Use full-length
+        fixtures so the test exercises the real code path."""
+        VID_A = "dQw4w9WgXcQ"  # 11 chars
+        VID_B = "oHg5SJYRHA0"  # 11 chars, distinct
+        plan = course_pipeline.CoursePlan(
+            course={"id": "1", "name": "Course"}, course_id="1", course_dir=Path("."),
+            weeks=[
+                course_pipeline.WeekPlan(
+                    label="w1",
+                    module={"items": [
+                        {"id": 1, "type": "ExternalUrl", "title": "v1", "external_url": f"https://youtu.be/{VID_A}"},
+                        # Same video via different URL form — should dedupe.
+                        {"id": 2, "type": "ExternalUrl", "title": "v2", "external_url": f"https://www.youtube.com/watch?v={VID_A}"},
+                        {"id": 3, "type": "ExternalUrl", "title": "v3", "external_url": f"https://youtu.be/{VID_B}"},
+                    ]},
+                    week_dir=Path("./w1"),
+                )
+            ],
+        )
+        self.assertEqual(course_pipeline._count_youtube_urls_in_plan(plan), 2)
+
+    def test_ignores_non_youtube_external_urls(self) -> None:
+        plan = course_pipeline.CoursePlan(
+            course={"id": "1", "name": "Course"}, course_id="1", course_dir=Path("."),
+            weeks=[
+                course_pipeline.WeekPlan(
+                    label="w1",
+                    module={"items": [
+                        {"id": 1, "type": "ExternalUrl", "title": "doc", "external_url": "https://drive.google.com/file/d/xyz"},
+                        {"id": 2, "type": "File", "title": "slides.pdf"},
+                    ]},
+                    week_dir=Path("./w1"),
+                )
+            ],
+        )
+        self.assertEqual(course_pipeline._count_youtube_urls_in_plan(plan), 0)
+
+
+if __name__ == "__main__":
+    unittest.main()

get_class_material-0.2.7/tests/test_course_pipeline.py

@@ -1,146 +0,0 @@
-"""Tests for course_pipeline._ensure_logged_in.
-
-These mock the Playwright Page object entirely — no actual browser is
-launched. The function under test is a thin policy layer over Playwright,
-so what matters is that we route correctly for each combination of
-(goto outcome, current URL) we might see during a real SSO redirect.
-
-Regression covered: an earlier version used `wait_until="domcontentloaded"`
-with a 120s timeout; NTU's SAML chain plus a slow user typing credentials
-sometimes never fires DOM-ready inside 120s, and the function crashed
-instead of falling through to wait_for_url's 600s budget.
-"""
-from __future__ import annotations
-
-import contextlib
-import io
-import unittest
-from unittest import mock
-
-from playwright.sync_api import TimeoutError as PWTimeout
-
-from ntu_cool_materials import course_pipeline
-
-
-def _make_page(*, url: str, goto_raises: Exception | None = None,
-               wait_for_url_raises: Exception | None = None,
-               url_after_wait: str | None = None):
-    """Build a mock Page with the URL state at each step of the flow.
-
-    `url` is what page.url reports when first read (after goto returns).
-    `url_after_wait` is what page.url reports after wait_for_url returns —
-    simulates the user finishing SSO. Defaults to keeping the same URL.
-    """
-    page = mock.MagicMock()
-
-    # page.url is a property that can change over time. Use PropertyMock so
-    # we can swap the underlying value mid-test.
-    url_state = {"value": url}
-
-    def get_url():
-        return url_state["value"]
-
-    type(page).url = mock.PropertyMock(side_effect=get_url)
-
-    if goto_raises is not None:
-        page.goto.side_effect = goto_raises
-    else:
-        page.goto.return_value = None
-
-    def _wait_for_url(*args, **kwargs):
-        if wait_for_url_raises is not None:
-            raise wait_for_url_raises
-        if url_after_wait is not None:
-            url_state["value"] = url_after_wait
-
-    page.wait_for_url.side_effect = _wait_for_url
-    return page
-
-
-class EnsureLoggedInTests(unittest.TestCase):
-    def setUp(self) -> None:
-        # `_ensure_logged_in` print()s status updates in Traditional Chinese.
-        # The CLI normally calls _force_utf8_streams() to reconfigure stdout
-        # before any such print, but unit tests bypass that path — and the
-        # GitHub Windows runner ships with cp1252 stdout, which can't encode
-        # CJK characters. Redirect stdout to a StringIO so the prints happen
-        # against an in-memory UTF-8-capable stream. (No assertion looks at
-        # captured output; we only care about the function's return value
-        # and which Playwright methods it called.)
-        self._stdout_ctx = contextlib.redirect_stdout(io.StringIO())
-        self._stdout_ctx.__enter__()
-        self.addCleanup(self._stdout_ctx.__exit__, None, None, None)
-
-    def test_already_logged_in_returns_true(self) -> None:
-        """Happy path: cookies are fresh, goto lands directly on /courses."""
-        page = _make_page(url="https://cool.ntu.edu.tw/courses")
-        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
-        self.assertTrue(ok)
-        page.goto.assert_called_once()
-        page.wait_for_url.assert_not_called()
-
-    def test_redirects_to_sso_then_user_logs_in(self) -> None:
-        """Stale cookies: goto lands on SSO, wait_for_url returns once the user
-        finishes typing creds. Function returns True."""
-        page = _make_page(
-            url="https://web2.cc.ntu.edu.tw/saml/sso?...",
-            url_after_wait="https://cool.ntu.edu.tw/courses",
-        )
-        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
-        self.assertTrue(ok)
-        page.wait_for_url.assert_called_once()
-
-    def test_goto_timeout_but_already_on_sso_still_proceeds(self) -> None:
-        """THE REGRESSION GUARD: goto raised TimeoutError because DOM never
-        fired inside its short budget, but the navigation DID commit to the
-        SSO page. wait_for_url should still run and succeed."""
-        page = _make_page(
-            url="https://web2.cc.ntu.edu.tw/saml/sso?...",
-            goto_raises=PWTimeout("Page.goto: Timeout 60000ms exceeded."),
-            url_after_wait="https://cool.ntu.edu.tw/courses",
-        )
-        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
-        self.assertTrue(ok, "must fall through to wait_for_url instead of bubbling TimeoutError")
-        page.wait_for_url.assert_called_once()
-
-    def test_goto_timeout_and_page_never_committed_returns_false(self) -> None:
-        """Real network failure: goto timed out AND the page is still on
-        about:blank (didn't commit any navigation). Bail with a clear
-        error rather than waiting 10 minutes on a dead URL."""
-        page = _make_page(
-            url="about:blank",
-            goto_raises=PWTimeout("Page.goto: Timeout 60000ms exceeded."),
-        )
-        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
-        self.assertFalse(ok)
-
-    def test_sso_login_actually_times_out(self) -> None:
-        """User opens the browser but never finishes SSO. wait_for_url
-        timeout should propagate as a False return, not crash."""
-        page = _make_page(
-            url="https://web2.cc.ntu.edu.tw/saml/sso?...",
-            wait_for_url_raises=PWTimeout("Page.wait_for_url: Timeout exceeded."),
-        )
-        ok = course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
-        self.assertFalse(ok)
-
-    def test_uses_course_specific_url_when_course_id_given(self) -> None:
-        """When called from the bulk download pipeline with a specific course,
-        we should navigate to /courses/<id> rather than /courses."""
-        page = _make_page(url="https://cool.ntu.edu.tw/courses/60804")
-        course_pipeline._ensure_logged_in(page, course_id="60804", sso_timeout_sec=600)
-        called_url = page.goto.call_args.args[0]
-        self.assertIn("/courses/60804", called_url)
-
-    def test_uses_commit_wait_until_not_domcontentloaded(self) -> None:
-        """Explicit regression guard against reverting to domcontentloaded.
-        The whole point of this fix is that we DON'T wait for DOM ready,
-        because SAML's redirect chain can take longer than any reasonable
-        timeout to settle into a stable DOM."""
-        page = _make_page(url="https://cool.ntu.edu.tw/courses")
-        course_pipeline._ensure_logged_in(page, course_id=None, sso_timeout_sec=600)
-        self.assertEqual(page.goto.call_args.kwargs.get("wait_until"), "commit")
-
-
-if __name__ == "__main__":
-    unittest.main()