@caddis/cli 0.1.0 → 0.2.0

package/bundles/antigravity-plugin-extras/skills/notebooklm/references/usage_patterns.md

@@ -0,0 +1,338 @@
+# NotebookLM Skill Usage Patterns
+
+Advanced patterns for using the NotebookLM skill effectively.
+
+## Critical: Always Use run.py
+
+**Every command must use the run.py wrapper:**
+```bash
+# ✅ CORRECT:
+python scripts/run.py auth_manager.py status
+python scripts/run.py ask_question.py --question "..."
+
+# ❌ WRONG:
+python scripts/auth_manager.py status  # Will fail!
+```
+
+## Pattern 1: Initial Setup
+
+```bash
+# 1. Check authentication (using run.py!)
+python scripts/run.py auth_manager.py status
+
+# 2. If not authenticated, setup (Browser MUST be visible!)
+python scripts/run.py auth_manager.py setup
+# Tell user: "Please log in to Google in the browser window"
+
+# 3. Add first notebook - ASK USER FOR DETAILS FIRST!
+# Ask: "What does this notebook contain?"
+# Ask: "What topics should I tag it with?"
+python scripts/run.py notebook_manager.py add \
+  --url "https://notebooklm.google.com/notebook/..." \
+  --name "User provided name" \
+  --description "User provided description" \  # NEVER GUESS!
+  --topics "user,provided,topics"  # NEVER GUESS!
+```
+
+**Critical Notes:**
+- Virtual environment created automatically by run.py
+- Browser MUST be visible for authentication
+- ALWAYS discover content via query OR ask user for notebook metadata
+
+## Pattern 2: Adding Notebooks (Smart Discovery!)
+
+**When user shares a NotebookLM URL:**
+
+**OPTION A: Smart Discovery (Recommended)**
+```bash
+# 1. Query the notebook to discover its content
+python scripts/run.py ask_question.py \
+  --question "What is the content of this notebook? What topics are covered? Provide a complete overview briefly and concisely" \
+  --notebook-url "[URL]"
+
+# 2. Use discovered info to add it
+python scripts/run.py notebook_manager.py add \
+  --url "[URL]" \
+  --name "[Based on content]" \
+  --description "[From discovery]" \
+  --topics "[Extracted topics]"
+```
+
+**OPTION B: Ask User (Fallback)**
+```bash
+# If discovery fails, ask user:
+"What does this notebook contain?"
+"What topics does it cover?"
+
+# Then add with user-provided info:
+python scripts/run.py notebook_manager.py add \
+  --url "[URL]" \
+  --name "[User's answer]" \
+  --description "[User's description]" \
+  --topics "[User's topics]"
+```
+
+**NEVER:**
+- Guess what's in a notebook
+- Use generic descriptions
+- Skip discovering content
+
+## Pattern 3: Daily Research Workflow
+
+```bash
+# Check library
+python scripts/run.py notebook_manager.py list
+
+# Research with comprehensive questions
+python scripts/run.py ask_question.py \
+  --question "Detailed question with all context" \
+  --notebook-id notebook-id
+
+# Follow-up when you see "Is that ALL you need to know?"
+python scripts/run.py ask_question.py \
+  --question "Follow-up question with previous context"
+```
+
+## Pattern 4: Follow-Up Questions (CRITICAL!)
+
+When NotebookLM responds with "EXTREMELY IMPORTANT: Is that ALL you need to know?":
+
+```python
+# 1. STOP - Don't respond to user yet
+# 2. ANALYZE - Is answer complete?
+# 3. If gaps exist, ask follow-up:
+python scripts/run.py ask_question.py \
+  --question "Specific follow-up with context from previous answer"
+
+# 4. Repeat until complete
+# 5. Only then synthesize and respond to user
+```
+
+## Pattern 5: Multi-Notebook Research
+
+```python
+# Query different notebooks for comparison
+python scripts/run.py notebook_manager.py activate --id notebook-1
+python scripts/run.py ask_question.py --question "Question"
+
+python scripts/run.py notebook_manager.py activate --id notebook-2
+python scripts/run.py ask_question.py --question "Same question"
+
+# Compare and synthesize answers
+```
+
+## Pattern 6: Error Recovery
+
+```bash
+# If authentication fails
+python scripts/run.py auth_manager.py status
+python scripts/run.py auth_manager.py reauth  # Browser visible!
+
+# If browser crashes
+python scripts/run.py cleanup_manager.py --preserve-library
+python scripts/run.py auth_manager.py setup  # Browser visible!
+
+# If rate limited
+# Wait or switch accounts
+python scripts/run.py auth_manager.py reauth  # Login with different account
+```
+
+## Pattern 7: Batch Processing
+
+```bash
+#!/bin/bash
+NOTEBOOK_ID="notebook-id"
+QUESTIONS=(
+    "First comprehensive question"
+    "Second comprehensive question"
+    "Third comprehensive question"
+)
+
+for question in "${QUESTIONS[@]}"; do
+    echo "Asking: $question"
+    python scripts/run.py ask_question.py \
+        --question "$question" \
+        --notebook-id "$NOTEBOOK_ID"
+    sleep 2  # Avoid rate limits
+done
+```
+
+## Pattern 8: Automated Research Script
+
+```python
+#!/usr/bin/env python
+import subprocess
+
+def research_topic(topic, notebook_id):
+    # Comprehensive question
+    question = f"""
+    Explain {topic} in detail:
+    1. Core concepts
+    2. Implementation details
+    3. Best practices
+    4. Common pitfalls
+    5. Examples
+    """
+
+    result = subprocess.run([
+        "python", "scripts/run.py", "ask_question.py",
+        "--question", question,
+        "--notebook-id", notebook_id
+    ], capture_output=True, text=True)
+
+    return result.stdout
+```
+
+## Pattern 9: Notebook Organization
+
+```python
+# Organize by domain - with proper metadata
+# ALWAYS ask user for descriptions!
+
+# Backend notebooks
+add_notebook("Backend API", "Complete API documentation", "api,rest,backend")
+add_notebook("Database", "Schema and queries", "database,sql,backend")
+
+# Frontend notebooks
+add_notebook("React Docs", "React framework documentation", "react,frontend")
+add_notebook("CSS Framework", "Styling documentation", "css,styling,frontend")
+
+# Search by domain
+python scripts/run.py notebook_manager.py search --query "backend"
+python scripts/run.py notebook_manager.py search --query "frontend"
+```
+
+## Pattern 10: Integration with Development
+
+```python
+# Query documentation during development
+def check_api_usage(api_endpoint):
+    result = subprocess.run([
+        "python", "scripts/run.py", "ask_question.py",
+        "--question", f"Parameters and response format for {api_endpoint}",
+        "--notebook-id", "api-docs"
+    ], capture_output=True, text=True)
+
+    # If follow-up needed
+    if "Is that ALL you need" in result.stdout:
+        # Ask for examples
+        follow_up = subprocess.run([
+            "python", "scripts/run.py", "ask_question.py",
+            "--question", f"Show code examples for {api_endpoint}",
+            "--notebook-id", "api-docs"
+        ], capture_output=True, text=True)
+
+    return combine_answers(result.stdout, follow_up.stdout)
+```
+
+## Best Practices
+
+### 1. Question Formulation
+- Be specific and comprehensive
+- Include all context in each question
+- Request structured responses
+- Ask for examples when needed
+
+### 2. Notebook Management
+- **ALWAYS ask user for metadata**
+- Use descriptive names
+- Add comprehensive topics
+- Keep URLs current
+
+### 3. Performance
+- Batch related questions
+- Use parallel processing for different notebooks
+- Monitor rate limits (50/day)
+- Switch accounts if needed
+
+### 4. Error Handling
+- Always use run.py to prevent venv issues
+- Check auth before operations
+- Implement retry logic
+- Have fallback notebooks ready
+
+### 5. Security
+- Use dedicated Google account
+- Never commit data/ directory
+- Regularly refresh auth
+- Track all access
+
+## Common Workflows for Claude
+
+### Workflow 1: User Sends NotebookLM URL
+
+```python
+# 1. Detect URL in message
+if "notebooklm.google.com" in user_message:
+    url = extract_url(user_message)
+
+    # 2. Check if in library
+    notebooks = run("notebook_manager.py list")
+
+    if url not in notebooks:
+        # 3. ASK USER FOR METADATA (CRITICAL!)
+        name = ask_user("What should I call this notebook?")
+        description = ask_user("What does this notebook contain?")
+        topics = ask_user("What topics does it cover?")
+
+        # 4. Add with user-provided info
+        run(f"notebook_manager.py add --url {url} --name '{name}' --description '{description}' --topics '{topics}'")
+
+    # 5. Use the notebook
+    answer = run(f"ask_question.py --question '{user_question}'")
+```
+
+### Workflow 2: Research Task
+
+```python
+# 1. Understand task
+task = "Implement feature X"
+
+# 2. Formulate comprehensive questions
+questions = [
+    "Complete implementation guide for X",
+    "Error handling for X",
+    "Performance considerations for X"
+]
+
+# 3. Query with follow-ups
+for q in questions:
+    answer = run(f"ask_question.py --question '{q}'")
+
+    # Check if follow-up needed
+    if "Is that ALL you need" in answer:
+        # Ask more specific question
+        follow_up = run(f"ask_question.py --question 'Specific detail about {q}'")
+
+# 4. Synthesize and implement
+```
+
+## Tips and Tricks
+
+1. **Always use run.py** - Prevents all venv issues
+2. **Ask for metadata** - Never guess notebook contents
+3. **Use verbose questions** - Include all context
+4. **Follow up automatically** - When you see the prompt
+5. **Monitor rate limits** - 50 queries per day
+6. **Batch operations** - Group related queries
+7. **Export important answers** - Save locally
+8. **Version control notebooks** - Track changes
+9. **Test auth regularly** - Before important tasks
+10. **Document everything** - Keep notes on notebooks
+
+## Quick Reference
+
+```bash
+# Always use run.py!
+python scripts/run.py [script].py [args]
+
+# Common operations
+run.py auth_manager.py status          # Check auth
+run.py auth_manager.py setup           # Login (browser visible!)
+run.py notebook_manager.py list        # List notebooks
+run.py notebook_manager.py add ...     # Add (ask user for metadata!)
+run.py ask_question.py --question ...  # Query
+run.py cleanup_manager.py ...          # Clean up
+```
+
+**Remember:** When in doubt, use run.py and ask the user for notebook details!

package/bundles/antigravity-plugin-extras/skills/notebooklm/scripts/__init__.py

@@ -0,0 +1,81 @@
+#!/usr/bin/env python3
+"""
+NotebookLM Skill Scripts Package
+Provides automatic environment management for all scripts
+"""
+
+import os
+import sys
+import subprocess
+from pathlib import Path
+
+
+def ensure_venv_and_run():
+    """
+    Ensure virtual environment exists and run the requested script.
+    This is called when any script is imported or run directly.
+    """
+    # Only do this if we're not already in the skill's venv
+    skill_dir = Path(__file__).parent.parent
+    venv_dir = skill_dir / ".venv"
+
+    # Check if we're in a venv
+    in_venv = hasattr(sys, 'real_prefix') or (
+        hasattr(sys, 'base_prefix') and sys.base_prefix != sys.prefix
+    )
+
+    # Check if it's OUR venv
+    if in_venv:
+        venv_path = Path(sys.prefix)
+        if venv_path == venv_dir:
+            # We're already in the correct venv
+            return
+
+    # We need to set up or switch to our venv
+    if not venv_dir.exists():
+        print("🔧 First-time setup detected...")
+        print("   Creating isolated environment for NotebookLM skill...")
+        print("   This ensures clean dependency management...")
+
+        # Create venv
+        import venv
+        venv.create(venv_dir, with_pip=True)
+
+        # Install requirements
+        requirements_file = skill_dir / "requirements.txt"
+        if requirements_file.exists():
+            if os.name == 'nt':  # Windows
+                pip_exe = venv_dir / "Scripts" / "pip.exe"
+            else:
+                pip_exe = venv_dir / "bin" / "pip"
+
+            print("   Installing dependencies in isolated environment...")
+            subprocess.run(
+                [str(pip_exe), "install", "-q", "-r", str(requirements_file)],
+                check=True
+            )
+
+            # Also install patchright's chromium
+            print("   Setting up browser automation...")
+            if os.name == 'nt':
+                python_exe = venv_dir / "Scripts" / "python.exe"
+            else:
+                python_exe = venv_dir / "bin" / "python"
+
+            subprocess.run(
+                [str(python_exe), "-m", "patchright", "install", "chromium"],
+                check=True,
+                capture_output=True
+            )
+
+        print("✅ Environment ready! All dependencies isolated in .venv/")
+
+    # If we're here and not in the venv, we should recommend using the venv
+    if not in_venv:
+        print("\n⚠️  Running outside virtual environment")
+        print("   Recommended: Use scripts/run.py to ensure clean execution")
+        print("   Or activate: source .venv/bin/activate")
+
+
+# Check environment when module is imported
+ensure_venv_and_run()

package/bundles/antigravity-plugin-extras/skills/notebooklm/scripts/ask_question.py

@@ -0,0 +1,256 @@
+#!/usr/bin/env python3
+"""
+Simple NotebookLM Question Interface
+Based on MCP server implementation - simplified without sessions
+
+Implements hybrid auth approach:
+- Persistent browser profile (user_data_dir) for fingerprint consistency
+- Manual cookie injection from state.json for session cookies (Playwright bug workaround)
+See: https://github.com/microsoft/playwright/issues/36139
+"""
+
+import argparse
+import sys
+import time
+import re
+from pathlib import Path
+
+from patchright.sync_api import sync_playwright
+
+# Add parent directory to path
+sys.path.insert(0, str(Path(__file__).parent))
+
+from auth_manager import AuthManager
+from notebook_manager import NotebookLibrary
+from config import QUERY_INPUT_SELECTORS, RESPONSE_SELECTORS
+from browser_utils import BrowserFactory, StealthUtils
+
+
+# Follow-up reminder (adapted from MCP server for stateless operation)
+# Since we don't have persistent sessions, we encourage comprehensive questions
+FOLLOW_UP_REMINDER = (
+    "\n\nEXTREMELY IMPORTANT: Is that ALL you need to know? "
+    "You can always ask another question! Think about it carefully: "
+    "before you reply to the user, review their original request and this answer. "
+    "If anything is still unclear or missing, ask me another comprehensive question "
+    "that includes all necessary context (since each question opens a new browser session)."
+)
+
+
+def ask_notebooklm(question: str, notebook_url: str, headless: bool = True) -> str:
+    """
+    Ask a question to NotebookLM
+
+    Args:
+        question: Question to ask
+        notebook_url: NotebookLM notebook URL
+        headless: Run browser in headless mode
+
+    Returns:
+        Answer text from NotebookLM
+    """
+    auth = AuthManager()
+
+    if not auth.is_authenticated():
+        print("⚠️ Not authenticated. Run: python auth_manager.py setup")
+        return None
+
+    print(f"💬 Asking: {question}")
+    print(f"📚 Notebook: {notebook_url}")
+
+    playwright = None
+    context = None
+
+    try:
+        # Start playwright
+        playwright = sync_playwright().start()
+
+        # Launch persistent browser context using factory
+        context = BrowserFactory.launch_persistent_context(
+            playwright,
+            headless=headless
+        )
+
+        # Navigate to notebook
+        page = context.new_page()
+        print("  🌐 Opening notebook...")
+        page.goto(notebook_url, wait_until="domcontentloaded")
+
+        # Wait for NotebookLM
+        page.wait_for_url(re.compile(r"^https://notebooklm\.google\.com/"), timeout=10000)
+
+        # Wait for query input (MCP approach)
+        print("  ⏳ Waiting for query input...")
+        query_element = None
+
+        for selector in QUERY_INPUT_SELECTORS:
+            try:
+                query_element = page.wait_for_selector(
+                    selector,
+                    timeout=10000,
+                    state="visible"  # Only check visibility, not disabled!
+                )
+                if query_element:
+                    print(f"  ✓ Found input: {selector}")
+                    break
+            except:
+                continue
+
+        if not query_element:
+            print("  ❌ Could not find query input")
+            return None
+
+        # Type question (human-like, fast)
+        print("  ⏳ Typing question...")
+        
+        # Use primary selector for typing
+        input_selector = QUERY_INPUT_SELECTORS[0]
+        StealthUtils.human_type(page, input_selector, question)
+
+        # Submit
+        print("  📤 Submitting...")
+        page.keyboard.press("Enter")
+
+        # Small pause
+        StealthUtils.random_delay(500, 1500)
+
+        # Wait for response (MCP approach: poll for stable text)
+        print("  ⏳ Waiting for answer...")
+
+        answer = None
+        stable_count = 0
+        last_text = None
+        deadline = time.time() + 120  # 2 minutes timeout
+
+        while time.time() < deadline:
+            # Check if NotebookLM is still thinking (most reliable indicator)
+            try:
+                thinking_element = page.query_selector('div.thinking-message')
+                if thinking_element and thinking_element.is_visible():
+                    time.sleep(1)
+                    continue
+            except:
+                pass
+
+            # Try to find response with MCP selectors
+            for selector in RESPONSE_SELECTORS:
+                try:
+                    elements = page.query_selector_all(selector)
+                    if elements:
+                        # Get last (newest) response
+                        latest = elements[-1]
+                        text = latest.inner_text().strip()
+
+                        if text:
+                            if text == last_text:
+                                stable_count += 1
+                                if stable_count >= 3:  # Stable for 3 polls
+                                    answer = text
+                                    break
+                            else:
+                                stable_count = 0
+                                last_text = text
+                except:
+                    continue
+
+            if answer:
+                break
+
+            time.sleep(1)
+
+        if not answer:
+            print("  ❌ Timeout waiting for answer")
+            return None
+
+        print("  ✅ Got answer!")
+        # Add follow-up reminder to encourage Claude to ask more questions
+        return answer + FOLLOW_UP_REMINDER
+
+    except Exception as e:
+        print(f"  ❌ Error: {e}")
+        import traceback
+        traceback.print_exc()
+        return None
+
+    finally:
+        # Always clean up
+        if context:
+            try:
+                context.close()
+            except:
+                pass
+
+        if playwright:
+            try:
+                playwright.stop()
+            except:
+                pass
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Ask NotebookLM a question')
+
+    parser.add_argument('--question', required=True, help='Question to ask')
+    parser.add_argument('--notebook-url', help='NotebookLM notebook URL')
+    parser.add_argument('--notebook-id', help='Notebook ID from library')
+    parser.add_argument('--show-browser', action='store_true', help='Show browser')
+
+    args = parser.parse_args()
+
+    # Resolve notebook URL
+    notebook_url = args.notebook_url
+
+    if not notebook_url and args.notebook_id:
+        library = NotebookLibrary()
+        notebook = library.get_notebook(args.notebook_id)
+        if notebook:
+            notebook_url = notebook['url']
+        else:
+            print(f"❌ Notebook '{args.notebook_id}' not found")
+            return 1
+
+    if not notebook_url:
+        # Check for active notebook first
+        library = NotebookLibrary()
+        active = library.get_active_notebook()
+        if active:
+            notebook_url = active['url']
+            print(f"📚 Using active notebook: {active['name']}")
+        else:
+            # Show available notebooks
+            notebooks = library.list_notebooks()
+            if notebooks:
+                print("\n📚 Available notebooks:")
+                for nb in notebooks:
+                    mark = " [ACTIVE]" if nb.get('id') == library.active_notebook_id else ""
+                    print(f"  {nb['id']}: {nb['name']}{mark}")
+                print("\nSpecify with --notebook-id or set active:")
+                print("python scripts/run.py notebook_manager.py activate --id ID")
+            else:
+                print("❌ No notebooks in library. Add one first:")
+                print("python scripts/run.py notebook_manager.py add --url URL --name NAME --description DESC --topics TOPICS")
+            return 1
+
+    # Ask the question
+    answer = ask_notebooklm(
+        question=args.question,
+        notebook_url=notebook_url,
+        headless=not args.show_browser
+    )
+
+    if answer:
+        print("\n" + "=" * 60)
+        print(f"Question: {args.question}")
+        print("=" * 60)
+        print()
+        print(answer)
+        print()
+        print("=" * 60)
+        return 0
+    else:
+        print("\n❌ Failed to get answer")
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())