@caddis/cli 0.1.0 → 0.2.0

package/bundles/antigravity-plugin-extras/skills/notebooklm/AUTHENTICATION.md

@@ -0,0 +1,154 @@
+# Authentication Architecture
+
+## Overview
+
+This skill uses a **hybrid authentication approach** that combines the best of both worlds:
+
+1. **Persistent Browser Profile** (`user_data_dir`) for consistent browser fingerprinting
+2. **Manual Cookie Injection** from `state.json` for reliable session cookie persistence
+
+## Why This Approach?
+
+### The Problem
+
+Playwright/Patchright has a known bug ([#36139](https://github.com/microsoft/playwright/issues/36139)) where **session cookies** (cookies without an `Expires` attribute) do not persist correctly when using `launch_persistent_context()` with `user_data_dir`.
+
+**What happens:**
+- ✅ Persistent cookies (with `Expires` date) → Saved correctly to browser profile
+- ❌ Session cookies (without `Expires`) → **Lost after browser restarts**
+
+**Impact:**
+- Some Google auth cookies are session cookies
+- Users experience random authentication failures
+- "Works on my machine" syndrome (depends on which cookies Google uses)
+
+### TypeScript vs Python
+
+The **MCP Server** (TypeScript) can work around this by passing `storage_state` as a parameter:
+
+```typescript
+// TypeScript - works!
+const context = await chromium.launchPersistentContext(userDataDir, {
+  storageState: "state.json",  // ← Loads cookies including session cookies
+  channel: "chrome"
+});
+```
+
+But **Python's Playwright API doesn't support this** ([#14949](https://github.com/microsoft/playwright/issues/14949)):
+
+```python
+# Python - NOT SUPPORTED!
+context = playwright.chromium.launch_persistent_context(
+    user_data_dir=profile_dir,
+    storage_state="state.json",  # ← Parameter not available in Python!
+    channel="chrome"
+)
+```
+
+## Our Solution: Hybrid Approach
+
+We use a **two-phase authentication system**:
+
+### Phase 1: Setup (`auth_manager.py setup`)
+
+1. Launch persistent context with `user_data_dir`
+2. User logs in manually
+3. **Save state to TWO places:**
+   - Browser profile directory (automatic, for fingerprint + persistent cookies)
+   - `state.json` file (explicit save, for session cookies)
+
+```python
+context = playwright.chromium.launch_persistent_context(
+    user_data_dir="browser_profile/",
+    channel="chrome"
+)
+# User logs in...
+context.storage_state(path="state.json")  # Save all cookies
+```
+
+### Phase 2: Runtime (`ask_question.py`)
+
+1. Launch persistent context with `user_data_dir` (loads fingerprint + persistent cookies)
+2. **Manually inject cookies** from `state.json` (adds session cookies)
+
+```python
+# Step 1: Launch with browser profile
+context = playwright.chromium.launch_persistent_context(
+    user_data_dir="browser_profile/",
+    channel="chrome"
+)
+
+# Step 2: Manually inject cookies from state.json
+with open("state.json", 'r') as f:
+    state = json.load(f)
+    context.add_cookies(state['cookies'])  # ← Workaround for session cookies!
+```
+
+## Benefits
+
+| Feature | Our Approach | Pure `user_data_dir` | Pure `storage_state` |
+|---------|--------------|----------------------|----------------------|
+| **Browser Fingerprint Consistency** | ✅ Same across restarts | ✅ Same | ❌ Changes each time |
+| **Session Cookie Persistence** | ✅ Manual injection | ❌ Lost (bug) | ✅ Native support |
+| **Persistent Cookie Persistence** | ✅ Automatic | ✅ Automatic | ✅ Native support |
+| **Google Trust** | ✅ High (same browser) | ✅ High | ❌ Low (new browser) |
+| **Cross-platform Reliability** | ✅ Chrome required | ⚠️ Chromium issues | ✅ Portable |
+| **Cache Performance** | ✅ Keeps cache | ✅ Keeps cache | ❌ No cache |
+
+## File Structure
+
+```
+~/.claude/skills/notebooklm/data/
+├── auth_info.json              # Metadata about authentication
+├── browser_state/
+│   ├── state.json             # Cookies + localStorage (for manual injection)
+│   └── browser_profile/       # Chrome user profile (for fingerprint + cache)
+│       ├── Default/
+│       │   ├── Cookies        # Persistent cookies only (session cookies missing!)
+│       │   ├── Local Storage/
+│       │   └── Cache/
+│       └── ...
+```
+
+## Why `state.json` is Critical
+
+Even though we use `user_data_dir`, we **still need `state.json`** because:
+
+1. **Session cookies** are not saved to the browser profile (Playwright bug)
+2. **Manual injection** is the only reliable way to load session cookies
+3. **Validation** - we can check if cookies are expired before launching
+
+## Code References
+
+**Setup:** `scripts/auth_manager.py:94-120`
+- Lines 100-113: Launch persistent context with `channel="chrome"`
+- Line 167: Save to `state.json` via `context.storage_state()`
+
+**Runtime:** `scripts/ask_question.py:77-118`
+- Lines 86-99: Launch persistent context
+- Lines 101-118: Manual cookie injection workaround
+
+**Validation:** `scripts/auth_manager.py:236-298`
+- Lines 262-275: Launch persistent context
+- Lines 277-287: Manual cookie injection for validation
+
+## Related Issues
+
+- [microsoft/playwright#36139](https://github.com/microsoft/playwright/issues/36139) - Session cookies not persisting
+- [microsoft/playwright#14949](https://github.com/microsoft/playwright/issues/14949) - Storage state with persistent context
+- [StackOverflow Question](https://stackoverflow.com/questions/79641481/) - Session cookie persistence issue
+
+## Future Improvements
+
+If Playwright adds support for `storage_state` parameter in Python's `launch_persistent_context()`, we can simplify to:
+
+```python
+# Future (when Python API supports it):
+context = playwright.chromium.launch_persistent_context(
+    user_data_dir="browser_profile/",
+    storage_state="state.json",  # ← Would handle everything automatically!
+    channel="chrome"
+)
+```
+
+Until then, our hybrid approach is the most reliable solution.

package/bundles/antigravity-plugin-extras/skills/notebooklm/SKILL.md

@@ -0,0 +1,270 @@
+---
+name: notebooklm
+context: fork
+description: Use this skill to query your Google NotebookLM notebooks directly from Claude Code for source-grounded, citation-backed answers from Gemini. Browser automation, library management, persistent auth. Drastically reduced hallucinations through document-only responses.
+---
+
+# NotebookLM Research Assistant Skill
+
+Interact with Google NotebookLM to query documentation with Gemini's source-grounded answers. Each question opens a fresh browser session, retrieves the answer exclusively from your uploaded documents, and closes.
+
+## When to Use This Skill
+
+Trigger when user:
+- Mentions NotebookLM explicitly
+- Shares NotebookLM URL (`https://notebooklm.google.com/notebook/...`)
+- Asks to query their notebooks/documentation
+- Wants to add documentation to NotebookLM library
+- Uses phrases like "ask my NotebookLM", "check my docs", "query my notebook"
+
+## ⚠️ CRITICAL: Add Command - Smart Discovery
+
+When user wants to add a notebook without providing details:
+
+**SMART ADD (Recommended)**: Query the notebook first to discover its content:
+```bash
+# Step 1: Query the notebook about its content
+python scripts/run.py ask_question.py --question "What is the content of this notebook? What topics are covered? Provide a complete overview briefly and concisely" --notebook-url "[URL]"
+
+# Step 2: Use the discovered information to add it
+python scripts/run.py notebook_manager.py add --url "[URL]" --name "[Based on content]" --description "[Based on content]" --topics "[Based on content]"
+```
+
+**MANUAL ADD**: If user provides all details:
+- `--url` - The NotebookLM URL
+- `--name` - A descriptive name
+- `--description` - What the notebook contains (REQUIRED!)
+- `--topics` - Comma-separated topics (REQUIRED!)
+
+NEVER guess or use generic descriptions! If details missing, use Smart Add to discover them.
+
+## Critical: Always Use run.py Wrapper
+
+**NEVER call scripts directly. ALWAYS use `python scripts/run.py [script]`:**
+
+```bash
+# ✅ CORRECT - Always use run.py:
+python scripts/run.py auth_manager.py status
+python scripts/run.py notebook_manager.py list
+python scripts/run.py ask_question.py --question "..."
+
+# ❌ WRONG - Never call directly:
+python scripts/auth_manager.py status  # Fails without venv!
+```
+
+The `run.py` wrapper automatically:
+1. Creates `.venv` if needed
+2. Installs all dependencies
+3. Activates environment
+4. Executes script properly
+
+## Core Workflow
+
+### Step 1: Check Authentication Status
+```bash
+python scripts/run.py auth_manager.py status
+```
+
+If not authenticated, proceed to setup.
+
+### Step 2: Authenticate (One-Time Setup)
+```bash
+# Browser MUST be visible for manual Google login
+python scripts/run.py auth_manager.py setup
+```
+
+**Important:**
+- Browser is VISIBLE for authentication
+- Browser window opens automatically
+- User must manually log in to Google
+- Tell user: "A browser window will open for Google login"
+
+### Step 3: Manage Notebook Library
+
+```bash
+# List all notebooks
+python scripts/run.py notebook_manager.py list
+
+# BEFORE ADDING: Ask user for metadata if unknown!
+# "What does this notebook contain?"
+# "What topics should I tag it with?"
+
+# Add notebook to library (ALL parameters are REQUIRED!)
+python scripts/run.py notebook_manager.py add \
+  --url "https://notebooklm.google.com/notebook/..." \
+  --name "Descriptive Name" \
+  --description "What this notebook contains" \  # REQUIRED - ASK USER IF UNKNOWN!
+  --topics "topic1,topic2,topic3"  # REQUIRED - ASK USER IF UNKNOWN!
+
+# Search notebooks by topic
+python scripts/run.py notebook_manager.py search --query "keyword"
+
+# Set active notebook
+python scripts/run.py notebook_manager.py activate --id notebook-id
+
+# Remove notebook
+python scripts/run.py notebook_manager.py remove --id notebook-id
+```
+
+### Quick Workflow
+1. Check library: `python scripts/run.py notebook_manager.py list`
+2. Ask question: `python scripts/run.py ask_question.py --question "..." --notebook-id ID`
+
+### Step 4: Ask Questions
+
+```bash
+# Basic query (uses active notebook if set)
+python scripts/run.py ask_question.py --question "Your question here"
+
+# Query specific notebook
+python scripts/run.py ask_question.py --question "..." --notebook-id notebook-id
+
+# Query with notebook URL directly
+python scripts/run.py ask_question.py --question "..." --notebook-url "https://..."
+
+# Show browser for debugging
+python scripts/run.py ask_question.py --question "..." --show-browser
+```
+
+## Follow-Up Mechanism (CRITICAL)
+
+Every NotebookLM answer ends with: **"EXTREMELY IMPORTANT: Is that ALL you need to know?"**
+
+**Required Claude Behavior:**
+1. **STOP** - Do not immediately respond to user
+2. **ANALYZE** - Compare answer to user's original request
+3. **IDENTIFY GAPS** - Determine if more information needed
+4. **ASK FOLLOW-UP** - If gaps exist, immediately ask:
+   ```bash
+   python scripts/run.py ask_question.py --question "Follow-up with context..."
+   ```
+5. **REPEAT** - Continue until information is complete
+6. **SYNTHESIZE** - Combine all answers before responding to user
+
+## Script Reference
+
+### Authentication Management (`auth_manager.py`)
+```bash
+python scripts/run.py auth_manager.py setup    # Initial setup (browser visible)
+python scripts/run.py auth_manager.py status   # Check authentication
+python scripts/run.py auth_manager.py reauth   # Re-authenticate (browser visible)
+python scripts/run.py auth_manager.py clear    # Clear authentication
+```
+
+### Notebook Management (`notebook_manager.py`)
+```bash
+python scripts/run.py notebook_manager.py add --url URL --name NAME --description DESC --topics TOPICS
+python scripts/run.py notebook_manager.py list
+python scripts/run.py notebook_manager.py search --query QUERY
+python scripts/run.py notebook_manager.py activate --id ID
+python scripts/run.py notebook_manager.py remove --id ID
+python scripts/run.py notebook_manager.py stats
+```
+
+### Question Interface (`ask_question.py`)
+```bash
+python scripts/run.py ask_question.py --question "..." [--notebook-id ID] [--notebook-url URL] [--show-browser]
+```
+
+### Data Cleanup (`cleanup_manager.py`)
+```bash
+python scripts/run.py cleanup_manager.py                    # Preview cleanup
+python scripts/run.py cleanup_manager.py --confirm          # Execute cleanup
+python scripts/run.py cleanup_manager.py --preserve-library # Keep notebooks
+```
+
+## Environment Management
+
+The virtual environment is automatically managed:
+- First run creates `.venv` automatically
+- Dependencies install automatically
+- Chromium browser installs automatically
+- Everything isolated in skill directory
+
+Manual setup (only if automatic fails):
+```bash
+python -m venv .venv
+source .venv/bin/activate  # Linux/Mac
+pip install -r requirements.txt
+python -m patchright install chromium
+```
+
+## Data Storage
+
+All data stored in `~/.claude/skills/notebooklm/data/`:
+- `library.json` - Notebook metadata
+- `auth_info.json` - Authentication status
+- `browser_state/` - Browser cookies and session
+
+**Security:** Protected by `.gitignore`, never commit to git.
+
+## Configuration
+
+Optional `.env` file in skill directory:
+```env
+HEADLESS=false           # Browser visibility
+SHOW_BROWSER=false       # Default browser display
+STEALTH_ENABLED=true     # Human-like behavior
+TYPING_WPM_MIN=160       # Typing speed
+TYPING_WPM_MAX=240
+DEFAULT_NOTEBOOK_ID=     # Default notebook
+```
+
+## Decision Flow
+
+```
+User mentions NotebookLM
+    ↓
+Check auth → python scripts/run.py auth_manager.py status
+    ↓
+If not authenticated → python scripts/run.py auth_manager.py setup
+    ↓
+Check/Add notebook → python scripts/run.py notebook_manager.py list/add (with --description)
+    ↓
+Activate notebook → python scripts/run.py notebook_manager.py activate --id ID
+    ↓
+Ask question → python scripts/run.py ask_question.py --question "..."
+    ↓
+See "Is that ALL you need?" → Ask follow-ups until complete
+    ↓
+Synthesize and respond to user
+```
+
+## Troubleshooting
+
+| Problem | Solution |
+|---------|----------|
+| ModuleNotFoundError | Use `run.py` wrapper |
+| Authentication fails | Browser must be visible for setup! --show-browser |
+| Rate limit (50/day) | Wait or switch Google account |
+| Browser crashes | `python scripts/run.py cleanup_manager.py --preserve-library` |
+| Notebook not found | Check with `notebook_manager.py list` |
+
+## Best Practices
+
+1. **Always use run.py** - Handles environment automatically
+2. **Check auth first** - Before any operations
+3. **Follow-up questions** - Don't stop at first answer
+4. **Browser visible for auth** - Required for manual login
+5. **Include context** - Each question is independent
+6. **Synthesize answers** - Combine multiple responses
+
+## Limitations
+
+- No session persistence (each question = new browser)
+- Rate limits on free Google accounts (50 queries/day)
+- Manual upload required (user must add docs to NotebookLM)
+- Browser overhead (few seconds per question)
+
+## Resources (Skill Structure)
+
+**Important directories and files:**
+
+- `scripts/` - All automation scripts (ask_question.py, notebook_manager.py, etc.)
+- `data/` - Local storage for authentication and notebook library
+- `references/` - Extended documentation:
+  - `api_reference.md` - Detailed API documentation for all scripts
+  - `troubleshooting.md` - Common issues and solutions
+  - `usage_patterns.md` - Best practices and workflow examples
+- `.venv/` - Isolated Python environment (auto-created on first run)
+- `.gitignore` - Protects sensitive data from being committed