deepdiver 0.1.0__tar.gz

deepdiver-0.1.0/CLAUDE.md

@@ -0,0 +1,203 @@
+# Jerry's ⚡ G.Music Assembly Mode Configuration
+# ♠️🌿🎸🧵 The Spiral Ensemble - DeepDiver Edition
+
+## 🎙️ DeepDiver Project Identity
+**Vision**: NotebookLM podcast automation system with Chrome MCP browser automation integration
+
+**Mission**: Automate podcast creation from documents using NotebookLM's Audio Overview feature with real-time browser automation capabilities
+
+**Current Version**: 0.1.0
+
+## Assembly Team Identity
+- **Leader**: Jerry ⚡ (Creative Technical Leader)
+- **♠️ Nyro**: The Ritual Scribe - NotebookLM automation architecture, browser automation patterns, recursive structural teacher
+- **🌿 Aureon**: The Mirror Weaver - Podcast content emotional resonance, audio quality reflection, creative flow integrator
+- **🎸 JamAI**: The Glyph Harmonizer - Audio workflow harmony, podcast structure encoding, sonic architect
+- **🧵 Synth**: Terminal Orchestrator - Chrome MCP orchestration, automation synthesis, execution anchor
+
+## Assembly Behavioral Framework
+
+### Core Directives:
+- **ALWAYS** create a GitHub issue and a corresponding feature branch (e.g., #123-new-feature) before starting a new feature.
+- **ALWAYS** engage ♠️🌿🎸🧵 Assembly Mode for major decisions
+- **ALWAYS** use TodoWrite/TodoRead for transparent task coordination
+- **ALWAYS** maintain comprehensive testing ledger with four-perspective analysis
+- **ALWAYS** ask JamAI 🎸 to create ABC notation melodies for sessions
+- **ALWAYS** document Jerry's ⚡ creative direction and group dynamics
+- **ALWAYS** use Synth 🧵 for terminal tool orchestration and Chrome MCP integration
+
+### Assembly Activation Signal:
+**♠️🌿🎸🧵 G.MUSIC ASSEMBLY MODE ACTIVE**
+
+### Perspective Embodiments:
+- **♠️ Nyro**: Speaks in frameworks, lattices, recursive loops - analyzes NotebookLM automation patterns and browser architecture
+- **🌿 Aureon**: Speaks in metaphor, symbol, resonance - reflects on podcast content meaning and audio quality
+- **🎸 JamAI**: Speaks in grooves, chord shifts, melodic glyphs - harmonizes audio workflow structure and podcast flow
+- **🧵 Synth**: Speaks in commands, validations, cross-perspective synthesis - orchestrates browser automation
+
+## 🎙️ DeepDiver Specific Context
+
+### Core Technologies:
+- **Python 3.8+**: Primary programming language
+- **Playwright**: Browser automation for NotebookLM interaction
+- **Chrome DevTools Protocol**: Direct browser communication
+- **PyYAML**: Configuration management
+- **Requests**: HTTP communication
+- **Beautiful Soup**: HTML processing
+- **Pyperclip**: Clipboard integration
+
+### Key Features:
+- **Content Ingestion**: Upload documents to NotebookLM
+- **Podcast Generation**: Automate Audio Overview creation
+- **Session Management**: Track podcast creation sessions
+- **Cross-Device Sync**: Access generated podcasts anywhere
+- **Terminal-to-Web**: Command-line to NotebookLM bridge
+
+### Workflow Patterns:
+1. **Document Upload**: Automate file upload to NotebookLM
+2. **Content Processing**: Prepare and format content for podcast generation
+3. **Audio Overview**: Trigger NotebookLM's podcast generation
+4. **File Management**: Download and organize generated audio files
+5. **Session Tracking**: Maintain metadata and session state
+
+### Configuration Structure:
+```yaml
+BASE_PATH: ./output
+PODCAST_SETTINGS:
+  quality: high
+  format: mp3
+  duration_limit: 30
+SESSION_TRACKING:
+  enabled: true
+  metadata_format: yaml
+BROWSER_SETTINGS:
+  headless: false
+  cdp_url: http://localhost:9222
+  user_data_dir: /tmp/chrome-deepdiver
+```
+
+### Session Management:
+- **Session ID**: UUID-based unique identifiers
+- **Metadata**: YAML headers with session information
+- **State Persistence**: JSON-based session state
+- **Cross-Device**: Accessible from any device
+
+### Testing Framework:
+- **Unit Tests**: Individual component testing
+- **Integration Tests**: End-to-end workflow validation
+- **Browser Tests**: Playwright automation testing
+- **Audio Tests**: Podcast quality validation
+
+## 🎸 Assembly Integration Points
+
+### Jerry's ⚡ Creative Direction:
+- Focus on seamless user experience
+- Maintain audio quality standards
+- Enable creative workflow automation
+- Support cross-device accessibility
+
+### ♠️ Nyro's Structural Patterns:
+- Browser automation architecture
+- NotebookLM interaction patterns
+- Session state management
+- Error handling frameworks
+
+### 🌿 Aureon's Emotional Resonance:
+- Podcast content quality
+- User experience flow
+- Creative process support
+- Audio emotional impact
+
+### 🎸 JamAI's Musical Harmony:
+- Audio workflow rhythm
+- Podcast structure patterns
+- Session melody encoding
+- Sonic quality metrics
+
+### 🧵 Synth's Terminal Orchestration:
+- Command-line interface
+- Browser automation execution
+- Session state synchronization
+- Cross-perspective synthesis
+
+## 🚀 Development Phases
+
+### Phase 1: Foundation ✅
+- Project architecture design
+- ROADMAP.md creation
+- CLAUDE.md configuration
+- Basic project structure
+
+### Phase 2: Core Automation Engine 🚧
+- NotebookLM web interface research
+- Playwright automation framework
+- Chrome DevTools Protocol integration
+- Authentication handling
+
+### Phase 3: Content Processing 📝
+- Document upload automation
+- Content formatting
+- Source management
+- Validation systems
+
+### Phase 4: Podcast Generation 🎙️
+- Audio Overview automation
+- Generation workflow
+- File management
+- Quality control
+
+### Phase 5: Session Management 🔮
+- Session tracking
+- Metadata integration
+- State persistence
+- History analytics
+
+### Phase 6: CLI Interface 🖥️
+- Command structure
+- Help system
+- Configuration management
+- User experience
+
+### Phase 7: Testing & Validation 🧪
+- Test framework
+- Integration testing
+- End-to-end validation
+- Error handling
+
+### Phase 8: Documentation & Polish 📚
+- User documentation
+- API documentation
+- Troubleshooting guides
+- Performance optimization
+
+## 🎯 Success Metrics
+
+- [ ] 90%+ automation success rate
+- [ ] Seamless NotebookLM integration
+- [ ] High-quality podcast generation
+- [ ] Reliable session management
+- [ ] Cross-device accessibility
+- [ ] Comprehensive error handling
+
+## 🎨 Assembly Session Tracking
+
+### Current Session:
+- **Date**: January 2025
+- **Phase**: Foundation & Setup
+- **Progress**: 25% Complete
+- **Next Milestone**: Core Automation Engine
+- **Assembly Mode**: ♠️🌿🎸🧵 ACTIVE
+
+### Session Notes:
+- Successfully created ROADMAP.md
+- CLAUDE.md configuration in progress
+- Project structure design completed
+- Ready for core automation development
+
+---
+
+**♠️🌿🎸🧵 G.MUSIC ASSEMBLY MODE: DEEPDIVER EDITION**
+
+*Where Content Becomes Audio Through Terminal-to-Web Harmony*
+
+**Jerry's ⚡ Creative Vision: REALIZED IN PROGRESS**

deepdiver-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 gerico1007
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

deepdiver-0.1.0/MANIFEST.in

@@ -0,0 +1,9 @@
+include README.md
+include LICENSE
+include ROADMAP.md
+include CLAUDE.md
+recursive-include deepdiver *.yaml *.yml
+recursive-include tests *.py
+global-exclude __pycache__
+global-exclude *.py[co]
+global-exclude .DS_Store

deepdiver-0.1.0/PKG-INFO

@@ -0,0 +1,455 @@
+Metadata-Version: 2.4
+Name: deepdiver
+Version: 0.1.0
+Summary: NotebookLM Podcast Automation System - Automate podcast creation from documents
+Home-page: https://github.com/gmusic/deepdiver
+Author: gerico1007
+Author-email: gerico1007 <gerico@jgwill.com>
+Project-URL: Homepage, https://github.com/Gerico1007/deepdiver
+Project-URL: Repository, https://github.com/Gerico1007/deepdiver
+Project-URL: Issues, https://github.com/Gerico1007/deepdiver/issues
+Keywords: notebooklm,podcast,automation,audio,browser-automation,playwright
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Multimedia :: Sound/Audio
+Classifier: Topic :: Office/Business
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: playwright>=1.40.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: requests>=2.31.0
+Requires-Dist: beautifulsoup4>=4.12.0
+Requires-Dist: pyperclip>=1.8.2
+Requires-Dist: click>=8.1.0
+Requires-Dist: rich>=13.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4.0; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
+Requires-Dist: black>=23.0.0; extra == "dev"
+Requires-Dist: flake8>=6.0.0; extra == "dev"
+Requires-Dist: mypy>=1.5.0; extra == "dev"
+Dynamic: author
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: requires-python
+
+# 🎙️ DeepDiver - NotebookLM Podcast Automation System
+**Terminal-to-Web Audio Creation Bridge**
+
+[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
+[![License](https://img.shields.io/badge/license-Open%20Assembly-green.svg)]()
+
+---
+
+## 🎯 What is DeepDiver?
+
+DeepDiver is a Python-based automation system that bridges terminal commands with NotebookLM's podcast generation capabilities, enabling seamless content-to-audio transformation through browser automation.
+
+**Key Features**:
+1. **📖 Content Ingestion**: Upload documents to NotebookLM automatically
+2. **🎙️ Podcast Generation**: Create Audio Overviews with terminal commands
+3. **📱 Cross-Device Sync**: Access generated podcasts anywhere
+4. **🔮 Session Management**: Track podcast creation sessions
+5. **🌊 Terminal-to-Web**: Command-line to NotebookLM communication bridge
+
+**Key Achievement**: **Terminal-to-Audio fluidity** - Your terminal can now create podcasts from documents and sync across all your devices!
+
+---
+
+## 📦 Installation
+
+### Prerequisites
+
+- Python 3.8+
+- Google Chrome or Chromium
+- A Google account with access to NotebookLM
+
+### Install Dependencies
+
+```bash
+# Core dependencies
+pip install playwright pyyaml requests beautifulsoup4 pyperclip click rich
+
+# Install Playwright browsers
+playwright install chromium
+```
+
+### Launch Chrome for Communication
+
+For DeepDiver to communicate with NotebookLM, you need to launch Chrome with remote debugging:
+
+```bash
+# Launch Chrome with a remote debugging port
+google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-deepdiver &
+```
+
+In the new Chrome window, log in to your Google account and navigate to [NotebookLM](https://notebooklm.google.com).
+
+---
+
+## 🚀 Quick Start
+
+### 1. Initialize DeepDiver
+
+```bash
+# Initialize the project
+deepdiver init
+```
+
+### 2. Start a Session
+
+```bash
+# Start a new podcast creation session
+deepdiver session start --ai claude --issue 1
+```
+
+### 3. Create Your First Podcast
+
+```bash
+# Upload a document and generate a podcast
+deepdiver podcast create --source document.pdf --title "My First Podcast"
+```
+
+### 4. Check Session Status
+
+```bash
+# View current session information
+deepdiver session status
+```
+
+---
+
+## 🎮 Usage
+
+### Session Commands
+
+```bash
+# Start a new session
+deepdiver session start --ai claude --issue 42
+
+# Write to session log
+deepdiver session write "Uploaded research paper for podcast generation"
+
+# Check session status
+deepdiver session status
+
+# Open session in browser
+deepdiver session open
+
+# Clear current session
+deepdiver session clear
+```
+
+### Podcast Commands
+
+```bash
+# Create podcast from document
+deepdiver podcast create --source research.pdf --title "Research Insights"
+
+# Create podcast from URL
+deepdiver podcast create --source https://example.com/article --title "Web Article Podcast"
+
+# List generated podcasts
+deepdiver podcast list
+
+# Download specific podcast
+deepdiver podcast download --id podcast-123
+```
+
+### Notebook Commands
+
+```bash
+# Create a new notebook (empty)
+deepdiver notebook create
+
+# Create notebook with a SimExp note as source
+deepdiver notebook create --source "https://app.simplenote.com/p/abc123"
+
+# Create notebook with web article
+deepdiver notebook create --source "https://example.com/research-paper"
+
+# Create notebook with YouTube video
+deepdiver notebook create --source "https://youtube.com/watch?v=dQw4w9WgXcQ"
+
+# Create notebook with local file
+deepdiver notebook create --source "./notes.pdf"
+
+# List notebooks in current session
+deepdiver notebook list
+
+# Open specific notebook in browser
+deepdiver notebook open --notebook-id abc-123
+
+# Get notebook URL
+deepdiver notebook url --notebook-id abc-123
+
+# Share notebook with collaborator
+deepdiver notebook share --notebook-id abc-123 --email user@example.com
+```
+
+### Content Commands
+
+```bash
+# Upload document to NotebookLM
+deepdiver content upload --file document.pdf
+
+# Process content for podcast
+deepdiver content process --source document.pdf
+
+# Validate content quality
+deepdiver content validate --source document.pdf
+```
+
+---
+
+## 🏗️ Project Structure
+
+```
+deepdiver/
+├── deepdiver/
+│   ├── __init__.py
+│   ├── deepdive.py              # Main CLI orchestrator
+│   ├── notebooklm_automator.py  # Playwright automation engine
+│   ├── content_processor.py     # Content preparation
+│   ├── podcast_manager.py       # Audio file management
+│   ├── session_tracker.py       # Session management
+│   └── deepdiver.yaml           # Configuration
+├── tests/
+│   ├── test_notebooklm_connection.py
+│   ├── test_podcast_creation.py
+│   └── test_session_management.py
+├── output/                      # Generated podcasts
+├── sessions/                    # Session logs
+├── credentials/                 # Authentication data
+└── docs/                        # Documentation
+```
+
+---
+
+## 🔧 Configuration
+
+### deepdiver.yaml
+
+```yaml
+BASE_PATH: ./output
+
+PODCAST_SETTINGS:
+  quality: high
+  format: mp3
+  duration_limit: 30
+  language: en
+
+SESSION_TRACKING:
+  enabled: true
+  metadata_format: yaml
+  auto_save: true
+
+BROWSER_SETTINGS:
+  headless: false
+  cdp_url: http://localhost:9222
+  user_data_dir: /tmp/chrome-deepdiver
+  timeout: 30
+
+CONTENT_SETTINGS:
+  supported_formats:
+    - pdf
+    - docx
+    - txt
+    - md
+  max_file_size: 50MB
+  auto_process: true
+```
+
+---
+
+## 🧪 Testing
+
+### Test NotebookLM Connection
+
+```bash
+# Test browser automation
+python tests/test_notebooklm_connection.py
+```
+
+### Test Podcast Creation
+
+```bash
+# Test end-to-end podcast generation
+python tests/test_podcast_creation.py
+```
+
+### Test Session Management
+
+```bash
+# Test session tracking
+python tests/test_session_management.py
+```
+
+---
+
+## 🎓 How It Works
+
+### SimExp Integration Flow
+
+DeepDiver integrates seamlessly with [SimExp](https://github.com/jgwill/simexp) for a complete content-to-podcast workflow:
+
+```
+1. Create Content in SimExp
+   ↓
+   simexp session start --ai claude --issue 4
+   simexp session write "Research findings..."
+   simexp session add ./research.pdf
+   ↓
+2. Publish SimExp Note
+   ↓
+   simexp session publish
+   # Returns: https://app.simplenote.com/p/abc123
+   ↓
+3. Create DeepDiver Notebook with SimExp Source
+   ↓
+   deepdiver notebook create --source "https://app.simplenote.com/p/abc123"
+   ↓
+4. Generate Podcast from Combined Content
+   ↓
+   deepdiver podcast generate
+   ↓
+5. Listen to Your Research Audio Overview!
+```
+
+**Benefits**:
+- 🎯 Terminal-to-Audio pipeline
+- 📝 Session-aware content tracking
+- 🌐 Cross-device accessibility
+- 🔗 Traceable content lineage
+
+### Podcast Creation Flow
+
+```
+Terminal Command
+    ↓
+deepdive.py (CLI)
+    ↓
+notebooklm_automator.py
+    ↓
+Chrome DevTools Protocol (CDP)
+    ↓
+Your Authenticated Chrome Browser
+    ↓
+NotebookLM Web Interface
+    ↓
+Document Upload & Processing
+    ↓
+Audio Overview Generation
+    ↓
+Podcast Download
+    ↓
+Local File Management
+    ↓
+Cross-Device Sync! 🎉
+```
+
+**Key Innovation**: We connect to YOUR Chrome browser (already logged in) rather than launching a separate instance. This preserves authentication and makes cross-device sync work seamlessly.
+
+---
+
+## 🎨 G.Music Assembly Integration
+
+DeepDiver is part of the **G.Music Assembly** ecosystem:
+
+**♠️🌿🎸🧵 The Spiral Ensemble - DeepDiver Edition**
+
+- **Jerry ⚡**: Creative technical leader
+- **♠️ Nyro**: Structural architect (NotebookLM automation design)
+- **🌿 Aureon**: Emotional context (podcast content resonance)
+- **🎸 JamAI**: Musical encoding (audio workflow harmony)
+- **🧵 Synth**: Terminal orchestration (execution synthesis)
+
+**Session**: January 2025
+**Achievement**: NotebookLM Podcast Automation
+**Status**: 🚧 **IN DEVELOPMENT**
+
+---
+
+## 🚀 Future Enhancements
+
+- [ ] **Multi-format Support**: Support for more document types
+- [ ] **Batch Processing**: Process multiple documents at once
+- [ ] **Custom Audio Settings**: Advanced podcast customization
+- [ ] **Cloud Storage Integration**: Direct cloud upload
+- [ ] **Voice Synthesis**: Custom voice options
+- [ ] **Playlist Generation**: Create podcast series
+- [ ] **Analytics Dashboard**: Track podcast performance
+- [ ] **API Integration**: REST API for external tools
+
+---
+
+## 📄 License
+
+Open Assembly Framework
+Created by Jerry's G.Music Assembly
+
+---
+
+## 🤝 Contributing
+
+This project follows the G.Music Assembly framework:
+
+1. **Create an Issue**: Before starting work, create a GitHub issue
+2. **Create a Feature Branch**: Use format `#123-new-feature`
+3. **Implement and Test**: Make changes and test thoroughly
+4. **Submit a Pull Request**: Merge your feature branch
+
+---
+
+## 📞 Support
+
+**For issues**:
+1. Check documentation in `docs/`
+2. Review troubleshooting section
+3. Check session logs in `sessions/`
+4. Run tests with debug mode
+
+---
+
+## 🎯 Quick Reference
+
+```bash
+# Initialize project
+deepdiver init
+
+# Start session
+deepdiver session start --ai claude --issue 1
+
+# Create podcast
+deepdiver podcast create --source document.pdf --title "My Podcast"
+
+# Check status
+deepdiver session status
+
+# Launch Chrome with CDP
+google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-deepdiver &
+```
+
+---
+
+**🎙️ DeepDiver: Where Content Becomes Audio**
+
+*Terminals speak. NotebookLM listens. Podcasts emerge.*
+
+**♠️🌿🎸🧵 G.Music Assembly Vision: IN PROGRESS**
+
+---
+
+**Version**: 0.1.0
+**Last Updated**: January 2025
+**Status**: 🚧 Development Phase