copilot-session-tools 0.1.1__tar.gz

copilot_session_tools-0.1.1/CHANGELOG.md

@@ -0,0 +1,31 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] - 2025-02-13
+
+### Added
+
+- **Scanner**: Scan VS Code workspace storage (Stable and Insiders editions) to find Copilot chat sessions
+- **Scanner**: GitHub Copilot CLI chat history support (JSONL format from `~/.copilot/session-state`)
+- **Scanner**: Support for VS Code JSONL append-log format (VS Code >=1.109)
+- **Database**: SQLite storage with FTS5 full-text search indexing
+- **Database**: Two-layer design with raw compressed JSON as source of truth and derived tables
+- **Database**: Incremental scan support (only imports new/changed sessions)
+- **CLI**: `scan` command to import sessions from VS Code and CLI
+- **CLI**: `search` command with advanced query syntax (field filters, exact phrases, boolean logic)
+- **CLI**: `stats` command for database statistics
+- **CLI**: `export` command for JSON export
+- **CLI**: `export-markdown` command for Markdown export
+- **CLI**: `export-html` command for self-contained HTML export
+- **CLI**: `import-json` command for JSON import
+- **CLI**: `rebuild` command to recreate derived tables from raw JSON
+- **Web**: Flask-based web interface for browsing chat sessions
+- **Web**: Full-text search with highlighting
+- **Web**: Dark mode support via CSS `prefers-color-scheme`
+- **Web**: Syntax highlighting for code blocks
+- **Web**: Incremental refresh without restarting
+- **Tracking**: Tool invocations, file changes, and command runs from chat sessions

copilot_session_tools-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 copilot-session-tools contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

copilot_session_tools-0.1.1/PKG-INFO

@@ -0,0 +1,375 @@
+Metadata-Version: 2.4
+Name: copilot-session-tools
+Version: 0.1.1
+Summary: A collection of tools for VS Code GitHub Copilot chat history
+Keywords: copilot,github,vscode,chat,archive,history
+Author: Arithmomaniac
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Utilities
+Requires-Dist: orjson>=3.9.0
+Requires-Dist: jinja2>=3.0.0
+Requires-Dist: markdown>=3.4.0
+Requires-Dist: copilot-session-tools[cli,web] ; extra == 'all'
+Requires-Dist: typer>=0.12.0 ; extra == 'cli'
+Requires-Dist: rich>=13.0.0 ; extra == 'cli'
+Requires-Dist: flask>=3.0.0 ; extra == 'web'
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/Arithmomaniac/copilot-session-tools
+Project-URL: Repository, https://github.com/Arithmomaniac/copilot-session-tools
+Project-URL: Issues, https://github.com/Arithmomaniac/copilot-session-tools/issues
+Provides-Extra: all
+Provides-Extra: cli
+Provides-Extra: web
+Description-Content-Type: text/markdown
+
+# Copilot Session Tools
+
+[![CI](https://github.com/Arithmomaniac/copilot-session-tools/actions/workflows/ci.yml/badge.svg)](https://github.com/Arithmomaniac/copilot-session-tools/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/copilot-session-tools)](https://pypi.org/project/copilot-session-tools/) [![Python](https://img.shields.io/pypi/pyversions/copilot-session-tools)](https://pypi.org/project/copilot-session-tools/) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+
+Create a searchable archiveof your VS Code and GitHub Copilot CLI chat history, with a web viewer similar to [simonw/claude-code-transcripts](https://github.com/simonw/claude-code-transcripts).
+
+This project was informed by and borrows patterns from several excellent open-source projects:
+
+| Project | What We Borrowed |
+|---------|------------------|
+| [simonw/claude-code-transcripts](https://github.com/simonw/claude-code-transcripts) | HTML transcript generation, pagination approach, CLI structure |
+| [Arbuzov/copilot-chat-history](https://github.com/Arbuzov/copilot-chat-history) | VS Code Copilot chat session data format, workspace organization |
+| [jazzyalex/agent-sessions](https://github.com/jazzyalex/agent-sessions) | Multi-agent session concept, SQLite indexing patterns |
+| [tad-hq/universal-session-viewer](https://github.com/tad-hq/universal-session-viewer) | FTS5 full-text search design, session metadata schema |
+
+## Features
+
+- **Scan** VS Code workspace storage to find Copilot chat sessions (format based on [Arbuzov/copilot-chat-history](https://github.com/Arbuzov/copilot-chat-history))
+- **Support** for both VS Code Stable and Insiders editions
+- **GitHub Copilot CLI** chat history support (JSONL format from `~/.copilot/session-state`)
+- **Store** chat history in a SQLite database with FTS5 full-text search (inspired by [tad-hq/universal-session-viewer](https://github.com/tad-hq/universal-session-viewer))
+- **Browse** your archive with a web interface (similar to [simonw/claude-code-transcripts](https://github.com/simonw/claude-code-transcripts))
+- **Export/Import** sessions as JSON, Markdown, or self-contained HTML for backup or migration
+- **Tool invocations & file changes** tracking from chat sessions
+
+## Project Structure
+
+This is a Python package with optional extras for CLI and web interfaces:
+
+- **Base package**: Core utilities (database, scanner, markdown exporter)
+- **[cli] extra**: Command-line interface built with Typer
+- **[web] extra**: Flask-based web interface for browsing chat sessions
+- **[all] extra**: Both CLI and web interfaces
+
+## Installation
+
+```bash
+# Install with CLI
+pip install copilot-session-tools[cli]
+
+# Install with web interface
+pip install copilot-session-tools[web]
+
+# Install everything (CLI + web)
+pip install copilot-session-tools[all]
+```
+
+> **Tip:** Also works with `pipx install` or `uvx --with` if you prefer isolated tool environments.
+
+### From source (development)
+
+```bash
+git clone https://github.com/Arithmomaniac/copilot-session-tools.git
+cd copilot-session-tools
+
+# Install uv if you haven't already
+pip install uv
+
+# Install with all dependencies
+uv sync --all-extras
+```
+
+## Usage
+
+### 1. Scan for Chat Sessions
+
+Scan your VS Code workspace storage and GitHub Copilot CLI sessions to import into the database:
+
+```bash
+# Scan both VS Code (Stable and Insiders) and CLI sessions
+copilot-session-tools scan
+
+# Scan only VS Code Stable
+copilot-session-tools scan --edition stable
+
+# Scan only VS Code Insiders
+copilot-session-tools scan --edition insider
+
+# Use a custom database path
+copilot-session-tools scan --db my_chats.db
+
+# Scan custom storage paths
+copilot-session-tools scan --storage-path /path/to/workspaceStorage
+
+# Verbose output
+copilot-session-tools scan --verbose
+
+# Force re-import of all sessions
+copilot-session-tools scan --full
+```
+
+**Incremental Updates**: By default, the `scan` command only adds new sessions and updates changed ones based on file modification time. Use `--full` to re-import all sessions.
+
+**CLI Support**: The scanner automatically detects and imports GitHub Copilot CLI chat sessions from `~/.copilot/session-state/` by default.
+
+### 2. Start the Web Server
+
+Browse your chat archive in a web interface:
+
+```bash
+# Start the web server (uses copilot_chats.db by default)
+copilot-session-tools web
+
+# Custom options
+copilot-session-tools web --db my_chats.db --port 8080 --title "My Copilot Chats"
+```
+
+Then open `http://127.0.0.1:5000/` in your browser.
+
+### 3. Search Chats
+
+Search through your chat history from the command line:
+
+```bash
+# Basic search
+copilot-session-tools search "authentication"
+
+# Limit results
+copilot-session-tools search "React hooks" --limit 50
+
+# Filter by role
+copilot-session-tools search "error" --role assistant
+
+# Search only tool invocations
+copilot-session-tools search "git" --tools-only
+
+# Show full content (not truncated)
+copilot-session-tools search "complex query" --full
+```
+
+**Advanced Search Syntax:**
+
+The search supports powerful query syntax:
+
+- **Multiple words:** `python function` matches messages containing both words (AND logic)
+- **Exact phrases:** `"python function"` matches the exact phrase
+- **Field filters:** Filter by specific fields directly in the query:
+  - `role:user` - Filter to user messages only
+  - `role:assistant` - Filter to assistant messages only
+  - `workspace:my-project` - Filter to a specific workspace
+  - `title:session-name` - Filter by session title
+
+```bash
+# Search for "function" only in user messages
+copilot-session-tools search "role:user function"
+
+# Search in a specific workspace
+copilot-session-tools search "workspace:my-project python"
+
+# Combine filters
+copilot-session-tools search "workspace:react role:assistant hooks"
+
+# Sort by date instead of relevance
+copilot-session-tools search "python" --sort date
+```
+
+### 4. View Statistics
+
+```bash
+copilot-session-tools stats
+```
+
+### 5. Export/Import
+
+```bash
+# Export all sessions to JSON
+copilot-session-tools export --output chats.json
+
+# Export to stdout
+copilot-session-tools export
+
+# Export as Markdown files
+copilot-session-tools export-markdown --output-dir ./markdown-archive
+
+# Export a single session
+copilot-session-tools export-markdown --session-id abc123 --output-dir ./session
+
+# Include file diffs in markdown
+copilot-session-tools export-markdown --include-diffs
+
+# Export as self-contained HTML (same rendering as web viewer, no server needed)
+copilot-session-tools export-html --output-dir ./html-archive
+
+# Export a single session as HTML
+copilot-session-tools export-html --session-id abc123 --output-dir ./session
+
+# Import from JSON
+copilot-session-tools import-json chats.json
+```
+
+## Chat Storage Locations
+
+### VS Code
+
+VS Code stores Copilot chat history in workspace-specific storage:
+
+| OS | Path |
+|----|------|
+| Windows | `%APPDATA%\Code\User\workspaceStorage\{hash}\` |
+| macOS | `~/Library/Application Support/Code/User/workspaceStorage/{hash}/` |
+| Linux | `~/.config/Code/User/workspaceStorage/{hash}/` |
+
+For VS Code Insiders, replace `Code` with `Code - Insiders`.
+
+### GitHub Copilot CLI
+
+The GitHub Copilot CLI stores chat history in JSONL format:
+
+| OS | Path |
+|----|------|
+| All | `~/.copilot/session-state/` (current format, v0.0.342+) |
+| All | `~/.copilot/history-session-state/` (legacy format) |
+
+The scanner automatically detects and imports both VS Code and CLI sessions by default.
+
+## Database Schema
+
+The SQLite database uses a two-layer design:
+
+1. **`raw_sessions` table** - Stores compressed raw JSON as the source of truth
+2. **Derived tables** - Can be dropped and recreated from raw_sessions without migrations
+
+```sql
+-- Raw sessions table (source of truth)
+CREATE TABLE raw_sessions (
+    id INTEGER PRIMARY KEY,
+    session_id TEXT UNIQUE NOT NULL,
+    raw_json_compressed BLOB NOT NULL,  -- zlib-compressed original JSON
+    workspace_name TEXT,
+    workspace_path TEXT,
+    source_file TEXT,
+    vscode_edition TEXT,
+    source_file_mtime REAL,
+    source_file_size INTEGER,
+    imported_at TIMESTAMP
+);
+
+-- Derived sessions table
+CREATE TABLE sessions (
+    id INTEGER PRIMARY KEY,
+    session_id TEXT UNIQUE NOT NULL,
+    workspace_name TEXT,
+    workspace_path TEXT,
+    created_at TEXT,
+    updated_at TEXT,
+    source_file TEXT,
+    vscode_edition TEXT,
+    custom_title TEXT,
+    imported_at TIMESTAMP,
+    type TEXT DEFAULT 'vscode'  -- 'vscode' or 'cli'
+);
+
+-- Messages table
+CREATE TABLE messages (
+    id INTEGER PRIMARY KEY,
+    session_id TEXT NOT NULL,
+    message_index INTEGER NOT NULL,
+    role TEXT NOT NULL,
+    content TEXT NOT NULL,
+    timestamp TEXT,
+    cached_markdown TEXT
+);
+
+-- Full-text search virtual table
+CREATE VIRTUAL TABLE messages_fts USING fts5(content);
+
+-- Tool invocations, file changes, and command runs are also tracked
+```
+
+### Rebuilding Derived Tables
+
+When the schema changes, you can rebuild all derived tables from the stored raw JSON:
+
+```bash
+copilot-session-tools rebuild --db copilot_chats.db
+```
+
+This drops and recreates the sessions, messages, and related tables without needing to re-scan the original VS Code storage.
+
+## Web Viewer Features
+
+The web interface includes:
+
+- **Session list** with workspace names and message counts, sorted by most recent message
+- **Workspace filtering** to focus on specific projects
+- **Full-text search** with highlighting
+- **Dark mode support** via CSS `prefers-color-scheme`
+- **Responsive design** for mobile and desktop
+- **Syntax highlighting** for code blocks
+- **Incremental refresh** to update without restarting
+
+## Development
+
+```bash
+# Clone the repository
+git clone https://github.com/Arithmomaniac/copilot-session-tools.git
+cd copilot-session-tools
+
+# Install uv
+pip install uv
+
+# Sync the workspace (installs all packages in development mode)
+uv sync
+
+# Run tests
+uv run pytest
+
+# Run tests with coverage
+uv run pytest --cov
+
+# Run the CLI
+uv run copilot-session-tools --help
+```
+
+## Agent Skills
+
+This repository includes [Agent Skills](https://claude-plugins.dev) for AI coding agents (Claude Code, Cursor, VS Code, Codex, and more):
+
+| Skill | Description |
+|-------|-------------|
+| **search-copilot-chats** | Search, browse, and export archived Copilot chat sessions using this tool's CLI |
+| **scanner-refresh** | Research recent changes in Copilot repos and update the scanner for new event types |
+
+### Install a skill
+
+```bash
+# Install the search skill (Claude Code is the default client)
+npx skills-installer install @Arithmomaniac/copilot-session-tools/search-copilot-chats
+
+# For other clients
+npx skills-installer install @Arithmomaniac/copilot-session-tools/search-copilot-chats --client cursor
+```
+
+Skills are automatically available when working in this repository (project-level `.claude/skills/`).
+
+## Related Projects
+
+- [simonw/claude-code-transcripts](https://github.com/simonw/claude-code-transcripts) - Inspiration for the web viewer
+- [Arbuzov/copilot-chat-history](https://github.com/Arbuzov/copilot-chat-history) - VS Code extension for viewing chat history
+- [microsoft/vscode-copilot-chat](https://github.com/microsoft/vscode-copilot-chat) - Official VS Code Copilot Chat extension
+
+## License
+
+MIT License - see [LICENSE](LICENSE) for details.