docpilot-cli 1.0.0__tar.gz

docpilot_cli-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ASWIN ASHOK
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

docpilot_cli-1.0.0/PKG-INFO

@@ -0,0 +1,160 @@
+Metadata-Version: 2.4
+Name: docpilot-cli
+Version: 1.0.0
+Summary: A local-first RAG pipeline CLI tool
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: bs4>=0.0.2
+Requires-Dist: beautifulsoup4>=4.12.0
+Requires-Dist: langchain>=0.2.0
+Requires-Dist: langchain-chroma>=0.1.0
+Requires-Dist: langchain-ollama>=0.1.0
+Requires-Dist: ollama>=0.2.0
+Requires-Dist: pandas>=2.0.0
+Requires-Dist: typer>=0.12.0
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: tomli-w>=1.0.0
+Requires-Dist: pypdf>=5.0.0
+Requires-Dist: pyfiglet>=0.8.0
+Requires-Dist: rich>=13.0.0
+Dynamic: license-file
+
+# 🚀 DocPilot CLI
+
+![PyPI - Version](https://img.shields.io/pypi/v/docpilot-cli?color=blue&style=flat-square)
+![Python Version](https://img.shields.io/badge/python-3.12%2B-blue?style=flat-square)
+![License](https://img.shields.io/badge/license-MIT-green?style=flat-square)
+
+**DocPilot** is a lightning-fast, local-first CLI for document ingestion and interactive question-answering. Powered by [Ollama](https://ollama.com/) and Chroma, it allows you to ingest websites, PDFs, and CSVs directly from your terminal and chat with your documents—keeping 100% of your data safely on your own machine.
+
+It’s built for practical developer workflows: crawl sites concurrently, prepare chunks with multi-threading, and iterate rapidly without ever paying for a cloud API.
+
+---
+
+## ✨ Features
+
+- **🔒 100% Local**: No data ever leaves your machine. Powered by Ollama.
+- **⚡ Interactive Setup Wizard**: Get up and running instantly with smart model auto-detection.
+- **🌐 Universal Ingestion**: Seamlessly ingest Website URLs, XML Sitemaps, PDFs, and CSVs.
+- **🚀 Concurrent Processing**: Lightning-fast crawling and multi-threaded document chunking.
+- **🎛️ Performance Profiles**: Switch between `fast`, `balanced`, and `quality` inference speeds on the fly.
+- **🎨 Beautiful Terminal UI**: Rich markdown rendering, ASCII art, and intuitive progress bars.
+
+---
+
+## 📦 Installation
+
+DocPilot is available on PyPI! You can install it globally using `pip`, `uv`, or `pipx`.
+
+```bash
+# Recommended: Install using uv or pipx
+uv tool install docpilot-cli
+
+# Or via standard pip
+pip install docpilot-cli
+
+# Optional: Add PDF parsing support
+pip install "docpilot-cli[pdf]"
+```
+
+### Prerequisites
+1. **Python 3.12+**
+2. **[Ollama](https://ollama.com/)**: Installed and running in the background.
+3. Pull your preferred models:
+```bash
+ollama pull qwen2.5:latest
+ollama pull mxbai-embed-large:335m
+```
+
+---
+
+## 🛠️ Quick Start
+
+The very first time you run a DocPilot command, it will launch the **Interactive Setup Wizard** to help you configure your chat and embedding models. 
+
+### 1. Ingest Knowledge
+Point DocPilot to any documentation site, sitemap, PDF, or CSV:
+
+```bash
+# Crawl a website
+docpilot ingest "https://docs.python.org/3/" --max-pages 100 --workers 16
+
+# Ingest a local PDF
+docpilot ingest "./docs/engineering_handbook.pdf"
+
+# Ingest a CSV
+docpilot ingest "./data/faq.csv"
+```
+
+### 2. Ask Questions
+Query your newly created local knowledge base:
+
+```bash
+docpilot ask "How do I create a virtual environment?"
+```
+
+---
+
+## 🧰 CLI Command Reference
+
+Manage your configuration, models, and local database with ease.
+
+### `docpilot setup`
+Re-run the interactive setup wizard at any time to change your default models.
+
+### `docpilot clear`
+Wipe your local Chroma vector database to start fresh. Prompts for safety confirmation.
+
+### `docpilot speed [profile]`
+Adjust the retrieval and generation settings for your desired use case.
+- `fast`: Lower latency, shorter context limits.
+- `balanced`: Default trade-off.
+- `quality`: Larger context, more comprehensive answers, slower inference.
+
+### `docpilot model`
+Manually manage your Ollama models without the interactive setup wizard.
+```bash
+docpilot model list
+docpilot model set <chat-model>
+docpilot model setembed <embedding-model>
+```
+
+### `docpilot render`
+Parse and beautifully render any markdown file or text string directly in your terminal.
+
+### `docpilot show`
+Display your current project version and configuration in beautiful ASCII art.
+
+---
+
+## 🏗️ Architecture Under the Hood
+
+DocPilot employs an optimized RAG (Retrieval-Augmented Generation) pipeline:
+
+1. **Ingestion**: Native Python extractors (BeautifulSoup4, `csv`, `pypdf`) parse the raw data.
+2. **Chunking**: Multi-threaded chunkers slice the documents into semantically coherent pieces.
+3. **Embedding**: `langchain-ollama` creates local vector embeddings via Ollama.
+4. **Storage**: `chromadb` persistently stores vectors on disk at `~/.docpilot/chroma_langchain_db`.
+5. **Retrieval**: User queries are embedded, matched via similarity search, and fed into a system prompt.
+6. **Generation**: The designated Ollama chat model generates the final response streamed to the terminal using `rich`.
+
+---
+
+## 🤝 Contributing
+
+Contributions are welcome! If you are using DocPilot for your daily workflows or in a hackathon, feel free to open issues and pull requests.
+
+To set up a local development environment:
+```bash
+git clone https://github.com/yourusername/docpilot.git
+cd docpilot
+uv pip install -e ".[dev]"
+uv run pytest
+```
+
+---
+
+## 📄 License
+
+This project is licensed under the MIT License.

docpilot_cli-1.0.0/README.md

@@ -0,0 +1,138 @@
+# 🚀 DocPilot CLI
+
+![PyPI - Version](https://img.shields.io/pypi/v/docpilot-cli?color=blue&style=flat-square)
+![Python Version](https://img.shields.io/badge/python-3.12%2B-blue?style=flat-square)
+![License](https://img.shields.io/badge/license-MIT-green?style=flat-square)
+
+**DocPilot** is a lightning-fast, local-first CLI for document ingestion and interactive question-answering. Powered by [Ollama](https://ollama.com/) and Chroma, it allows you to ingest websites, PDFs, and CSVs directly from your terminal and chat with your documents—keeping 100% of your data safely on your own machine.
+
+It’s built for practical developer workflows: crawl sites concurrently, prepare chunks with multi-threading, and iterate rapidly without ever paying for a cloud API.
+
+---
+
+## ✨ Features
+
+- **🔒 100% Local**: No data ever leaves your machine. Powered by Ollama.
+- **⚡ Interactive Setup Wizard**: Get up and running instantly with smart model auto-detection.
+- **🌐 Universal Ingestion**: Seamlessly ingest Website URLs, XML Sitemaps, PDFs, and CSVs.
+- **🚀 Concurrent Processing**: Lightning-fast crawling and multi-threaded document chunking.
+- **🎛️ Performance Profiles**: Switch between `fast`, `balanced`, and `quality` inference speeds on the fly.
+- **🎨 Beautiful Terminal UI**: Rich markdown rendering, ASCII art, and intuitive progress bars.
+
+---
+
+## 📦 Installation
+
+DocPilot is available on PyPI! You can install it globally using `pip`, `uv`, or `pipx`.
+
+```bash
+# Recommended: Install using uv or pipx
+uv tool install docpilot-cli
+
+# Or via standard pip
+pip install docpilot-cli
+
+# Optional: Add PDF parsing support
+pip install "docpilot-cli[pdf]"
+```
+
+### Prerequisites
+1. **Python 3.12+**
+2. **[Ollama](https://ollama.com/)**: Installed and running in the background.
+3. Pull your preferred models:
+```bash
+ollama pull qwen2.5:latest
+ollama pull mxbai-embed-large:335m
+```
+
+---
+
+## 🛠️ Quick Start
+
+The very first time you run a DocPilot command, it will launch the **Interactive Setup Wizard** to help you configure your chat and embedding models. 
+
+### 1. Ingest Knowledge
+Point DocPilot to any documentation site, sitemap, PDF, or CSV:
+
+```bash
+# Crawl a website
+docpilot ingest "https://docs.python.org/3/" --max-pages 100 --workers 16
+
+# Ingest a local PDF
+docpilot ingest "./docs/engineering_handbook.pdf"
+
+# Ingest a CSV
+docpilot ingest "./data/faq.csv"
+```
+
+### 2. Ask Questions
+Query your newly created local knowledge base:
+
+```bash
+docpilot ask "How do I create a virtual environment?"
+```
+
+---
+
+## 🧰 CLI Command Reference
+
+Manage your configuration, models, and local database with ease.
+
+### `docpilot setup`
+Re-run the interactive setup wizard at any time to change your default models.
+
+### `docpilot clear`
+Wipe your local Chroma vector database to start fresh. Prompts for safety confirmation.
+
+### `docpilot speed [profile]`
+Adjust the retrieval and generation settings for your desired use case.
+- `fast`: Lower latency, shorter context limits.
+- `balanced`: Default trade-off.
+- `quality`: Larger context, more comprehensive answers, slower inference.
+
+### `docpilot model`
+Manually manage your Ollama models without the interactive setup wizard.
+```bash
+docpilot model list
+docpilot model set <chat-model>
+docpilot model setembed <embedding-model>
+```
+
+### `docpilot render`
+Parse and beautifully render any markdown file or text string directly in your terminal.
+
+### `docpilot show`
+Display your current project version and configuration in beautiful ASCII art.
+
+---
+
+## 🏗️ Architecture Under the Hood
+
+DocPilot employs an optimized RAG (Retrieval-Augmented Generation) pipeline:
+
+1. **Ingestion**: Native Python extractors (BeautifulSoup4, `csv`, `pypdf`) parse the raw data.
+2. **Chunking**: Multi-threaded chunkers slice the documents into semantically coherent pieces.
+3. **Embedding**: `langchain-ollama` creates local vector embeddings via Ollama.
+4. **Storage**: `chromadb` persistently stores vectors on disk at `~/.docpilot/chroma_langchain_db`.
+5. **Retrieval**: User queries are embedded, matched via similarity search, and fed into a system prompt.
+6. **Generation**: The designated Ollama chat model generates the final response streamed to the terminal using `rich`.
+
+---
+
+## 🤝 Contributing
+
+Contributions are welcome! If you are using DocPilot for your daily workflows or in a hackathon, feel free to open issues and pull requests.
+
+To set up a local development environment:
+```bash
+git clone https://github.com/yourusername/docpilot.git
+cd docpilot
+uv pip install -e ".[dev]"
+uv run pytest
+```
+
+---
+
+## 📄 License
+
+This project is licensed under the MIT License.

docpilot_cli-1.0.0/pyproject.toml

@@ -0,0 +1,41 @@
+[project]
+name = "docpilot-cli"
+version = "1.0.0"
+description = "A local-first RAG pipeline CLI tool"
+readme = "README.md"
+requires-python = ">=3.12"
+dependencies = [
+    "bs4>=0.0.2",
+    "beautifulsoup4>=4.12.0",
+    "langchain>=0.2.0",
+    "langchain-chroma>=0.1.0",
+    "langchain-ollama>=0.1.0",
+    "ollama>=0.2.0",
+    "pandas>=2.0.0",
+    "typer>=0.12.0",
+    "httpx>=0.27.0",
+    "tomli-w>=1.0.0",
+    "pypdf>=5.0.0",
+    "pyfiglet>=0.8.0",
+    "rich>=13.0.0",
+]
+
+[project.scripts]
+docpilot = "docpilot.cli:app"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[build-system]
+requires = ["setuptools>=68", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0.0",
+    "pytest-mock>=3.12.0",
+    "ruff>=0.4.0",
+]
+
+[tool.ruff]
+line-length = 120

docpilot_cli-1.0.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

docpilot_cli-1.0.0/src/docpilot/chat.py

@@ -0,0 +1,66 @@
+from langchain_ollama.llms import OllamaLLM as Ollama
+from langchain_core.prompts import ChatPromptTemplate
+from .embed import vectorstore
+import os
+from . import store
+
+config = store.load_config()
+retrieval_k = int(config.get("retrieval_k", 6))
+max_context_chars = int(config.get("max_context_chars", 3500))
+max_doc_chars = int(config.get("max_doc_chars", 700))
+
+model = Ollama(
+    model=config.get("default_model", "deepseek-coder-v2"),
+    num_predict=int(config.get("num_predict", 192)),
+    num_ctx=int(config.get("num_ctx", 2048)),
+    num_thread=int(config.get("num_thread", max(1, (os.cpu_count() or 4) - 1))),
+    temperature=float(config.get("temperature", 0.1)),
+)
+
+retriever = vectorstore.as_retriever(search_kwargs={"k": retrieval_k})
+template = """
+You are an assistant for answering questions based on the following ingested documents. 
+Use the information in the documents to answer the question as best as you can. 
+If you don't know the answer, say you don't know.
+Always use the information in the documents and never make up an answer.
+Here are some relevant docs: {reviews}
+Here is the question to answer: {question}
+"""
+prompt = ChatPromptTemplate.from_template(template)
+
+chain = prompt | model
+
+
+def _build_bounded_context(docs, max_total_chars: int = 6000, max_doc_chars: int = 1200) -> str:
+    lines = []
+    total = 0
+    for doc in docs:
+        doc_text = doc.page_content.strip()
+        if len(doc_text) > max_doc_chars:
+            doc_text = doc_text[:max_doc_chars].rsplit(" ", 1)[0].strip() or doc_text[:max_doc_chars]
+        line = f"[ID: {doc.id}] {doc_text}"
+        if total + len(line) > max_total_chars:
+            break
+        lines.append(line)
+        total += len(line)
+    return "\n".join(lines)
+
+
+def askai(question):
+    rag = retriever.invoke(question)
+    rag_text = _build_bounded_context(
+        rag,
+        max_total_chars=max_context_chars,
+        max_doc_chars=max_doc_chars,
+    )
+    result = chain.invoke({"reviews": rag_text, "question": question})
+    return result
+
+
+if __name__ == "__main__":
+    while True:
+        question = input("Ask a question (or type 'exit'): ")
+        if question.lower() == "exit":
+            break
+        result = askai(question)
+        print(result, "\n\n")