docusense-mcp 0.1.0__tar.gz

docusense_mcp-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Harsh Kumar Sharma
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

docusense_mcp-0.1.0/PKG-INFO

@@ -0,0 +1,143 @@
+Metadata-Version: 2.4
+Name: docusense-mcp
+Version: 0.1.0
+Summary: MCP server that lets AI agents read your PDFs and docs in sections, not wholesale: ~20x fewer tokens, hash-verified citations, version diffs
+Keywords: mcp,model-context-protocol,claude,rag,pdf,documents,citations,llm,tokens,agents
+Author: hksharma2004
+Author-email: hksharma2004 <hksharma2004@users.noreply.github.com>
+License-Expression: MIT
+License-File: LICENSE
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Indexing
+Requires-Dist: pymupdf>=1.24
+Requires-Dist: markdown-it-py>=3.0
+Requires-Dist: pydantic-settings>=2.4
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: typer>=0.12
+Requires-Dist: httpx>=0.27
+Requires-Dist: rich>=13.7
+Requires-Dist: mcp>=1.10
+Requires-Dist: bm25s>=0.2
+Requires-Dist: pystemmer>=2.2
+Requires-Dist: fastembed>=0.4
+Requires-Dist: numpy>=1.26
+Requires-Dist: fastapi>=0.115
+Requires-Dist: uvicorn>=0.30
+Requires-Dist: anthropic>=1.0 ; extra == 'anthropic'
+Requires-Dist: psycopg[binary,pool]>=3.2 ; extra == 'postgres'
+Requires-Dist: pgvector>=0.3 ; extra == 'postgres'
+Requires-Python: >=3.12
+Project-URL: Homepage, https://github.com/hksharma2004/docu-sense
+Project-URL: Issues, https://github.com/hksharma2004/docu-sense/issues
+Provides-Extra: anthropic
+Provides-Extra: postgres
+Description-Content-Type: text/markdown
+
+# DocuSense
+
+**Stop pasting whole documents into your agent.** DocuSense is a local MCP server that lets Claude, Cursor
+and other agents read your PDFs, Word files and docs *by section and by claim* — so they load the 300
+tokens that answer the question, not the 50,000-token file. Every passage comes back with a citation
+to its exact bytes and a SHA-256 you can verify.
+
+```
+pasting the docs   ████████████████████████████████████████  160,179 tokens
+vector RAG (top-5) ████                                         15,765 tokens
+DocuSense          ██                                            6,869 tokens   → 23x less than pasting, 2.3x less than RAG
+```
+<sub>Three real tasks over RFC specs and a policy file; measured by [`scripts/token_demo.py`](scripts/token_demo.py) on the actual tool outputs.</sub>
+
+## Why not just RAG?
+
+Chunk-and-embed retrieval answers "find text like this". Most document questions are not that:
+
+| Question | Paste / file read | Vector RAG | DocuSense |
+|---|---|---|---|
+| *"What's the max PKCE verifier length? Cite it."* | 9,090 tok | 5,362 tok of nearby chunks | **478 tok**: the one paragraph + verified citation |
+| *"What changed between v1 and v2 of the spec?"* | 97,895 tok (both versions) | 5,292 tok, can't actually diff | **2,776 tok**: claim-level diff (`diff_versions`) |
+| *"Does our policy conflict with the standard?"* | 53,194 tok | 5,111 tok, hopes the right chunks surface | **3,615 tok**: `find_contradictions`, both sides cited |
+
+What makes the difference:
+
+- **Structure-aware.** Documents are parsed into a section tree, paragraphs, tables, definitions,
+  cross-references and normative claims (*MUST / MUST NOT / numeric limits*). Agents navigate like a
+  person with a table of contents instead of re-reading everything.
+- **Budgeted responses.** Every tool takes `max_tokens`; outputs are compact JSON with no padding.
+- **Versions are first-class.** Re-adding a changed file makes `v2`. Ask what changed, who changed it
+  (`blame`), or which other documents are affected — without loading either version.
+- **Verifiable.** Each passage is `ds://doc@version#§section:b<start>-<end>` + SHA-256. `get_evidence`
+  re-hashes against the stored bytes, so a hallucinated quote can't pass as a citation.
+- **Local and free.** Embedded SQLite + local embeddings. No API key, no server, nothing leaves your machine.
+- **Proves its own value.** `docusense savings` shows how many tokens your agents actually saved.
+
+## Get started (2 minutes)
+
+```bash
+uv tool install docusense-mcp           # or: pipx install docusense-mcp
+docusense setup                         # registers with Claude Code (also: claude-desktop | cursor | print)
+docusense add ~/work/specs handbook.pdf # PDF, .docx, HTML, Markdown, .txt — folders scanned recursively
+```
+
+Then ask your agent about your documents. Later, see what it saved (example output):
+
+```console
+$ docusense savings
+12 agent sessions · 87 tool calls
+read via DocuSense:         41,302 tokens
+pasting the docs:          912,440 tokens
+saved:                     871,138 tokens (22.1x less context)
+```
+
+The first indexing run downloads the local embedding model (`bge-small-en-v1.5`, ~130 MB) once.
+Claude Desktop users can also install the one-click `docusense.mcpb` bundle from the
+[releases page](https://github.com/hksharma2004/docu-sense/releases).
+
+<!-- mcp-name: io.github.hksharma2004/docusense -->
+
+
+Agents can add files themselves with the `add_document` tool (restricted to `DOCUSENSE_ADD_ROOTS`,
+default: your home folder).
+
+## Good fits
+
+- **Engineers** working against long specs, RFCs, API references, design docs and ADRs in Claude Code / Cursor.
+- **Compliance and policy work**: checking internal policies against standards, tracking what changed between revisions.
+- **Contracts, handbooks, research papers**: precise quotes with page numbers instead of paraphrase.
+- **Any long-running agent** whose context fills up from repeatedly reading the same large files.
+
+Less useful for: a handful of short files that fit comfortably in context anyway. The tool list costs
+~3.2k tokens per session (~2.3k with `DOCUSENSE_TOOLS=core`), which one avoided document read repays.
+
+## Tools
+
+| Purpose | Tools |
+|---|---|
+| Orient | `list_documents`, `get_section_map`, `read_section` |
+| Find | `search` (hybrid BM25 + embeddings), `search_claims`, `lookup_definition`, `resolve_reference` |
+| Verify | `get_evidence`, `verify_span`, `assemble_context` (packs evidence into a token budget) |
+| Change over time | `list_versions`, `diff_versions`, `blame`, `find_affected_documents`, `find_contradictions` |
+| Ingest | `add_document`, `ingest_status` |
+
+Plus resources (`docusense://documents`, canonical text per version) and prompts (`answer_with_evidence`,
+`what_changed`, `audit_contradictions`). `DOCUSENSE_TOOLS=core` exposes 11 tools for tight budgets.
+
+## Accuracy
+
+On a 14-question agent benchmark over the RFC corpus (multi-hop, structural and temporal questions),
+an agent using DocuSense answered 100% correctly with 93% citation hit-rate
+([`bench/`](bench/), `nvidia/nemotron-3-ultra` via OpenRouter). Retrieval-only eval on non-RFC PDFs
+(IRS Pub 463, the Transformer paper): hybrid hit@5 0.85, MRR 0.65 ([`scripts/eval_retrieval.py`](scripts/eval_retrieval.py)).
+
+## More
+
+- Other ways to run it: `docusense serve-mcp --http` (Streamable HTTP, several agents sharing one store;
+  non-loopback requires `DOCUSENSE_HTTP_TOKEN`), Postgres + pgvector backend, CLI search.
+- Configuration, development, provenance model and releasing: [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md).
+- Inspect the tools: `npx @modelcontextprotocol/inspector docusense serve-mcp`.
+
+MIT licensed. Originally built as a B.Tech IT project (USICT, GGSIPU); see [PLAN.md](PLAN.md).

docusense_mcp-0.1.0/README.md

@@ -0,0 +1,103 @@
+# DocuSense
+
+**Stop pasting whole documents into your agent.** DocuSense is a local MCP server that lets Claude, Cursor
+and other agents read your PDFs, Word files and docs *by section and by claim* — so they load the 300
+tokens that answer the question, not the 50,000-token file. Every passage comes back with a citation
+to its exact bytes and a SHA-256 you can verify.
+
+```
+pasting the docs   ████████████████████████████████████████  160,179 tokens
+vector RAG (top-5) ████                                         15,765 tokens
+DocuSense          ██                                            6,869 tokens   → 23x less than pasting, 2.3x less than RAG
+```
+<sub>Three real tasks over RFC specs and a policy file; measured by [`scripts/token_demo.py`](scripts/token_demo.py) on the actual tool outputs.</sub>
+
+## Why not just RAG?
+
+Chunk-and-embed retrieval answers "find text like this". Most document questions are not that:
+
+| Question | Paste / file read | Vector RAG | DocuSense |
+|---|---|---|---|
+| *"What's the max PKCE verifier length? Cite it."* | 9,090 tok | 5,362 tok of nearby chunks | **478 tok**: the one paragraph + verified citation |
+| *"What changed between v1 and v2 of the spec?"* | 97,895 tok (both versions) | 5,292 tok, can't actually diff | **2,776 tok**: claim-level diff (`diff_versions`) |
+| *"Does our policy conflict with the standard?"* | 53,194 tok | 5,111 tok, hopes the right chunks surface | **3,615 tok**: `find_contradictions`, both sides cited |
+
+What makes the difference:
+
+- **Structure-aware.** Documents are parsed into a section tree, paragraphs, tables, definitions,
+  cross-references and normative claims (*MUST / MUST NOT / numeric limits*). Agents navigate like a
+  person with a table of contents instead of re-reading everything.
+- **Budgeted responses.** Every tool takes `max_tokens`; outputs are compact JSON with no padding.
+- **Versions are first-class.** Re-adding a changed file makes `v2`. Ask what changed, who changed it
+  (`blame`), or which other documents are affected — without loading either version.
+- **Verifiable.** Each passage is `ds://doc@version#§section:b<start>-<end>` + SHA-256. `get_evidence`
+  re-hashes against the stored bytes, so a hallucinated quote can't pass as a citation.
+- **Local and free.** Embedded SQLite + local embeddings. No API key, no server, nothing leaves your machine.
+- **Proves its own value.** `docusense savings` shows how many tokens your agents actually saved.
+
+## Get started (2 minutes)
+
+```bash
+uv tool install docusense-mcp           # or: pipx install docusense-mcp
+docusense setup                         # registers with Claude Code (also: claude-desktop | cursor | print)
+docusense add ~/work/specs handbook.pdf # PDF, .docx, HTML, Markdown, .txt — folders scanned recursively
+```
+
+Then ask your agent about your documents. Later, see what it saved (example output):
+
+```console
+$ docusense savings
+12 agent sessions · 87 tool calls
+read via DocuSense:         41,302 tokens
+pasting the docs:          912,440 tokens
+saved:                     871,138 tokens (22.1x less context)
+```
+
+The first indexing run downloads the local embedding model (`bge-small-en-v1.5`, ~130 MB) once.
+Claude Desktop users can also install the one-click `docusense.mcpb` bundle from the
+[releases page](https://github.com/hksharma2004/docu-sense/releases).
+
+<!-- mcp-name: io.github.hksharma2004/docusense -->
+
+
+Agents can add files themselves with the `add_document` tool (restricted to `DOCUSENSE_ADD_ROOTS`,
+default: your home folder).
+
+## Good fits
+
+- **Engineers** working against long specs, RFCs, API references, design docs and ADRs in Claude Code / Cursor.
+- **Compliance and policy work**: checking internal policies against standards, tracking what changed between revisions.
+- **Contracts, handbooks, research papers**: precise quotes with page numbers instead of paraphrase.
+- **Any long-running agent** whose context fills up from repeatedly reading the same large files.
+
+Less useful for: a handful of short files that fit comfortably in context anyway. The tool list costs
+~3.2k tokens per session (~2.3k with `DOCUSENSE_TOOLS=core`), which one avoided document read repays.
+
+## Tools
+
+| Purpose | Tools |
+|---|---|
+| Orient | `list_documents`, `get_section_map`, `read_section` |
+| Find | `search` (hybrid BM25 + embeddings), `search_claims`, `lookup_definition`, `resolve_reference` |
+| Verify | `get_evidence`, `verify_span`, `assemble_context` (packs evidence into a token budget) |
+| Change over time | `list_versions`, `diff_versions`, `blame`, `find_affected_documents`, `find_contradictions` |
+| Ingest | `add_document`, `ingest_status` |
+
+Plus resources (`docusense://documents`, canonical text per version) and prompts (`answer_with_evidence`,
+`what_changed`, `audit_contradictions`). `DOCUSENSE_TOOLS=core` exposes 11 tools for tight budgets.
+
+## Accuracy
+
+On a 14-question agent benchmark over the RFC corpus (multi-hop, structural and temporal questions),
+an agent using DocuSense answered 100% correctly with 93% citation hit-rate
+([`bench/`](bench/), `nvidia/nemotron-3-ultra` via OpenRouter). Retrieval-only eval on non-RFC PDFs
+(IRS Pub 463, the Transformer paper): hybrid hit@5 0.85, MRR 0.65 ([`scripts/eval_retrieval.py`](scripts/eval_retrieval.py)).
+
+## More
+
+- Other ways to run it: `docusense serve-mcp --http` (Streamable HTTP, several agents sharing one store;
+  non-loopback requires `DOCUSENSE_HTTP_TOKEN`), Postgres + pgvector backend, CLI search.
+- Configuration, development, provenance model and releasing: [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md).
+- Inspect the tools: `npx @modelcontextprotocol/inspector docusense serve-mcp`.
+
+MIT licensed. Originally built as a B.Tech IT project (USICT, GGSIPU); see [PLAN.md](PLAN.md).

docusense_mcp-0.1.0/pyproject.toml

@@ -0,0 +1,82 @@
+[project]
+name = "docusense-mcp"
+version = "0.1.0"
+description = "MCP server that lets AI agents read your PDFs and docs in sections, not wholesale: ~20x fewer tokens, hash-verified citations, version diffs"
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.12"
+keywords = [
+    "mcp",
+    "model-context-protocol",
+    "claude",
+    "rag",
+    "pdf",
+    "documents",
+    "citations",
+    "llm",
+    "tokens",
+    "agents",
+]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Text Processing :: Indexing",
+]
+dependencies = [
+    "pymupdf>=1.24",
+    "markdown-it-py>=3.0",
+    "pydantic-settings>=2.4",
+    "pyyaml>=6.0",
+    "typer>=0.12",
+    "httpx>=0.27",
+    "rich>=13.7",
+    "mcp>=1.10",
+    "bm25s>=0.2",
+    "pystemmer>=2.2",
+    "fastembed>=0.4",
+    "numpy>=1.26",
+    "fastapi>=0.115",
+    "uvicorn>=0.30",
+]
+
+[[project.authors]]
+name = "hksharma2004"
+email = "hksharma2004@users.noreply.github.com"
+
+[project.urls]
+Homepage = "https://github.com/hksharma2004/docu-sense"
+Issues = "https://github.com/hksharma2004/docu-sense/issues"
+
+[project.optional-dependencies]
+postgres = [
+    "psycopg[binary,pool]>=3.2",
+    "pgvector>=0.3",
+]
+anthropic = ["anthropic>=1.0"]
+
+[project.scripts]
+docusense = "docusense.cli:main"
+docusense-mcp = "docusense.cli:main"
+
+[dependency-groups]
+dev = [
+    "pytest>=8.0",
+    "psycopg[binary,pool]>=3.2",
+    "pgvector>=0.3",
+    "anthropic>=1.0",
+]
+
+[tool.uv.build-backend]
+module-name = "docusense"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+
+[build-system]
+requires = ["uv_build>=0.11.8,<0.12.0"]
+build-backend = "uv_build"

docusense_mcp-0.1.0/pyproject.toml.orig

@@ -0,0 +1,68 @@
+[project]
+name = "docusense-mcp"
+version = "0.1.0"
+description = "MCP server that lets AI agents read your PDFs and docs in sections, not wholesale: ~20x fewer tokens, hash-verified citations, version diffs"
+authors = [
+    { name = "hksharma2004", email = "hksharma2004@users.noreply.github.com" }
+]
+readme = "README.md"
+license = "MIT"
+license-files = ["LICENSE"]
+requires-python = ">=3.12"
+keywords = ["mcp", "model-context-protocol", "claude", "rag", "pdf", "documents", "citations", "llm", "tokens", "agents"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Text Processing :: Indexing",
+]
+dependencies = [
+    "pymupdf>=1.24",
+    "markdown-it-py>=3.0",
+    "pydantic-settings>=2.4",
+    "pyyaml>=6.0",
+    "typer>=0.12",
+    "httpx>=0.27",
+    "rich>=13.7",
+    "mcp>=1.10",
+    "bm25s>=0.2",
+    "pystemmer>=2.2",
+    "fastembed>=0.4",
+    "numpy>=1.26",
+    "fastapi>=0.115",
+    "uvicorn>=0.30",
+]
+
+[project.urls]
+Homepage = "https://github.com/hksharma2004/docu-sense"
+Issues = "https://github.com/hksharma2004/docu-sense/issues"
+
+[project.optional-dependencies]
+# Postgres + pgvector backend (team / hosted). The default backend is embedded SQLite.
+postgres = [
+    "psycopg[binary,pool]>=3.2",
+    "pgvector>=0.3",
+]
+
+# Anthropic (Claude) as the optional LLM provider for extraction fallback / contradiction judging.
+anthropic = ["anthropic>=1.0"]
+
+[project.scripts]
+docusense = "docusense.cli:main"
+docusense-mcp = "docusense.cli:main"
+
+[dependency-groups]
+dev = ["pytest>=8.0", "psycopg[binary,pool]>=3.2", "pgvector>=0.3", "anthropic>=1.0"]
+
+[tool.uv.build-backend]
+module-name = "docusense"
+
+[build-system]
+requires = ["uv_build>=0.11.8,<0.12.0"]
+build-backend = "uv_build"
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]

docusense_mcp-0.1.0/src/docusense/__init__.py

@@ -0,0 +1 @@
+"""DocuSense: a verifiable and version-native document layer for AI agents."""

docusense_mcp-0.1.0/src/docusense/api.py

@@ -0,0 +1,107 @@
+"""REST API for the viewer. Thin wrapper over services.DocuSense.
+
+Run:  uv run docusense serve-api   (http://localhost:8000, docs at /docs)
+"""
+
+from contextlib import asynccontextmanager
+
+from fastapi import FastAPI, HTTPException, Query, Request
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi.responses import JSONResponse
+
+from docusense.db.conn import ConnectionPool
+from docusense.services import DocuSense, NotFound
+
+pool: ConnectionPool | None = None
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    global pool
+    pool = ConnectionPool()
+    yield
+    pool.close()
+
+
+app = FastAPI(title="DocuSense API", version="0.1.0", lifespan=lifespan)
+app.add_middleware(CORSMiddleware, allow_origins=["http://localhost:3000"], allow_methods=["GET"], allow_headers=["*"])
+
+
+@app.exception_handler(NotFound)
+async def not_found(_: Request, exc: NotFound):
+    return JSONResponse(status_code=404, content={"detail": str(exc)})
+
+
+@app.exception_handler(ValueError)
+async def bad_request(_: Request, exc: ValueError):
+    return JSONResponse(status_code=400, content={"detail": str(exc)})
+
+
+def run(fn, *args, **kwargs):
+    with pool.connection() as conn:
+        return fn(DocuSense(conn), *args, **kwargs)
+
+
+# Sync endpoints run in FastAPI's thread pool, so blocking DB calls don't stall the event loop.
+
+@app.get("/api/health")
+def health():
+    return run(lambda ds: {"ok": True, "versions": ds.conn.execute("SELECT count(*) AS n FROM versions").fetchone()["n"]})
+
+
+@app.get("/api/documents")
+def documents():
+    return run(lambda ds: ds.list_documents())
+
+
+@app.get("/api/documents/{doc}/versions")
+def versions(doc: str):
+    return run(lambda ds: ds.list_versions(doc))
+
+
+@app.get("/api/documents/{doc}/affected")
+def affected(doc: str):
+    return run(lambda ds: ds.find_affected_documents(doc, max_tokens=20000))
+
+
+@app.get("/api/versions/{version_id}")
+def version_document(version_id: str):
+    return run(lambda ds: ds.version_document(version_id))
+
+
+@app.get("/api/spans/{span_id}")
+def span(span_id: int):
+    return run(lambda ds: ds.get_evidence(span_id))
+
+
+@app.get("/api/spans/{span_id}/verify")
+def verify(span_id: int, deep: bool = False):
+    return run(lambda ds: ds.verify_span(span_id, deep))
+
+
+@app.get("/api/spans/{span_id}/blame")
+def blame(span_id: int):
+    return run(lambda ds: ds.blame(span_id))
+
+
+@app.get("/api/spans/{span_id}/references")
+def references(span_id: int):
+    return run(lambda ds: ds.resolve_reference(span_id=span_id, max_tokens=20000))
+
+
+@app.get("/api/diff")
+def diff(doc: str, from_version: str | None = Query(None, alias="from"),
+         to_version: str | None = Query(None, alias="to"), level: str = "section"):
+    return run(lambda ds: ds.diff_versions(doc, from_version, to_version, level, max_tokens=200000))
+
+
+@app.get("/api/search")
+def search(q: str, mode: str = "hybrid", doc: str | None = None, k: int = 10):
+    if not q.strip():
+        raise HTTPException(400, "query is empty")
+    return run(lambda ds: ds.search(q, mode, doc, k, max_tokens=20000))
+
+
+@app.get("/api/contradictions")
+def contradictions(doc: str | None = None, history: bool = False, rejected: bool = False, k: int = 50):
+    return run(lambda ds: ds.find_contradictions(doc, history, rejected, k, max_tokens=200000))