ctxvault 0.1.0__tar.gz

ctxvault-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Filippo Venturini
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ctxvault-0.1.0/PKG-INFO

@@ -0,0 +1,186 @@
+Metadata-Version: 2.4
+Name: ctxvault
+Version: 0.1.0
+Summary: ctxvault is a local-first knowledge vault that indexes your documents, generates embeddings, and enables fast semantic search via CLI or API. Designed for personal knowledge bases, RAG pipelines, and AI agents.
+Author-email: Filippo Venturini <filippoventurini00@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/Filippo-Venturini/ctx-vault
+Project-URL: Repository, https://github.com/Filippo-Venturini/ctx-vault
+Project-URL: Issues, https://github.com/Filippo-Venturini/ctx-vault/issues
+Keywords: rag,retrieval-augmented-generation,semantic-search,embeddings,vector-database,chroma,llm,ai,knowledge-base,document-search,local-ai,developer-tools,cli,fastapi
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Database
+Classifier: Topic :: Text Processing :: Indexing
+Classifier: Topic :: Utilities
+Classifier: Environment :: Console
+Classifier: Framework :: FastAPI
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: typer>=0.20.0
+Requires-Dist: chromadb>=1.3.0
+Requires-Dist: sentence-transformers>=5.0.0
+Requires-Dist: fastapi>=0.124.0
+Requires-Dist: uvicorn>=0.38.0
+Requires-Dist: python-dotenv>=1.2.0
+Requires-Dist: pydantic>=2.12.0
+Requires-Dist: rich>=14.0.0
+Requires-Dist: pypdf>=6.0.0
+Requires-Dist: python-docx>=1.0.0
+Requires-Dist: markdown>=3.0.0
+Requires-Dist: strip-tags>=0.5.0
+Provides-Extra: dev
+Requires-Dist: pytest>=9.0.0; extra == "dev"
+Requires-Dist: pytest-mock>=3.15.0; extra == "dev"
+Requires-Dist: pytest-anyio>=0.0.0; extra == "dev"
+Requires-Dist: black>=25.0.0; extra == "dev"
+Requires-Dist: isort>=7.0.0; extra == "dev"
+Requires-Dist: pre-commit>=3.5.0; extra == "dev"
+Dynamic: license-file
+
+# CtxVault
+
+Local semantic search vault for LLMs.
+
+CtxVault lets you index documents locally, generate embeddings, and query them with semantic search.  
+Designed as a lightweight RAG backend for agents, scripts, and LLM workflows.
+
+## Why CtxVault
+
+- 100% local (no cloud, no data sharing)
+- simple CLI
+- works offline
+- persistent vector store (Chroma)
+- file-based workflow
+- agent/API ready (future)
+
+Ideal for:
+- personal knowledge bases
+- private documents
+- local RAG pipelines
+- AI agents needing contextual memory
+
+---
+
+## Install
+
+Python 3.10+
+
+```bash
+pip install -e .
+````
+
+---
+
+## Quickstart
+
+Initialize a vault:
+
+```bash
+ctxvault init ./my-vault
+```
+
+Index files or folders:
+
+```bash
+ctxvault index ./my-vault/docs
+```
+
+Query:
+
+```bash
+ctxvault query "what is project Orion?"
+```
+
+---
+
+## CLI Commands
+
+### init
+
+Initialize a vault directory.
+
+```bash
+ctxvault init <path>
+```
+
+---
+
+### index
+
+Index a file or directory.
+
+```bash
+ctxvault index <path>
+```
+
+---
+
+### query
+
+Semantic search inside the vault.
+
+```bash
+ctxvault query "<text>"
+```
+
+---
+
+### delete
+
+Remove a document from the vault.
+
+```bash
+ctxvault delete <path>
+```
+
+---
+
+### reindex
+
+Reindex a document after changes.
+
+```bash
+ctxvault reindex <path>
+```
+
+---
+
+### list
+
+List indexed documents.
+
+```bash
+ctxvault list
+```
+
+---
+
+## Privacy
+
+All processing happens locally.
+No data is sent to external services.
+
+---
+
+## Roadmap
+
+* [x] CLI MVP
+* [ ] FastAPI server
+* [ ] sync and file watcher
+* [ ] multi-vault support
+
+---
+
+## License
+
+MIT

ctxvault-0.1.0/README.md

@@ -0,0 +1,137 @@
+# CtxVault
+
+Local semantic search vault for LLMs.
+
+CtxVault lets you index documents locally, generate embeddings, and query them with semantic search.  
+Designed as a lightweight RAG backend for agents, scripts, and LLM workflows.
+
+## Why CtxVault
+
+- 100% local (no cloud, no data sharing)
+- simple CLI
+- works offline
+- persistent vector store (Chroma)
+- file-based workflow
+- agent/API ready (future)
+
+Ideal for:
+- personal knowledge bases
+- private documents
+- local RAG pipelines
+- AI agents needing contextual memory
+
+---
+
+## Install
+
+Python 3.10+
+
+```bash
+pip install -e .
+````
+
+---
+
+## Quickstart
+
+Initialize a vault:
+
+```bash
+ctxvault init ./my-vault
+```
+
+Index files or folders:
+
+```bash
+ctxvault index ./my-vault/docs
+```
+
+Query:
+
+```bash
+ctxvault query "what is project Orion?"
+```
+
+---
+
+## CLI Commands
+
+### init
+
+Initialize a vault directory.
+
+```bash
+ctxvault init <path>
+```
+
+---
+
+### index
+
+Index a file or directory.
+
+```bash
+ctxvault index <path>
+```
+
+---
+
+### query
+
+Semantic search inside the vault.
+
+```bash
+ctxvault query "<text>"
+```
+
+---
+
+### delete
+
+Remove a document from the vault.
+
+```bash
+ctxvault delete <path>
+```
+
+---
+
+### reindex
+
+Reindex a document after changes.
+
+```bash
+ctxvault reindex <path>
+```
+
+---
+
+### list
+
+List indexed documents.
+
+```bash
+ctxvault list
+```
+
+---
+
+## Privacy
+
+All processing happens locally.
+No data is sent to external services.
+
+---
+
+## Roadmap
+
+* [x] CLI MVP
+* [ ] FastAPI server
+* [ ] sync and file watcher
+* [ ] multi-vault support
+
+---
+
+## License
+
+MIT

ctxvault-0.1.0/pyproject.toml

@@ -0,0 +1,87 @@
+[build-system]
+requires = ["setuptools>=61.0"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "ctxvault"
+version = "0.1.0"
+description = "ctxvault is a local-first knowledge vault that indexes your documents, generates embeddings, and enables fast semantic search via CLI or API. Designed for personal knowledge bases, RAG pipelines, and AI agents."
+readme = "README.md"
+requires-python = ">=3.9"
+license = {text = "MIT"}
+authors = [
+    {name = "Filippo Venturini", email = "filippoventurini00@gmail.com"}
+]
+keywords = [
+    "rag",
+    "retrieval-augmented-generation",
+    "semantic-search",
+    "embeddings",
+    "vector-database",
+    "chroma",
+    "llm",
+    "ai",
+    "knowledge-base",
+    "document-search",
+    "local-ai",
+    "developer-tools",
+    "cli",
+    "fastapi"
+]
+classifiers = [
+    "Development Status :: 3 - Alpha",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Science/Research",
+    "License :: OSI Approved :: MIT License",
+    "Operating System :: OS Independent",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Topic :: Scientific/Engineering :: Artificial Intelligence",
+    "Topic :: Database",
+    "Topic :: Text Processing :: Indexing",
+    "Topic :: Utilities",
+    "Environment :: Console",
+    "Framework :: FastAPI"
+]
+dependencies = [
+    "typer>=0.20.0",
+    "chromadb>=1.3.0",
+    "sentence-transformers>=5.0.0",
+    "fastapi>=0.124.0",
+    "uvicorn>=0.38.0",
+    "python-dotenv>=1.2.0",
+    "pydantic>=2.12.0",
+    "rich>=14.0.0",
+    "pypdf>=6.0.0",
+    "python-docx>=1.0.0",
+    "markdown>=3.0.0",
+    "strip-tags>=0.5.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=9.0.0",
+    "pytest-mock>=3.15.0",
+    "pytest-anyio>=0.0.0",
+    "black>=25.0.0",
+    "isort>=7.0.0",
+    "pre-commit>=3.5.0",
+]
+
+[project.scripts]
+ctxvault = "ctxvault.cli.app:main"
+
+[project.urls]
+Homepage = "https://github.com/Filippo-Venturini/ctx-vault"
+Repository = "https://github.com/Filippo-Venturini/ctx-vault"
+Issues = "https://github.com/Filippo-Venturini/ctx-vault/issues"
+
+[tool.setuptools]
+packages = ["ctxvault"]
+package-dir = {"" = "src"}
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]

ctxvault-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

ctxvault-0.1.0/src/ctxvault/cli/app.py

@@ -0,0 +1,101 @@
+from pathlib import Path
+import typer
+from ctxvault.core import vault
+from ctxvault.core.exceptions import VaultAlreadyExistsError
+
+app = typer.Typer()
+
+@app.command()
+def init(path: str = "."):
+    try:
+        typer.echo(f"Initializing Context Vault at: {path} ...")
+        vault_path, config_path = vault.init_vault(path=path)
+        typer.secho("Context Vault initialized succesfully!", fg=typer.colors.GREEN, bold=True)
+        typer.echo(f"Context Vault path: {vault_path}")
+        typer.echo(f"Config file path: {config_path}")
+    except VaultAlreadyExistsError as e:
+        typer.secho("Warning: Context Vault already initialized in this path!", fg=typer.colors.YELLOW, bold=True)
+        typer.echo(f"Existing vault path: {e.existing_path}")
+        raise typer.Exit(1)
+
+@app.command()
+def index(path: str = "."):
+    indexed_files, skipped_files = vault.index_files(base_path=Path(path))
+
+    for file in indexed_files:
+        typer.secho(f"Indexed: {file}", fg=typer.colors.GREEN)
+
+    for file in skipped_files:
+        typer.secho(f"Skipped: {file}", fg=typer.colors.YELLOW)
+
+    typer.secho(f"Indexed: {len(indexed_files)}", fg=typer.colors.GREEN, bold=True)
+    typer.secho(f"Skipped: {len(skipped_files)}", fg=typer.colors.YELLOW, bold=True)
+
+@app.command()
+def query(text: str = ""):
+    result = vault.query(text=text)
+    if not result.results:
+        typer.secho("No results found.", fg=typer.colors.YELLOW)
+        return
+
+    typer.secho(f"\n Found {len(result.results)} chunks", fg=typer.colors.GREEN, bold=True)
+    typer.echo("─" * 80)
+    
+    for idx, chunk in enumerate(result.results, 1):
+        typer.secho(f"\n[{idx}] ", fg=typer.colors.CYAN, bold=True, nl=False)
+        typer.secho(f"score: {chunk.score:.3f}", fg=typer.colors.MAGENTA)
+        typer.secho(f"    ▸ {chunk.source} ", fg=typer.colors.BLUE, nl=False)
+        typer.echo(f"(chunk {chunk.chunk_index})")
+
+        preview = chunk.text.strip().replace("\n", " ")
+        if len(preview) > 200:
+            preview = preview[:200] + "..."
+        typer.echo(f"    {preview}")
+    
+    typer.echo("\n" + "─" * 80)
+
+@app.command()
+def delete(path: str = "."):
+    deleted_files, skipped_files = vault.delete_files(base_path=Path(path))
+
+    for file in deleted_files:
+        typer.secho(f"Deleted: {file}", fg=typer.colors.RED)
+
+    for file in skipped_files:
+        typer.secho(f"Skipped: {file}", fg=typer.colors.YELLOW)
+
+    typer.secho(f"Deleted: {len(deleted_files)}", fg=typer.colors.RED, bold=True)
+    typer.secho(f"Skipped: {len(skipped_files)}", fg=typer.colors.YELLOW, bold=True)
+
+@app.command()
+def reindex(path: str = "."):
+    reindexed_files, skipped_files = vault.reindex_files(base_path=Path(path))
+
+    for file in reindexed_files:
+        typer.secho(f"Reindexed: {file}", fg=typer.colors.GREEN)
+
+    for file in skipped_files:
+        typer.secho(f"Skipped: {file}", fg=typer.colors.YELLOW)
+
+    typer.secho(f"Reindexed: {len(reindexed_files)}", fg=typer.colors.GREEN, bold=True)
+    typer.secho(f"Skipped: {len(skipped_files)}", fg=typer.colors.YELLOW, bold=True)
+
+@app.command()
+def sync():
+    typer.echo(f"Synchronizing vault")
+
+@app.command()
+def list():
+    documents = vault.list_documents()
+
+    typer.secho(f"\nFound {len(documents)} documents\n", fg=typer.colors.GREEN, bold=True)
+
+    for i in range(len(documents)):
+        typer.echo(f"{i+1}. {documents[i].source} ({documents[i].chunks_count} chunks)")
+
+
+def main():
+    app()
+
+if __name__ == "__main__":
+    main()

ctxvault-0.1.0/src/ctxvault/core/embedding.py

@@ -0,0 +1,13 @@
+from sentence_transformers import SentenceTransformer
+
+MODEL: SentenceTransformer = None
+
+def get_model():
+    global MODEL
+    if MODEL is None:
+        MODEL = SentenceTransformer("all-MiniLM-L6-v2")
+    return MODEL
+
+def embed_list(chunks: list[str])-> list[list[float]]:
+    embeddings = get_model().encode(sentences=chunks)
+    return embeddings

ctxvault-0.1.0/src/ctxvault/core/exceptions.py

@@ -0,0 +1,21 @@
+class UnsupportedFileTypeError(Exception):
+    """Raised when a file type is not supported by the extractor."""
+    pass
+
+class ExtractionError(Exception):
+    """Raised when text extraction fails for reasons other than file type."""
+    pass
+
+class VaultAlreadyExistsError(Exception):
+    """Raised when a Context Vault is already initialized at that path."""
+    def __init__(self, existing_path: str):
+        self.existing_path = existing_path
+        super().__init__(f"Vault already initialized at {existing_path}")
+
+class VaultNotInitializedError(Exception):
+    """Raised when a Context Vault is not initialized at that path."""
+    pass
+
+class FileOutsideVault(Exception):
+    """Raised when try to index a file outside the Context Vault"""
+    pass

ctxvault-0.1.0/src/ctxvault/core/indexer.py

@@ -0,0 +1,26 @@
+from ctxvault.utils.text_extraction import extract_text
+from ctxvault.core.identifiers import get_doc_id
+from ctxvault.utils.chuncking import chunking
+from ctxvault.core.embedding import embed_list
+from ctxvault.storage.chroma_store import add_document, delete_document
+from ctxvault.utils.metadata_builder import build_chunks_metadatas
+
+def index_file(file_path: str)-> dict:
+    text, file_type = extract_text(path=file_path)
+    doc_id = get_doc_id(path=file_path)
+
+    chunks = chunking(text, chunk_size=50)
+
+    embeddings = embed_list(chunks=chunks)
+
+    chunk_ids, metadatas = build_chunks_metadatas(doc_id=doc_id, chunks_size=len(chunks), source=file_path, filetype=file_type)
+
+    add_document(ids=chunk_ids, embeddings=embeddings, metadatas=metadatas, chunks=chunks)
+
+def delete_file(file_path: str)-> None:
+    doc_id = get_doc_id(path=file_path)
+    delete_document(doc_id=doc_id)
+
+def reindex_file(file_path: str)->None:
+    delete_file(file_path=file_path)
+    index_file(file_path=file_path)

ctxvault-0.1.0/src/ctxvault/core/querying.py

@@ -0,0 +1,38 @@
+from ctxvault.core.embedding import embed_list
+from ctxvault.models.documents import DocumentInfo
+from ctxvault.storage import chroma_store
+
+def build_documents_from_metadatas(metadatas)-> list[DocumentInfo]:
+    acc = {}
+
+    for row in metadatas:
+        doc_id = row["doc_id"]
+
+        if doc_id not in acc:
+            acc[doc_id] = (
+                row["source"],
+                row["filetype"],
+                1
+            )
+        else:
+            source, filetype, count = acc[doc_id]
+            acc[doc_id] = (source, filetype, count + 1)
+
+    return [
+        DocumentInfo(
+            doc_id=doc_id,
+            source=source,
+            filetype=filetype,
+            chunks_count=count
+        )
+        for doc_id, (source, filetype, count) in acc.items()
+    ]
+
+def query(query_txt: str)-> dict:
+    query_embedding = embed_list(chunks=[query_txt])
+    return chroma_store.query(query_embedding=query_embedding)
+
+def list_documents()-> list[DocumentInfo]:
+    metadatas = chroma_store.get_all_metadatas()
+    return build_documents_from_metadatas(metadatas=metadatas)
+    

ctxvault-0.1.0/src/ctxvault/storage/chroma_store.py

@@ -0,0 +1,43 @@
+from chromadb import PersistentClient
+from pathlib import Path
+from ctxvault.models.documents import DocumentInfo
+from ctxvault.utils.config import get_db_path
+
+_chroma_client = None
+_collection = None
+
+def get_collection():
+    global _chroma_client, _collection
+    if _collection is None:
+        path = get_db_path()
+        _chroma_client = PersistentClient(path=path)
+        _collection = _chroma_client.get_or_create_collection("ctxvault")
+    return _collection
+
+def add_document(ids: list[str], embeddings: list[list[float]], metadatas: list[dict], chunks: list[str]):
+    collection = get_collection()
+    collection.add(
+        ids=ids, 
+        embeddings=embeddings, 
+        metadatas=metadatas, 
+        documents=chunks
+    )
+
+def query(query_embedding: list[float],  n_results: int = 5)-> dict:
+    collection = get_collection()
+    results = collection.query(
+        query_embeddings=query_embedding,
+        n_results=n_results
+    )
+    return results
+
+def delete_document(doc_id: str):
+    collection = get_collection()
+    collection.delete(
+        where={"doc_id": doc_id}
+    )
+
+def get_all_metadatas():
+    collection = get_collection()
+    results = collection.get(include=["metadatas"])
+    return results["metadatas"]