tokenkeeper 0.1.0__tar.gz

tokenkeeper-0.1.0/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,87 @@
+name: Bug Report
+description: Report a bug or unexpected behavior
+labels: [bug]
+body:
+  - type: textarea
+    id: description
+    attributes:
+      label: Description
+      description: What happened?
+      placeholder: A clear description of the bug.
+    validations:
+      required: true
+
+  - type: textarea
+    id: steps-to-reproduce
+    attributes:
+      label: Steps to Reproduce
+      description: How can we reproduce this?
+      placeholder: |
+        1. Install tokenkeeper
+        2. Run `tokenkeeper`
+        3. ...
+    validations:
+      required: true
+
+  - type: textarea
+    id: expected-behavior
+    attributes:
+      label: Expected Behavior
+      description: What should have happened?
+    validations:
+      required: true
+
+  - type: textarea
+    id: actual-behavior
+    attributes:
+      label: Actual Behavior
+      description: What actually happened?
+      placeholder: Include error messages or stack traces if available.
+    validations:
+      required: true
+
+  - type: dropdown
+    id: os
+    attributes:
+      label: Operating System
+      options:
+        - Windows
+        - macOS
+        - Linux
+    validations:
+      required: true
+
+  - type: input
+    id: python-version
+    attributes:
+      label: Python Version
+      description: Output of `python --version`
+      placeholder: "3.12.0"
+    validations:
+      required: true
+
+  - type: input
+    id: tokenkeeper-version
+    attributes:
+      label: tokenkeeper Version
+      description: Output of `pip show tokenkeeper | grep Version`
+      placeholder: "0.1.0"
+    validations:
+      required: true
+
+  - type: input
+    id: ollama-version
+    attributes:
+      label: Ollama Version
+      description: Output of `ollama --version` (if applicable)
+      placeholder: "0.6.2"
+    validations:
+      required: false
+
+  - type: textarea
+    id: additional-context
+    attributes:
+      label: Additional Context
+      description: Logs, screenshots, configuration, or anything else that might help.
+    validations:
+      required: false

tokenkeeper-0.1.0/.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,36 @@
+name: Feature Request
+description: Suggest a new feature or enhancement
+labels: [enhancement]
+body:
+  - type: textarea
+    id: use-case
+    attributes:
+      label: Use Case
+      description: What problem does this solve? Describe the situation where this feature would be useful.
+      placeholder: I want to be able to ...
+    validations:
+      required: true
+
+  - type: textarea
+    id: proposed-solution
+    attributes:
+      label: Proposed Solution
+      description: How should it work? Describe your ideal implementation.
+    validations:
+      required: true
+
+  - type: textarea
+    id: alternatives-considered
+    attributes:
+      label: Alternatives Considered
+      description: Other approaches you have thought about and why they are less ideal.
+    validations:
+      required: false
+
+  - type: textarea
+    id: additional-context
+    attributes:
+      label: Additional Context
+      description: Mockups, examples, links, or anything else that helps illustrate the request.
+    validations:
+      required: false

tokenkeeper-0.1.0/.github/pull_request_template.md

@@ -0,0 +1,11 @@
+## Summary
+
+<!-- What does this PR do and why? -->
+
+## Checklist
+
+- [ ] Tests pass (`uv run pytest tests/ -m "not ollama"`)
+- [ ] New tests added for new functionality
+- [ ] Documentation updated (if applicable)
+- [ ] CHANGELOG.md updated under "Unreleased"
+- [ ] No breaking changes (or documented in PR description)

tokenkeeper-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,47 @@
+name: CI
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+permissions:
+  contents: read
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          enable-cache: true
+
+      - name: Install Python
+        run: uv python install
+
+      - name: Install dependencies
+        run: uv sync --locked --dev
+
+      - name: Run tests
+        if: matrix.os != 'ubuntu-latest'
+        run: uv run pytest tests/ -m "not ollama" -v --tb=short
+
+      - name: Run tests with coverage
+        if: matrix.os == 'ubuntu-latest'
+        run: uv run pytest tests/ -m "not ollama" -v --tb=short --cov=src/tokenkeeper --cov-report=xml --cov-report=term-missing
+
+      - name: Upload coverage to Codecov
+        if: matrix.os == 'ubuntu-latest'
+        uses: codecov/codecov-action@v5
+        with:
+          files: ./coverage.xml
+          token: ${{ secrets.CODECOV_TOKEN }}
+          fail_ci_if_error: false

tokenkeeper-0.1.0/.github/workflows/publish.yml

@@ -0,0 +1,57 @@
+name: Publish to PyPI
+
+on:
+  push:
+    tags:
+      - 'v*'
+
+permissions:
+  contents: read
+  id-token: write
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          enable-cache: true
+
+      - name: Install Python
+        run: uv python install
+
+      - name: Install dependencies
+        run: uv sync --locked --dev
+
+      - name: Run tests
+        run: uv run pytest tests/ -m "not ollama" -v --tb=short
+
+      - name: Build package
+        run: uv build
+
+      - name: Upload build artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - name: Download build artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          packages-dir: dist/

tokenkeeper-0.1.0/.gitignore

@@ -0,0 +1,47 @@
+# Virtual environment
+.venv/
+
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+dist/
+build/
+
+# IDE
+.vscode/
+.idea/
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Environment
+.env
+.env.local
+
+# RAG data directory (per-project, generated at runtime)
+.rag/
+
+# Windows artifacts
+NUL
+
+# Acceptance test cache (cloned repos)
+.acceptance-cache/
+
+# Internal planning docs (not for distribution)
+.planning/
+
+# Test artifacts
+_seg_test/
+.pytest_cache/
+.coverage
+coverage.xml
+htmlcov/
+
+# Claude Code project settings (personal paths)
+.claude/
+
+# Old git history backup
+.git.bak/

tokenkeeper-0.1.0/.mcp.json

@@ -0,0 +1,11 @@
+{
+  "mcpServers": {
+    "tokenkeeper": {
+      "command": "/path/to/TokenKeeper/.venv/bin/python",
+      "args": ["-m", "tokenkeeper"],
+      "env": {
+        "TOKENKEEPER_PROJECT": "${workspaceFolder}"
+      }
+    }
+  }
+}

tokenkeeper-0.1.0/.mcp.json.example

@@ -0,0 +1,11 @@
+{
+  "mcpServers": {
+    "tokenkeeper": {
+      "command": "/path/to/TokenKeeper/.venv/bin/python",
+      "args": ["-m", "tokenkeeper"],
+      "env": {
+        "TOKENKEEPER_PROJECT": "${workspaceFolder}"
+      }
+    }
+  }
+}

tokenkeeper-0.1.0/.python-version

@@ -0,0 +1 @@
+3.12

tokenkeeper-0.1.0/CHANGELOG.md

@@ -0,0 +1,27 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-02-20
+
+### Added
+
+- Initial release of TokenKeeper (tokenkeeper)
+- Hybrid search: semantic (vector) + keyword (BM25) with Reciprocal Rank Fusion
+- Local-first architecture: Ollama embeddings, ChromaDB persistent storage
+- FastMCP server with 4 tools: search_knowledge, indexing_status, reindex_documents, get_index_stats
+- File watcher for automatic reindexing on changes
+- Per-project isolation via .rag/ directory
+- Content modes: docs, code, both
+- Heading-aware markdown chunking with structure protection
+- Language-aware code chunking (Python AST, TypeScript/JavaScript regex)
+- Google Gemini embedding provider (optional cloud alternative)
+- 445 tests passing
+
+[Unreleased]: https://github.com/admin-sosys/TokenKeeper/compare/v0.1.0...HEAD
+[0.1.0]: https://github.com/admin-sosys/TokenKeeper/releases/tag/v0.1.0

tokenkeeper-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,84 @@
+# Contributing to TokenKeeper
+
+Thank you for your interest in contributing to TokenKeeper! This guide will help you get started.
+
+## Development Setup
+
+### Prerequisites
+
+- **Python 3.10+** (up to 3.13)
+- **[Ollama](https://ollama.com/)** installed and running (for embedding tests)
+- **[uv](https://docs.astral.sh/uv/)** package manager
+
+### Clone and Install
+
+```bash
+git clone https://github.com/admin-sosys/TokenKeeper.git
+cd TokenKeeper
+uv sync --dev
+```
+
+### Pull the Embedding Model
+
+```bash
+ollama pull nomic-embed-text
+```
+
+This downloads the `nomic-embed-text` model used for local embeddings (768 dimensions, runs on CPU).
+
+## Running Tests
+
+Run the full test suite:
+
+```bash
+uv run pytest tests/ -v
+```
+
+Run tests **without Ollama** (useful if Ollama is not installed):
+
+```bash
+uv run pytest tests/ -m "not ollama" -v
+```
+
+Run tests with **coverage reporting**:
+
+```bash
+uv run pytest tests/ -m "not ollama" --cov=src/tokenkeeper --cov-report=term-missing
+```
+
+## Code Style
+
+- **Follow existing patterns** in the codebase. When in doubt, look at how similar code is structured nearby.
+- **Type hints** are required for all public functions and methods.
+- **Pydantic** models are used for configuration and data validation. Use them for new config or structured data types.
+- Keep functions focused and small. Prefer composition over inheritance.
+
+## Security
+
+**Never commit secrets, tokens, API keys, or credentials.** This repository has GitHub secret scanning and push protection enabled — pushes containing detected secrets will be blocked automatically.
+
+- Store all secrets in `.env` (which is gitignored)
+- Use `${{ secrets.NAME }}` for GitHub Actions workflows
+- If you accidentally commit a secret, revoke it immediately and notify the maintainers
+
+## Pull Request Process
+
+1. **Fork** the repository and create a feature branch from `master`.
+2. **Write tests** for any new functionality. We aim for 80%+ test coverage.
+3. **Ensure all tests pass** before submitting:
+   ```bash
+   uv run pytest tests/ -m "not ollama" -v
+   ```
+4. **Update `CHANGELOG.md`** under the "Unreleased" section with a brief description of your changes.
+5. **Submit a pull request** with a clear description of what you changed and why.
+
+## Reporting Issues
+
+We use GitHub Issues to track bugs and feature requests.
+
+- **Found a bug?** Use the [Bug Report](https://github.com/admin-sosys/TokenKeeper/issues/new?template=bug_report.yml) template. Include steps to reproduce, expected vs. actual behavior, and your environment details.
+- **Have an idea?** Use the [Feature Request](https://github.com/admin-sosys/TokenKeeper/issues/new?template=feature_request.yml) template. Describe the use case, your proposed solution, and any alternatives you considered.
+
+## License
+
+By contributing, you agree that your contributions will be licensed under the [MIT License](LICENSE).