docs-kit 0.1.3__tar.gz → 0.1.5__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- docs_kit-0.1.5/AGENTS.md +76 -0
- docs_kit-0.1.5/CHANGELOG.md +92 -0
- docs_kit-0.1.5/CLAUDE.md +95 -0
- docs_kit-0.1.5/PKG-INFO +270 -0
- docs_kit-0.1.5/README.md +231 -0
- docs_kit-0.1.5/docs_kit/_version.py +1 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/cli/__main__.py +3 -5
- docs_kit-0.1.5/docs_kit/cli/commands.py +574 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/vector_stores/qdrant.py +81 -52
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/core/chunking.py +167 -2
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/core/config.py +0 -8
- docs_kit-0.1.5/docs_kit/templates/claude_code_skill.md +84 -0
- docs_kit-0.1.5/docs_kit/templates/claude_desktop_skill.md +36 -0
- docs_kit-0.1.5/docs_kit/templates/cursor_agents_md.md +33 -0
- docs_kit-0.1.5/docs_kit/templates/skill.md +74 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/pyproject.toml +3 -5
- docs_kit-0.1.5/tests/test_chunking.py +290 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_cli.py +8 -82
- docs_kit-0.1.5/tests/test_install_cmd.py +488 -0
- docs_kit-0.1.3/AGENTS.md +0 -64
- docs_kit-0.1.3/CHANGELOG.md +0 -20
- docs_kit-0.1.3/CLAUDE.md +0 -85
- docs_kit-0.1.3/PKG-INFO +0 -316
- docs_kit-0.1.3/README.md +0 -271
- docs_kit-0.1.3/docs_kit/_version.py +0 -1
- docs_kit-0.1.3/docs_kit/cli/commands.py +0 -668
- docs_kit-0.1.3/docs_kit/mcp/server.py +0 -100
- docs_kit-0.1.3/docs_kit/mcp/tools.py +0 -10
- docs_kit-0.1.3/tests/test_chunking.py +0 -41
- docs_kit-0.1.3/tests/test_install_cmd.py +0 -226
- docs_kit-0.1.3/tests/test_mcp_tools.py +0 -96
- {docs_kit-0.1.3 → docs_kit-0.1.5}/.github/workflows/ci.yml +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/.github/workflows/publish.yml +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/.gitignore +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/CONTRIBUTING.md +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/LICENSE +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/data/sample_docs/claude-code-changelog.md +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/data/sample_docs/the-adventure-of-the-speckled-band.md +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs-kit.yaml +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/__main__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/agent.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/cli/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/cli/help.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/embeddings/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/embeddings/base.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/embeddings/fastembed.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/fetchers/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/fetchers/base.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/fetchers/gitbook.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/fetchers/llms_txt.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/fetchers/mintlify.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/parsers/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/parsers/base.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/parsers/markdown.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/parsers/text.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/vector_stores/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/connectors/vector_stores/base.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/core/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/core/html_utils.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/docs_kit/core/models.py +0 -0
- {docs_kit-0.1.3/docs_kit/mcp → docs_kit-0.1.5/docs_kit/templates}/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/npx-wrapper/bin/docs-kit.js +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/npx-wrapper/package.json +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/scripts/smoke_test.sh +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/__init__.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_agent.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_config.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_embeddings_fastembed.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_fetcher_gitbook.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_fetcher_mintlify.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_models.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_parsers.py +0 -0
- {docs_kit-0.1.3 → docs_kit-0.1.5}/tests/test_vector_store_qdrant.py +0 -0
docs_kit-0.1.5/AGENTS.md
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
# docs-kit — Agent Instructions
|
|
2
|
+
|
|
3
|
+
## Scope
|
|
4
|
+
|
|
5
|
+
This document applies to AI agents working on the **docs-kit** codebase: a **PyPI** package (and optional **npx** wrapper) that **fetches documentation** (GitBook, Mintlify, or local files), **stores and embeds it locally** (markdown on disk + Qdrant), and **teaches coding agents to call the CLI** via **installed skills** — not via an MCP server.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## What the product does
|
|
10
|
+
|
|
11
|
+
1. **Fetch** — Fetchers use `/llms-full.txt` and `/llms.txt` on public doc sites; **Mintlify** also falls back to `/sitemap.xml` when needed. **GitBook** and **Mintlify** are supported; `docs-kit ingest` accepts `--provider auto|gitbook|mintlify`.
|
|
12
|
+
2. **Local storage** — `docs-kit fetch` writes `.md` files; `docs-kit ingest` chunks, embeds with FastEmbed (dense + sparse/BM25), and upserts into local Qdrant. Concurrent CLI processes coordinate with **filelock** on the Qdrant path.
|
|
13
|
+
3. **Agent integration** — `docs-kit install <agent>` installs **skill templates** (`docs_kit/templates/`) so agents run **`docs-kit` CLI** commands (`query`, `list`, `ingest`, `remove`, `inspect`, `doctor`, etc.). **Claude Desktop** gets a zip to upload in the app; **Cursor** may also get a home-level `AGENTS.md` fallback fragment.
|
|
14
|
+
4. **Install targets** (see `INSTALL_TARGETS` in `docs_kit/cli/commands.py`): `claude-code`, `claude-desktop`, `cursor`, `codex`, `codex-app`, `codex-desktop`, `opencode`.
|
|
15
|
+
|
|
16
|
+
The **npx** package under `npx-wrapper/` ensures Python 3.11+ and installs/runs the **`docs-kit`** Python module; behavior lives in Python.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Security (Non-Negotiable)
|
|
21
|
+
|
|
22
|
+
- **MUST NOT** read, access, or reference `.env`, `.env.local`, `.env.*`, or any secret/credentials files.
|
|
23
|
+
- **MUST NOT** recommend reading environment files. If config is needed, ask the user for non-sensitive details.
|
|
24
|
+
- **MUST** treat `.env` files as if they do not exist.
|
|
25
|
+
|
|
26
|
+
---
|
|
27
|
+
|
|
28
|
+
## Verification After Changes
|
|
29
|
+
|
|
30
|
+
From the repository root:
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
pytest tests/ -v
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
- **MUST NOT** claim work is complete without a successful test run when behavior or dependencies changed.
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## Skill templates (`docs_kit/templates/`)
|
|
41
|
+
|
|
42
|
+
Skills installed by `docs-kit install` come from these Markdown templates.
|
|
43
|
+
|
|
44
|
+
- **MUST** review `docs_kit/templates/` after **major** CLI or install-flow changes (new or removed commands, renamed options, changed defaults, new `INSTALL_TARGETS`, different recommended workflows).
|
|
45
|
+
- **MUST** edit the relevant templates when skill text or examples would be wrong or misleading; **need not** update templates when the change does not affect what installed skills describe.
|
|
46
|
+
- If you are unsure whether agents would see outdated instructions, check the templates and align them with `docs_kit/cli/commands.py` and install behavior.
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Git Commit Workflow
|
|
51
|
+
|
|
52
|
+
When the user asks to commit, stage, or write a commit message:
|
|
53
|
+
|
|
54
|
+
1. **MUST** analyze staged/unstaged changes via `git status` and `git diff` in the relevant git root.
|
|
55
|
+
2. **MUST** write a commit message following Conventional Commits:
|
|
56
|
+
- `feat:` — new feature
|
|
57
|
+
- `fix:` — bug fix
|
|
58
|
+
- `chore:` — maintenance, deps, config
|
|
59
|
+
- `docs:` — documentation only
|
|
60
|
+
- `style:` — formatting, no logic change
|
|
61
|
+
- `refactor:` — code restructure
|
|
62
|
+
- `test:` — tests only
|
|
63
|
+
- `perf:` — performance
|
|
64
|
+
3. **SHOULD** keep subject line under 50 chars, imperative mood; use bullet points in body.
|
|
65
|
+
4. **SHOULD** suggest splitting into atomic commits if changes span unrelated concerns.
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## Project Context (Brief)
|
|
70
|
+
|
|
71
|
+
- **Package name:** `docs-kit` (import `docs_kit`, CLI `docs-kit`). **Python:** `>=3.11,<3.14` (`pyproject.toml`); no `mcp` dependency in this tree.
|
|
72
|
+
- **Main types:** `DocsKitConfig` (`docs_kit/core/config.py`), `DocsKitAgent` (`docs_kit/agent.py`).
|
|
73
|
+
- **CLI:** `docs_kit/cli/commands.py` + `docs_kit/cli/__main__.py` — `init`, `fetch`, `ingest`, `query`, `inspect`, `doctor`, `install`, `remove`, `list`.
|
|
74
|
+
- **Fetchers:** `docs_kit/connectors/fetchers/gitbook.py`, `mintlify.py`, shared `llms_txt` helpers.
|
|
75
|
+
- **Skills / install artifacts:** `docs_kit/templates/` (e.g. `skill.md`, `claude_code_skill.md`, `cursor_agents_md.md`).
|
|
76
|
+
- **Docs:** [README.md](README.md) for user-facing quickstart (if it diverges from the repo, trust the code and `docs/project-overview.md` for architecture).
|
|
@@ -0,0 +1,92 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented in this file.
|
|
4
|
+
|
|
5
|
+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
|
|
7
|
+
## [0.1.5] - 2026-03-28
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- `docs-kit query --full` to print full chunk text; default preview truncation increased to 1500 characters (from 300).
|
|
12
|
+
- Markdown chunking that keeps tables and fenced code blocks intact when possible: large tables split on row boundaries with header repeated; large code blocks split on lines with opening/closing fences preserved in each part.
|
|
13
|
+
- Tests covering table/code chunking, list-heavy sections, and headers inside fences.
|
|
14
|
+
|
|
15
|
+
### Changed
|
|
16
|
+
|
|
17
|
+
- `docs-kit install codex` (and `codex-app` / `codex-desktop`) writes the skill to `~/.codex/skills/docs-kit/SKILL.md` and mirrors the same content to `~/.agents/skills/docs-kit/SKILL.md` for compatibility with shared agent layouts.
|
|
18
|
+
- `docs-kit doctor` lists installed skills with install-target labels (`claude-code`, `cursor`, `codex`, `codex-shared`, `opencode`, `claude-desktop`) and suggests the matching `docs-kit install <target>` command when missing.
|
|
19
|
+
|
|
20
|
+
### Fixed
|
|
21
|
+
|
|
22
|
+
- Chunking no longer merges or normalizes markdown tables and code fences in ways that broke pipe layout or list structure.
|
|
23
|
+
|
|
24
|
+
## [0.1.4] - 2026-03-28
|
|
25
|
+
|
|
26
|
+
### Changed (breaking)
|
|
27
|
+
|
|
28
|
+
- Removed the MCP server and MCP dependencies; agents use installed skill files that invoke the `docs-kit` CLI directly (`query`, `list`, `ingest`, `remove`, `inspect`, `doctor`, etc.).
|
|
29
|
+
- CLI: added `list` and `remove`; removed `serve` and `stop`.
|
|
30
|
+
- `docs-kit install` writes skills for Claude Code, Cursor, Codex, OpenCode, Claude Desktop, and related targets under `docs_kit/templates/`.
|
|
31
|
+
- `doctor` checks PATH, config, Qdrant, and installed skill paths.
|
|
32
|
+
- Local Qdrant access is serialized with `filelock` for concurrent CLI use.
|
|
33
|
+
- Python support remains `>=3.11,<3.14`.
|
|
34
|
+
|
|
35
|
+
## [0.1.3] - 2026-03-28
|
|
36
|
+
|
|
37
|
+
### Added
|
|
38
|
+
|
|
39
|
+
- `docs-kit serve` as a background daemon (PID + logs) and `docs-kit stop`.
|
|
40
|
+
- Default MCP transport: SSE on `localhost:45139` to reduce Qdrant lock contention.
|
|
41
|
+
- Install flows write URL entries pointing at `/sse` for Claude, Cursor, and Codex.
|
|
42
|
+
- `doctor` checks MCP process and port reachability.
|
|
43
|
+
|
|
44
|
+
### Changed
|
|
45
|
+
|
|
46
|
+
- Project install without a local `docs-kit.yaml` uses in-memory defaults (no user bootstrap in that path).
|
|
47
|
+
|
|
48
|
+
### Fixed
|
|
49
|
+
|
|
50
|
+
- Serve daemon test log path mock (`MagicMock` parent breaking `mkdir`).
|
|
51
|
+
|
|
52
|
+
## [0.1.2] - 2026-03-27
|
|
53
|
+
|
|
54
|
+
### Added
|
|
55
|
+
|
|
56
|
+
- Bootstrap `~/.docs-kit/docs-kit.yaml` and default Qdrant path when running a global `install` with no project config.
|
|
57
|
+
- `DocsKitConfig.from_yaml` loading from the user config after `./docs-kit.yaml`.
|
|
58
|
+
|
|
59
|
+
### Changed
|
|
60
|
+
|
|
61
|
+
- Documented pipx-first install, config precedence, and Python 3.11–3.13 cap; `requires-python` set to `<3.14` (onnxruntime wheels).
|
|
62
|
+
|
|
63
|
+
## [0.1.1] - 2026-03-27
|
|
64
|
+
|
|
65
|
+
### Added
|
|
66
|
+
|
|
67
|
+
- Mintlify fetcher (`llms-full.txt`, `llms.txt`, sitemap fallback) and shared `llms_txt` helpers.
|
|
68
|
+
- `docs-kit ingest --provider auto|gitbook|mintlify`.
|
|
69
|
+
- Formatted CLI help and banner (`DocsKitGroup` / `DocsKitCommand`, examples in epilogs).
|
|
70
|
+
|
|
71
|
+
### Changed
|
|
72
|
+
|
|
73
|
+
- GitBook fetcher refactor; HTML stripped before chunking via `html_utils`.
|
|
74
|
+
- `install` resolves absolute `docs-kit` command and config paths; warns if YAML is missing.
|
|
75
|
+
|
|
76
|
+
### Chore
|
|
77
|
+
|
|
78
|
+
- Ignore local `docs/` and add release helper script to `.gitignore`.
|
|
79
|
+
|
|
80
|
+
## [0.1.0] - 2026-03-27
|
|
81
|
+
|
|
82
|
+
### Added
|
|
83
|
+
|
|
84
|
+
- `DocsKitAgent` API: `ingest()`, `query()`.
|
|
85
|
+
- CLI: `docs-kit init`, `fetch`, `ingest`, `serve`, `install`, `query`, `inspect`, `doctor`.
|
|
86
|
+
- GitBook fetch via `/llms-full.txt` / `/llms.txt` and linked pages.
|
|
87
|
+
- Local embeddings with FastEmbed (dense + sparse/BM25).
|
|
88
|
+
- Vector store: Qdrant (local path or remote URL).
|
|
89
|
+
- Document parsers for `.txt` and `.md`.
|
|
90
|
+
- `DocsKitConfig` with YAML and environment variable support.
|
|
91
|
+
- MCP server tools: `search_docs`, `list_sources`, `get_collection_info`, `get_full_document`.
|
|
92
|
+
- Annotated `docs-kit.yaml` example, npx wrapper, sample data, CI and publish workflows.
|
docs_kit-0.1.5/CLAUDE.md
ADDED
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# docs-kit — Project Memory
|
|
2
|
+
|
|
3
|
+
## Project Overview
|
|
4
|
+
|
|
5
|
+
**docs-kit** is a distributable toolkit (PyPI **`docs-kit`**, optional **npm/npx** wrapper) that:
|
|
6
|
+
|
|
7
|
+
1. **Pulls** documentation from public **GitBook** and **Mintlify** sites via AI-oriented endpoints (`/llms-full.txt` or `/llms.txt` and linked pages).
|
|
8
|
+
2. **Stores** content locally — either as downloaded `.md` files (`docs-kit fetch`) or in a **local Qdrant** vector store after embedding (`docs-kit ingest`).
|
|
9
|
+
3. **Ingests** with **local embeddings** (FastEmbed by default; no API keys required for embeddings).
|
|
10
|
+
4. **Installs skill files** into AI agents (Claude Code, Cursor, Codex, OpenCode, Claude Desktop) that teach them to call `docs-kit` CLI commands directly — no background server required.
|
|
11
|
+
|
|
12
|
+
**Pitch:** Point at any public docs, embed locally, let your coding agents query them directly via CLI skills.
|
|
13
|
+
|
|
14
|
+
**Target:** Developers who want product/docs RAG in the agent loop without shipping docs to a remote embedding API.
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Distribution
|
|
19
|
+
|
|
20
|
+
| Channel | What it is |
|
|
21
|
+
|---------|------------|
|
|
22
|
+
| **PyPI** | Canonical install: `pipx install docs-kit`. CLI entry: `docs-kit`. |
|
|
23
|
+
| **npm/npx** | `npx-wrapper/` — thin Node shim that ensures Python 3.11+ and `pip install docs-kit`, then runs `python -m docs_kit …`. |
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## Repo Structure
|
|
28
|
+
|
|
29
|
+
| Path | Purpose |
|
|
30
|
+
|------|---------|
|
|
31
|
+
| `docs_kit/` | Python package: `DocsKitAgent`, GitBook/Mintlify fetchers, parsers, FastEmbed, Qdrant store, Click CLI |
|
|
32
|
+
| `docs_kit/cli/` | CLI (`docs-kit init`, `fetch`, `ingest`, `install`, `query`, `inspect`, `list`, `remove`, `doctor`) |
|
|
33
|
+
| `docs_kit/templates/` | Skill/instruction template files installed by `docs-kit install` |
|
|
34
|
+
| `docs_kit/connectors/fetchers/gitbook.py` | GitBook `llms-full.txt` / `llms.txt` fetch |
|
|
35
|
+
| `docs_kit/connectors/fetchers/mintlify.py` | Mintlify fetch with sitemap.xml fallback |
|
|
36
|
+
| `npx-wrapper/` | npm package wrapping the Python CLI |
|
|
37
|
+
| `tests/` | pytest suite |
|
|
38
|
+
| `data/sample_docs/` | Sample markdown for local ingest tests |
|
|
39
|
+
|
|
40
|
+
---
|
|
41
|
+
|
|
42
|
+
## Tech Stack
|
|
43
|
+
|
|
44
|
+
- **Python** ≥ 3.11, **Click**, **Pydantic** / **pydantic-settings**, **httpx**, **PyYAML**
|
|
45
|
+
- **Embeddings:** FastEmbed (local)
|
|
46
|
+
- **Vector store:** Qdrant (local embedded, path `~/.docs-kit/qdrant` for user config)
|
|
47
|
+
- **Concurrency:** `filelock` for serializing concurrent embedded Qdrant access
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## Security & Privacy
|
|
52
|
+
|
|
53
|
+
- **NEVER** touch `.env`, `.env.local`, `.env.*`, or other secret/credentials files. Treat them as if they do not exist.
|
|
54
|
+
- **NEVER** read, access, reference, or recommend reading environment or secret files.
|
|
55
|
+
- **ALWAYS** work on `.env.example` when documenting or adding environment variables — use placeholder values only.
|
|
56
|
+
- If config is needed, ask the user for non-sensitive details.
|
|
57
|
+
|
|
58
|
+
---
|
|
59
|
+
|
|
60
|
+
## Build & Verification
|
|
61
|
+
|
|
62
|
+
From the project root (where `pyproject.toml` lives):
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
/usr/local/bin/python3.13 -m pytest tests/ -v
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
After substantive changes, run the full test suite before claiming completion.
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## Skill templates (`docs_kit/templates/`)
|
|
73
|
+
|
|
74
|
+
Installed agent skills are generated from these templates. They must stay aligned with the real CLI and install flow.
|
|
75
|
+
|
|
76
|
+
- **MUST** open and review `docs_kit/templates/` after **major** changes to commands, flags, defaults, install targets, or agent integration (anything that changes what users or agents actually run).
|
|
77
|
+
- **MUST** update the affected template files when those changes alter skill instructions; **need not** touch templates when the change is clearly unrelated (e.g. internal refactors with no CLI or install behavior change).
|
|
78
|
+
- Stale templates mislead agents; treat template drift as a product bug on par with wrong CLI help text.
|
|
79
|
+
|
|
80
|
+
---
|
|
81
|
+
|
|
82
|
+
## Git Commits
|
|
83
|
+
|
|
84
|
+
- **MUST** write descriptive commit messages using [Conventional Commits](https://www.conventionalcommits.org/): `feat`, `fix`, `chore`, `docs`, `style`, `refactor`, `test`, `perf`.
|
|
85
|
+
- **SHOULD** keep subject line under 50 characters, imperative mood; use bullet points in body for context.
|
|
86
|
+
|
|
87
|
+
---
|
|
88
|
+
|
|
89
|
+
## Key Paths & Commands
|
|
90
|
+
|
|
91
|
+
- **Config:** auto-created at `~/.docs-kit/docs-kit.yaml` on first use; or project-local `docs-kit.yaml` via `docs-kit init`
|
|
92
|
+
- **Fetch only:** `docs-kit fetch <url> --output docs-kit-docs`
|
|
93
|
+
- **Ingest URL or path:** `docs-kit ingest https://example.gitbook.io/docs` or `docs-kit ingest ./path/to/markdown`
|
|
94
|
+
- **Agent setup:** `docs-kit install claude-code` (or `cursor`, `codex`, `opencode`, `claude-desktop`)
|
|
95
|
+
- **No MCP server:** `serve` and `stop` commands do not exist; agents call CLI directly via installed skill files
|
docs_kit-0.1.5/PKG-INFO
ADDED
|
@@ -0,0 +1,270 @@
|
|
|
1
|
+
Metadata-Version: 2.3
|
|
2
|
+
Name: docs-kit
|
|
3
|
+
Version: 0.1.5
|
|
4
|
+
Summary: Fetch docs, embed locally, expose to AI agents via skills.
|
|
5
|
+
License: MIT License
|
|
6
|
+
|
|
7
|
+
Copyright (c) 2026 Docs Kit Limited
|
|
8
|
+
|
|
9
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
10
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
11
|
+
in the Software without restriction, including without limitation the rights
|
|
12
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
13
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
14
|
+
furnished to do so, subject to the following conditions:
|
|
15
|
+
|
|
16
|
+
The above copyright notice and this permission notice shall be included in all
|
|
17
|
+
copies or substantial portions of the Software.
|
|
18
|
+
|
|
19
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
20
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
21
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
22
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
23
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
24
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
25
|
+
SOFTWARE.
|
|
26
|
+
Requires-Python: <3.14,>=3.11
|
|
27
|
+
Requires-Dist: click>=8.0.0
|
|
28
|
+
Requires-Dist: fastembed>=0.6.0
|
|
29
|
+
Requires-Dist: filelock>=3.0.0
|
|
30
|
+
Requires-Dist: httpx>=0.27.0
|
|
31
|
+
Requires-Dist: pydantic-settings>=2.2.1
|
|
32
|
+
Requires-Dist: pydantic>=2.0.0
|
|
33
|
+
Requires-Dist: pyyaml>=6.0
|
|
34
|
+
Requires-Dist: qdrant-client>=1.10.0
|
|
35
|
+
Provides-Extra: dev
|
|
36
|
+
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
|
|
37
|
+
Requires-Dist: pytest>=8.0.0; extra == 'dev'
|
|
38
|
+
Description-Content-Type: text/markdown
|
|
39
|
+
|
|
40
|
+
# docs-kit
|
|
41
|
+
|
|
42
|
+
[](https://pypi.org/project/docs-kit/)
|
|
43
|
+
[](LICENSE)
|
|
44
|
+
[](https://pypi.org/project/docs-kit/)
|
|
45
|
+
[](https://github.com/docs-kit/docs-kit/actions/workflows/ci.yml)
|
|
46
|
+
|
|
47
|
+
Fetch docs from GitBook, Mintlify, or local files, embed them locally, and teach AI agents to search them via installed **skills** (CLI commands). No API keys for the default embedding path. No background server or MCP.
|
|
48
|
+
|
|
49
|
+
## What it does
|
|
50
|
+
|
|
51
|
+
- Fetches public docs from **GitBook** and **Mintlify** sites via `/llms-full.txt` and `/llms.txt`, with a **sitemap.xml** fallback on Mintlify when those endpoints are missing
|
|
52
|
+
- **`docs-kit ingest`:** `--provider auto` (default) uses a combined strategy (llms endpoints → sitemap). **`--provider gitbook`** uses the GitBook-only fetcher; **`--provider mintlify`** uses the same pipeline as `auto`
|
|
53
|
+
- Ingests local `.md` and `.txt` files
|
|
54
|
+
- Stores vectors in **embedded Qdrant** on disk — typically `~/.docs-kit/qdrant` when using the auto-created user config, or `.docs-kit/qdrant` (resolved relative to your `docs-kit.yaml`) for a project-local config from `docs-kit init`
|
|
55
|
+
- **Hybrid retrieval** (dense embeddings + sparse / BM25-style vectors) for `docs-kit query` and agent-driven queries
|
|
56
|
+
- Installs **skill files** into Claude Code, Cursor, Codex, OpenCode, and Claude Desktop so agents run `docs-kit` over the shell
|
|
57
|
+
- Concurrent CLI runs coordinate with a **file lock** on the local Qdrant path
|
|
58
|
+
|
|
59
|
+
## Install
|
|
60
|
+
|
|
61
|
+
Recommended: **`pipx`** so `docs-kit` is on your PATH:
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
pipx install docs-kit
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Install `pipx` if needed:
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
brew install pipx
|
|
71
|
+
pipx ensurepath
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Or **`pip`** inside a virtual environment:
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
pip install docs-kit
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
Optional: the repo includes an **`npx-wrapper/`** npm package that bootstraps Python and runs the same CLI (`npx docs-kit …` when published or linked locally).
|
|
81
|
+
|
|
82
|
+
## Quickstart
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
# 1. Install (once)
|
|
86
|
+
pipx install docs-kit
|
|
87
|
+
|
|
88
|
+
# 2. Ingest docs
|
|
89
|
+
docs-kit ingest https://docs.example.com
|
|
90
|
+
|
|
91
|
+
# 3. Install skill into your agent
|
|
92
|
+
docs-kit install claude-code # or: cursor, codex, opencode, claude-desktop
|
|
93
|
+
|
|
94
|
+
# 4. Use the agent — it can run docs-kit query / list / ingest when relevant
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
No server to start. On first use, config and the default vector store path are created under **`~/.docs-kit/`** unless you use a project-local `docs-kit.yaml`.
|
|
98
|
+
|
|
99
|
+
## How it works
|
|
100
|
+
|
|
101
|
+
docs-kit installs a **skill** (Markdown + YAML frontmatter) into each tool’s skills directory. The skill tells the agent when to use docs-kit and which commands to run (`query`, `list`, `ingest`, `remove`, `inspect`, `doctor`, …). Agents execute **`docs-kit`** via their normal terminal integration.
|
|
102
|
+
|
|
103
|
+
For more architecture detail (config resolution, layout, security posture), see [docs/project-overview.md](docs/project-overview.md).
|
|
104
|
+
|
|
105
|
+
## Install targets
|
|
106
|
+
|
|
107
|
+
| Command | Where the skill lands |
|
|
108
|
+
|---------|------------------------|
|
|
109
|
+
| `docs-kit install claude-code` | `~/.claude/skills/docs-kit/SKILL.md` |
|
|
110
|
+
| `docs-kit install codex` | `~/.codex/skills/docs-kit/SKILL.md` (also mirrors to `~/.agents/skills/docs-kit/SKILL.md` for compatibility) |
|
|
111
|
+
| `docs-kit install cursor` | `~/.cursor/skills/docs-kit/SKILL.md` + marked block in `~/.cursor/AGENTS.md` when needed |
|
|
112
|
+
| `docs-kit install opencode` | `~/.config/opencode/skills/docs-kit/SKILL.md` (and may mirror under `~/.agents/skills/` if absent) |
|
|
113
|
+
| `docs-kit install claude-desktop` | `~/.docs-kit/exports/claude-desktop/docs-kit.zip` — upload via Claude Desktop **Customize → Skills** |
|
|
114
|
+
|
|
115
|
+
`codex-app` and `codex-desktop` are accepted aliases for the Codex install path.
|
|
116
|
+
|
|
117
|
+
Use **`--project`** with `claude-code` for `.claude/skills/docs-kit/SKILL.md` in the current repo.
|
|
118
|
+
|
|
119
|
+
## Commands
|
|
120
|
+
|
|
121
|
+
### `docs-kit install <agent>`
|
|
122
|
+
|
|
123
|
+
Install the docs-kit skill into a supported agent.
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
docs-kit install claude-code
|
|
127
|
+
docs-kit install cursor
|
|
128
|
+
docs-kit install codex
|
|
129
|
+
docs-kit install opencode
|
|
130
|
+
docs-kit install claude-desktop
|
|
131
|
+
docs-kit install claude-code --project
|
|
132
|
+
docs-kit install cursor --config ./docs-kit.yaml
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
If no config is found, **`~/.docs-kit/docs-kit.yaml`** is created automatically (non-project installs).
|
|
136
|
+
|
|
137
|
+
### `docs-kit ingest <path-or-url>`
|
|
138
|
+
|
|
139
|
+
Ingest a local file, directory, or documentation URL.
|
|
140
|
+
|
|
141
|
+
```bash
|
|
142
|
+
docs-kit ingest ./docs
|
|
143
|
+
docs-kit ingest https://docs.example.com
|
|
144
|
+
docs-kit ingest https://docs.example.com --provider gitbook
|
|
145
|
+
docs-kit ingest ./docs --recreate
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
`--provider`: `auto` (default), `gitbook`, or `mintlify`.
|
|
149
|
+
|
|
150
|
+
### `docs-kit query <text>`
|
|
151
|
+
|
|
152
|
+
Run hybrid retrieval from the CLI.
|
|
153
|
+
|
|
154
|
+
```bash
|
|
155
|
+
docs-kit query "How do I authenticate?"
|
|
156
|
+
docs-kit query "getting started" --limit 3
|
|
157
|
+
docs-kit query "season 5 end date" --full
|
|
158
|
+
docs-kit query "..." --config ./docs-kit.yaml
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Use `--full` when you want the entire chunk body instead of the default preview.
|
|
162
|
+
|
|
163
|
+
### `docs-kit list`
|
|
164
|
+
|
|
165
|
+
List ingested sources with ingestion timestamps.
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
docs-kit list
|
|
169
|
+
docs-kit list --config ./docs-kit.yaml
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
### `docs-kit fetch <url>`
|
|
173
|
+
|
|
174
|
+
Download **GitBook** docs as Markdown files only (does not embed). For Mintlify or mixed hosting, use **`docs-kit ingest`** or copy files locally first.
|
|
175
|
+
|
|
176
|
+
```bash
|
|
177
|
+
docs-kit fetch https://docs.example.com
|
|
178
|
+
docs-kit fetch https://docs.example.com --output ./downloaded-docs
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
### `docs-kit remove <source>`
|
|
182
|
+
|
|
183
|
+
Remove an ingested source by URL or file path.
|
|
184
|
+
|
|
185
|
+
```bash
|
|
186
|
+
docs-kit remove https://docs.example.com/page
|
|
187
|
+
docs-kit remove ./docs/getting-started.md
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
### `docs-kit inspect`
|
|
191
|
+
|
|
192
|
+
Show collection stats and embedding settings.
|
|
193
|
+
|
|
194
|
+
```bash
|
|
195
|
+
docs-kit inspect
|
|
196
|
+
docs-kit inspect --config ./docs-kit.yaml
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
### `docs-kit doctor`
|
|
200
|
+
|
|
201
|
+
Check `docs-kit` on PATH, effective config, Qdrant path / connectivity, and which skills are installed.
|
|
202
|
+
|
|
203
|
+
For Codex installs, `doctor` reports both the native `~/.codex/skills/...` install and the shared compatibility mirror under `~/.agents/skills/...` when present.
|
|
204
|
+
|
|
205
|
+
```bash
|
|
206
|
+
docs-kit doctor
|
|
207
|
+
docs-kit doctor --config ./docs-kit.yaml
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
### `docs-kit init`
|
|
211
|
+
|
|
212
|
+
Create a project-local **`docs-kit.yaml`** (optional if you rely on `~/.docs-kit/docs-kit.yaml`).
|
|
213
|
+
|
|
214
|
+
```bash
|
|
215
|
+
docs-kit init
|
|
216
|
+
docs-kit init --dir ./sandbox
|
|
217
|
+
```
|
|
218
|
+
|
|
219
|
+
## Configuration
|
|
220
|
+
|
|
221
|
+
**Resolution order:** `--config` → `./docs-kit.yaml` in the current working directory → `~/.docs-kit/docs-kit.yaml` (created on first use when applicable).
|
|
222
|
+
|
|
223
|
+
Example **project-local** `docs-kit.yaml` from `docs-kit init`:
|
|
224
|
+
|
|
225
|
+
```yaml
|
|
226
|
+
embedding:
|
|
227
|
+
provider: fastembed
|
|
228
|
+
model: BAAI/bge-small-en-v1.5
|
|
229
|
+
|
|
230
|
+
vector_store:
|
|
231
|
+
provider: qdrant
|
|
232
|
+
local_path: .docs-kit/qdrant # resolved relative to this file’s directory
|
|
233
|
+
collection_name: knowledge_base
|
|
234
|
+
retrieval_limit: 5
|
|
235
|
+
score_threshold: 0.35
|
|
236
|
+
|
|
237
|
+
ingestion:
|
|
238
|
+
chunk_size: 800
|
|
239
|
+
chunk_overlap: 120
|
|
240
|
+
bm25_model: Qdrant/bm25
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
The auto-created **user** config under `~/.docs-kit/` sets `local_path` to an absolute directory under **`~/.docs-kit/qdrant`**. To use a **remote** Qdrant instance, set `vector_store.url` and leave local storage unused.
|
|
244
|
+
|
|
245
|
+
## Supported sources
|
|
246
|
+
|
|
247
|
+
| Source | Strategy |
|
|
248
|
+
|--------|----------|
|
|
249
|
+
| GitBook sites | `/llms-full.txt` → `/llms.txt` (GitBook fetcher); `auto` / `mintlify` use the broader pipeline below |
|
|
250
|
+
| Mintlify & typical llms.txt sites | `/llms-full.txt` → `/llms.txt` → `/sitemap.xml` |
|
|
251
|
+
| Local `.md` / `.txt` | Read from disk |
|
|
252
|
+
|
|
253
|
+
## Requirements
|
|
254
|
+
|
|
255
|
+
- **Python 3.11–3.13** (`requires-python` excludes 3.14+ while dependencies such as `onnxruntime` lack wheels)
|
|
256
|
+
- Disk space for the embedding model (on the order of tens of MB for the default model)
|
|
257
|
+
|
|
258
|
+
If your default `python` is 3.14:
|
|
259
|
+
|
|
260
|
+
```bash
|
|
261
|
+
pipx install docs-kit --python python3.13
|
|
262
|
+
```
|
|
263
|
+
|
|
264
|
+
## Migration from v0.1.x
|
|
265
|
+
|
|
266
|
+
Older releases configured an **MCP server** in agent settings. That integration has been removed in favor of **skills + CLI**.
|
|
267
|
+
|
|
268
|
+
1. Remove legacy **`mcpServers.docs-kit`** (or equivalent) from `~/.claude/settings.json`, Cursor MCP config, `~/.codex/config.toml`, etc.
|
|
269
|
+
2. Run **`docs-kit install <agent>`** again to install skills.
|
|
270
|
+
3. Existing **`~/.docs-kit/docs-kit.yaml`** and ingested data remain valid; any unused **`mcp:`** section in YAML can be deleted for clarity.
|