@ontos-ai/knowhere-claw 0.1.0-beta.0

package/skills/knowhere/SKILL.md

@@ -0,0 +1,243 @@
+---
+name: knowhere
+description: Use Knowhere plugin tools to ingest local files or URLs, browse stored document structure with path and chunk tools, inspect raw result files, inspect jobs, and manage the current Knowhere scope. Use this whenever the user asks to parse, inspect, summarize, quote, import, remove, or clear Knowhere-managed documents or attachments.
+user-invocable: false
+---
+
+# Knowhere Skill
+
+Use the `knowhere_*` tools for explicit document ingestion and browse-first stored-result workflows.
+
+## Terminology
+
+Use these terms consistently:
+
+- `stored document`: one locally stored document identified by `docId`
+- `scope`: the current local storage scope
+- `path`: a structural path derived from Knowhere chunk paths
+- `chunk`: one stored parsed unit identified by `chunkId`
+- `asset path`: the relative path to a stored image or table asset under `result/`
+
+## Response Style
+
+Keep tool-driven replies short and labeled.
+
+- Reuse labels such as `Scope`, `Source`, `File`, `Chunks`, `Job ID`, and `Next` when relaying tool results.
+- Prefer one short status line plus the key fields the user needs for the next step.
+
+## When to use Knowhere
+
+Use Knowhere when the user wants to:
+
+- ingest a local file or URL into the current scope
+- inspect, summarize, or quote previously ingested documents
+- inspect ingest jobs or import a completed Knowhere job
+- preview, list, remove, or clear stored documents
+- understand what fields exist inside the stored result package
+
+Do not assume an uploaded attachment was already ingested. If the user asks you to use an attached file and no existing Knowhere result already covers it, call `knowhere_ingest_document` yourself.
+
+## Attachment markers
+
+When a prompt contains a marker like:
+
+```text
+[media attached: /absolute/path/to/file.pdf (application/pdf) | handbook.pdf]
+```
+
+Use:
+
+- the exact absolute path as `filePath`
+- the visible filename as `fileName`
+
+This preserves the original filename when the local path is a temporary UUID-based attachment path.
+
+## Stored result model
+
+The canonical Knowhere delivery format behind `result_url` is a ZIP package.
+This plugin stores that package in extracted form on disk: lightweight local
+metadata in `metadata.json`, a plugin-local browse index in `browse-index.json`,
+and the unzipped Knowhere result files under `result/`.
+
+The important package files are:
+
+- `metadata.json`: plugin-local sidecar. It maps the local `docId`, source labels, tags, timestamps, and job metadata to the extracted Knowhere package under `result/`.
+- `browse-index.json`: plugin-local browse helper. It tracks structural paths, chunk order, and result-file inventory for fast path and file browsing.
+- `manifest.json`: package inventory. Read this first. It tells you what optional files and assets exist and carries `job_id`, `data_id`, `source_file_name`, checksum, and statistics.
+- `chunks.json`: primary retrieval dataset. It contains stable chunk IDs, chunk types, `path`, `content`, summaries, and metadata.
+- `hierarchy.json`: document tree rebuilt from parser paths. Use it when structural context matters and the raw path tree is not enough.
+- `full.md`: convenience whole-document markdown view. Good for broad skimming, but do not treat it as the only grounding source.
+- `images/` and `tables/`: raw extracted assets referenced by chunk metadata and manifest entries. The plugin stores these untouched.
+- `kb.csv`: raw parser export when present. Use it when you need parser rows, joins, or custom downstream transforms.
+
+Each stored chunk exposes these core fields through the chunk tools:
+
+- `chunkId`: stable chunk identifier
+- `type`: `text`, `image`, or `table`
+- `path`: best-effort hierarchy/path label
+- `summary`: chunk summary
+- `content`: original stored text payload
+- `contentLength`: character count of the content field (available in summary mode to help budget reads)
+- `tokens`: parser token hint when present
+- `keywords`: search hints from the parser
+- `relationships`: best-effort related chunk metadata
+- `assetFilePath`: relative path to the stored asset for image/table chunks
+
+Useful joins:
+
+- `manifest.files.images[].id == chunks[].chunkId` for image chunks
+- `manifest.files.tables[].id == chunks[].chunkId` for table chunks
+- `chunks[].assetFilePath` points to the stored asset path under `result/`
+
+## String truncation
+
+Chunk and file tools truncate string fields to `maxStringChars` (default 4000).
+When `truncatedStrings: true` appears, retry with a higher value (e.g. 12000, up to 20000) to get full content.
+
+## Tool selection
+
+- `knowhere_ingest_document` for new local files or URLs
+- `knowhere_list_documents` to find candidate document IDs in the current scope
+- `knowhere_read_result_file` for `manifest.json`, `hierarchy.json`, `kb.csv`, table HTML files, or other text-like files under `result/`
+- `knowhere_preview_document` for a quick overview: markdown preview plus the structural path tree (like a book index)
+- `knowhere_grep` to search chunks with composable AND conditions across fields. This is the recommended default for text search — just pass `conditions: [{ pattern: "your query" }]` with no target to search all text fields at once. Use targeted conditions only when you need to narrow by specific fields like `chunk.type` or `chunk.path`.
+- `knowhere_list_jobs`, `knowhere_get_job_status`, and `knowhere_import_completed_job` for async Knowhere jobs
+- `knowhere_remove_document` and `knowhere_clear_scope` for cleanup
+
+After ingesting a document, use the returned document or job identifiers for follow-up operations instead of guessing names.
+
+## Recommended workflow
+
+1. Ingest or import the document if it is not already in the store.
+2. Call `knowhere_list_documents` if you need to confirm the right `docId`.
+3. Call `knowhere_preview_document` to get a structural overview (table of contents with summaries).
+4. When you know what to search for, call `knowhere_grep` with `conditions: [{ pattern: "your query" }]` — this searches all text fields (content, summary, keywords, path) in one call. Add more conditions to narrow results (e.g. filter by `chunk.type` or `chunk.path`).
+5. Call `knowhere_grep` with a path condition to narrow results to a specific branch when browsing by structure.
+6. Call `knowhere_read_result_file` for `hierarchy.json`, `kb.csv`, or table HTML when the answer depends on parser rows or rich table structure.
+
+## Reasoning rules
+
+- Prefer `knowhere_grep` for all text search. It supports composable AND conditions, regex, and normalizes HTML/LaTeX/unicode before matching. Use `knowhere_preview_document` when you need a quick overview and structural browsing by path.
+- Use `knowhere_preview_document` before broad reads when the document is large or the relevant branch is unclear.
+- Keep `path` in your reasoning and in your answer when possible. It restores section position and improves grounding.
+- Cite `chunkId` and `path` when answering from retrieved chunks.
+- For image or table questions, inspect matching `image` or `table` chunks and the related manifest asset entries before answering.
+- Do not rely on `full.md` alone if the question depends on exact section boundaries, tables, or images.
+- If the task needs raw `kb.csv`, raw HTML tables, or another stored text file under `result/`, read it directly with `knowhere_read_result_file`.
+- When a tool response contains `truncatedStrings: true`, retry with `maxStringChars: 12000` (or up to 20000) before answering from incomplete content.
+
+## Tool usage examples
+
+Use a real stored `docId` returned by `knowhere_ingest_document`, `knowhere_import_completed_job`, or `knowhere_list_documents`.
+
+Ingest a local file:
+
+```json
+{
+  "filePath": "/tmp/uploads/handbook.pdf",
+  "fileName": "handbook.pdf"
+}
+```
+
+Ingest a URL:
+
+```json
+{
+  "url": "https://example.com/report-2026.pdf",
+  "title": "Q1 Report"
+}
+```
+
+Check job status:
+
+```json
+{
+  "jobId": "job_f6f12930906a"
+}
+```
+
+Import a completed job:
+
+```json
+{
+  "jobId": "job_f6f12930906a"
+}
+```
+
+Read manifest JSON:
+
+```json
+{
+  "docId": "handbook-1234",
+  "filePath": "manifest.json",
+  "mode": "json"
+}
+```
+
+Preview a document:
+
+```json
+{
+  "docId": "handbook-1234"
+}
+```
+
+Grep for text across all chunk fields (recommended default):
+
+```json
+{
+  "docId": "paper-pdf-a370ef58",
+  "conditions": [{ "pattern": "npm audit" }]
+}
+```
+
+Grep with multiple AND conditions (e.g. table chunks mentioning "precision"):
+
+```json
+{
+  "docId": "paper-pdf-a370ef58",
+  "conditions": [{ "target": "chunk.type", "pattern": "table" }, { "pattern": "precision" }]
+}
+```
+
+Grep with path filter and sibling context:
+
+```json
+{
+  "docId": "paper-pdf-a370ef58",
+  "conditions": [{ "target": "chunk.path", "pattern": "EVALUATION" }],
+  "includeContext": true
+}
+```
+
+Grep with regex:
+
+```json
+{
+  "docId": "paper-pdf-a370ef58",
+  "conditions": [{ "pattern": "\\bF1[- ]score\\b", "regex": true }]
+}
+```
+
+Read a table HTML file:
+
+```json
+{
+  "docId": "handbook-1234",
+  "filePath": "tables/table-1.html",
+  "mode": "text"
+}
+```
+
+Example workflow for a question like `What does the handbook say about hotel limits?`:
+
+1. Call `knowhere_list_documents` to find the right `docId` if you do not already have it.
+2. Call `knowhere_read_result_file` with `filePath: "manifest.json"` to confirm the package layout.
+3. Call `knowhere_preview_document` with `mode: "paths"` and look for a relevant branch such as `Travel` or `Expenses`.
+4. Call `knowhere_grep` with `conditions: [{ "pattern": "hotel" }]` to find matching chunks, or add a path condition to narrow to the branch.
+5. If the answer depends on a table, read the linked `assetFilePath` with `knowhere_read_result_file`.
+6. Answer from the matched chunks and cite `chunkId` and `path`.
+
+Do not rely on prompt-loaded excerpts as the main workflow. Prefer reading the stored result directly from disk.
+
+If Knowhere returns an API error, quote the error directly in your reply.