pi-doc-injector 0.3.1 → 0.4.0

package/README.md

@@ -27,7 +27,23 @@ git clone https://github.com/yourname/pi-doc-injector.git .pi/extensions/doc-inj
 ## Quick Start
 
 1. Create a `docs/` folder in your project root.
-2. Add markdown files with YAML frontmatter:
+2. Add markdown files with frontmatter (`title` + `keywords`). See [Document Format](#document-format) for supported formats.
+3. Start Pi. The extension scans `docs/` on session start.
+4. When the user mentions a keyword, the matching doc is injected into the
+   system prompt **before the assistant responds** — no one-turn delay.
+5. If the assistant mentions a NEW keyword mid-response, generation is
+   automatically aborted and restarted with the doc injected immediately.
+
+## Document Format
+
+Documents are markdown files (`.md` or `.txt`) that the extension scans for injection.
+Each file can declare `title` and `keywords` via **frontmatter** — a metadata block at the top of the file.
+
+### Supported Frontmatter Formats
+
+The extension tries formats in this order and uses the first match it finds:
+
+**1. YAML (recommended)**
 
 ```md
 ---
@@ -36,28 +52,57 @@ keywords: [test, testing, jest, vitest]
 ---
 
 # Testing Workflow
+```
+
+**2. C-style block comment** — useful for `.ts`/`.js` doc files:
+
+```md
+/*---
+title: "Testing Workflow"
+keywords: [test, testing, jest, vitest]
+---*/
 
-Your documentation here...
+# Testing Workflow
 ```
 
-Keywords can also be specified in block format:
+**3. HTML comment** — useful for HTML-generated docs:
 
 ```md
----
+<!--
 title: "Testing Workflow"
-keywords:
+keywords: [test, testing, jest, vitest]
+-->
+
+# Testing Workflow
+```
+
+**4. Slash-slash comment** — useful for `.js`/`.ts` sidecar docs:
+
+```md
+//---
+title: "Testing Workflow"
+keywords: [test, testing, jest, vitest]
+
+# Testing Workflow
+```
+
+### Keyword Array Syntax
+
+Both **flow** and **block** keyword array syntaxes are supported:
+
+```md
+keywords: [test, testing, jest]          # flow: comma-separated in brackets
+keywords:                              # block: one per line
   - test
   - testing
   - jest
-  - vitest
----
 ```
 
-3. Start Pi. The extension scans `docs/` on session start.
-4. When the user mentions a keyword, the matching doc is injected into the
-   system prompt **before the assistant responds** — no one-turn delay.
-5. If the assistant mentions a NEW keyword mid-response, generation is
-   automatically aborted and restarted with the doc injected immediately.
+### Auto-Keywords Fallback
+
+If a file has **no frontmatter** and `autoKeywords` is enabled (default: `true`), the extension generates keywords heuristically from the filename and content — no metadata needed.
+
+If `autoKeywords` is `false`, files without valid frontmatter are **skipped** with a warning.
 
 ## Configuration
 
@@ -68,7 +113,10 @@ Create `.pi/doc-injector.json` in your project root to customize behavior:
   "docsPath": "./docs",
   "matchThreshold": 1,
   "contextThreshold": 80,
-  "recursive": true
+  "recursive": true,
+  "autoKeywords": true,
+  "llmKeywords": false,
+  "llmBatchSize": 20
 }
 ```
 
@@ -78,6 +126,9 @@ Create `.pi/doc-injector.json` in your project root to customize behavior:
 | `matchThreshold`   | `1`        | Minimum keyword matches required to inject a doc         |
 | `contextThreshold` | `80`       | Skip injection when context usage exceeds this % (0–100) |
 | `recursive`        | `true`     | Scan docs subdirectories recursively                     |
+| `autoKeywords`     | `true`     | Generate keywords heuristically when frontmatter is missing |
+| `llmKeywords`      | `false`    | Enable LLM-based keyword generation (see below)          |
+| `llmBatchSize`     | `20`       | Max files per LLM keyword batch                          |
 
 ### Keyword Matching
 
@@ -96,6 +147,50 @@ Injection is also skipped if the current context usage exceeds 80% of the token
 | `/doc-inject reset`  | Reset all injected flags (docs become re-injectable) |
 | `/doc-inject status` | Show current injection status and config             |
 | `/doc-reload`        | Re-scan docs folder and rebuild registry             |
+| `/doc-keywords-gen`  | Generate LLM keywords for files without frontmatter (requires `llmKeywords: true` in config) |
+
+## Keyword Generation
+
+When a document has no frontmatter keywords, the extension handles it in two ways:
+
+### Heuristic (Automatic)
+
+If `autoKeywords` is `true` (default), keywords are generated automatically from:
+
+- **Filename parts**: `"api-authentication.md"` → `[api, authentication]`
+- **Markdown headings**: `"# Getting Started"` → `[getting, started]`
+- **Code symbols**: `"function foo()"` → `[foo]`
+
+All keywords are filtered through a stop-word list, lowercased, and capped at 20.
+
+### LLM Generation (Manual)
+
+For better keywords, enable LLM generation in config:
+
+```json
+{
+  "autoKeywords": true,
+  "llmKeywords": true,
+  "llmBatchSize": 20
+}
+```
+
+Then run `/doc-keywords-gen [path]` to generate keywords via LLM. Without a path argument, it processes all keyword-less files.
+
+The LLM reads each file's content and produces 3–10 relevant, searchable keywords per file. Results are saved to the cache and reused on subsequent scans.
+
+### Keyword Source Tracking
+
+The cache stores which method was used for each file's keywords:
+
+| Source       | How set                                          |
+| ------------ | ------------------------------------------------ |
+| `frontmatter` | Keywords declared in file frontmatter             |
+| `cache`      | Reused from previous scan (mtime match)          |
+| `heuristic`  | Auto-generated from filename/content             |
+| `llm`        | Generated via `/doc-keywords-gen`                |
+
+Use `/doc-inject list` to see each file's keyword source (shown as `[source]` tag).
 
 ## Injection Lifecycle
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-doc-injector",
-  "version": "0.3.1",
+  "version": "0.4.0",
   "description": "Auto-inject relevant project documentation into Pi's LLM context based on keyword matching",
   "type": "module",
   "main": "./index.ts",