@majkapp/plugin-kit 3.7.9 → 3.7.11

package/docs/KNOWLEDGE.md

@@ -1,225 +1,490 @@
 # Knowledge API
 
-Store and search knowledge nodes organized into trees with reference tracking.
+MAJK provides two knowledge systems:
 
-## Quick Start
+1. **Knowledge Fabric** (Recommended) - Unified semantic search across all system knowledge with vector embeddings. Use this for searching, browsing, and contributing plugin knowledge.
+
+2. **Legacy Knowledge Trees** - Conversation-scoped knowledge trees. Still available but deprecated in favor of the Fabric.
+
+**Start with the Knowledge Fabric** - it's more powerful, searches across everything, and supports plugin contributions.
+
+---
+
+## Quick Start (Knowledge Fabric)
 
 ```typescript
 handler: async (input, ctx) => {
-  // Add knowledge
-  await ctx.majk.knowledge.add({
-    conversationId: 'conv-123',
-    title: 'API Design Decision',
-    content: 'We decided to use REST over GraphQL because...',
-    references: ['file:src/api/routes.ts#L10-50'],
-    tree: 'decisions'
+  const fabric = ctx.majk.knowledge.fabric;
+
+  // Search all knowledge semantically
+  const results = await fabric.search('authentication patterns');
+
+  // Register a contribution namespace
+  const handle = await fabric.registerContribution('contacts', {
+    initialNodes: [
+      { key: 'john', title: 'John Doe', content: 'CEO at Acme Corp, key decision maker...' }
+    ]
   });
 
-  // Search knowledge
-  const nodes = await ctx.majk.knowledge.search('REST API', {
-    conversationId: 'conv-123'
+  // Add more nodes dynamically
+  await handle.contribute({
+    key: 'jane',
+    title: 'Jane Smith',
+    content: 'VP Engineering, technical buyer...'
   });
 
-  return { found: nodes.length };
+  return { searchResults: results.results.length };
 }
 ```
 
-## API Reference
+## Knowledge Fabric API
+
+The Knowledge Fabric provides unified semantic search across:
+- Project notes (knowledge trees)
+- Teammates
+- Tools
+- Plugins
+- Skills
+- Reports
+- Plugin-contributed knowledge
+
+Access via `ctx.majk.knowledge.fabric`.
 
-### add(input)
+### search(query, options?)
 
-Add a knowledge node.
+Semantic search across all knowledge.
 
 ```typescript
-await ctx.majk.knowledge.add({
-  conversationId: string,   // Conversation to associate with
-  title: string,            // Node title
-  content: string,          // Knowledge content
-  references?: string[],    // Reference links (files, URLs, etc.)
-  tree?: string             // Tree name (default: 'default')
+const response = await ctx.majk.knowledge.fabric.search('authentication', {
+  under: '/projects/my-project',  // Scope to prefix
+  limit: 10,                       // Max results (default: 10, max: 100)
+  minScore: 0.3                    // Minimum similarity (0-1)
 });
+
+// Response structure
+{
+  results: [
+    {
+      node: KnowledgeFabricNode,
+      score: 0.95  // Similarity score (1 = most similar)
+    }
+  ],
+  indexingStatus: {
+    complete: true,     // Are all nodes indexed?
+    pendingCount: 0     // Nodes still being embedded
+  }
+}
 ```
 
-### search(query, options?)
+### getRoots()
 
-Search knowledge by text.
+Get top-level nodes of the knowledge hierarchy.
 
 ```typescript
-const nodes = await ctx.majk.knowledge.search(query, {
-  conversationId?: string,  // Filter by conversation
-  tree?: string,            // Filter by tree
-  limit?: number            // Max results
-});
-// Returns: KnowledgeNode[]
+const roots = await ctx.majk.knowledge.fabric.getRoots();
+// Returns: ['/projects', '/plugins', '/teammates', '/tools', ...]
 ```
 
-### getByReference(ref)
+### expand(key)
 
-Find nodes that reference a specific resource.
+Expand a node to see its children.
 
 ```typescript
-const nodes = await ctx.majk.knowledge.getByReference('file:src/api.ts');
-// Returns: KnowledgeNode[]
+const result = await ctx.majk.knowledge.fabric.expand('/plugins/my-plugin');
+// Returns: { node: KnowledgeFabricNode, children: KnowledgeFabricNode[] }
 ```
 
-### getTree(conversationId, treeName)
+### getNode(key)
 
-Get all nodes in a tree.
+Get a single node by key.
 
 ```typescript
-const tree = await ctx.majk.knowledge.getTree('conv-123', 'decisions');
-// Returns: { name, conversationId, nodes: KnowledgeNode[] }
+const node = await ctx.majk.knowledge.fabric.getNode('/projects/proj-123/notes/a1');
+// Returns: KnowledgeFabricNode | null
 ```
 
-### listTrees(conversationId)
+### getStatus()
 
-List all trees in a conversation.
+Get overall fabric status.
 
 ```typescript
-const trees = await ctx.majk.knowledge.listTrees('conv-123');
-// Returns: string[] (tree names)
+const status = await ctx.majk.knowledge.fabric.getStatus();
+// Returns: {
+//   totalNodes: 1500,
+//   activeNodes: 1450,
+//   pendingNodes: 50,
+//   totalEmbeddings: 1450,
+//   searchAvailable: true
+// }
 ```
 
-## Types
+## Contributing Knowledge
+
+Plugins can contribute their own knowledge to the fabric, making it searchable alongside system knowledge.
+
+### registerContribution(subPrefix, options?)
+
+Register a namespace for contributing knowledge.
 
 ```typescript
-interface KnowledgeNode {
-  id: string;
-  title: string;
-  content: string;
-  references: string[];
-  conversationId: string;
-  tree: string;
-  createdAt: Date;
-}
+// Default prefix: /plugins/{pluginId}/{subPrefix}
+const handle = await ctx.majk.knowledge.fabric.registerContribution('contacts');
+// Prefix: /plugins/my-plugin/contacts
 
-interface KnowledgeTree {
-  name: string;
-  conversationId: string;
-  nodes: KnowledgeNode[];
-}
+// With custom prefix (except reserved: /projects, /teammates, /tools, /skills, /reports)
+const handle = await ctx.majk.knowledge.fabric.registerContribution('contacts', {
+  prefix: '/crm/contacts'  // Custom prefix
+});
+
+// With initial nodes
+const handle = await ctx.majk.knowledge.fabric.registerContribution('contacts', {
+  initialNodes: [
+    { key: 'c1', title: 'Contact 1', content: 'Details...' },
+    { key: 'c2', title: 'Contact 2', content: 'Details...' }
+  ]
+});
 ```
 
-## Reference Patterns
+### KnowledgeContributionHandle
 
-Use consistent reference formats:
+The handle returned by `registerContribution()` manages your contribution.
 
 ```typescript
-// File references
-'file:src/api/routes.ts'
-'file:src/api/routes.ts#L10-50'
+interface KnowledgeContributionHandle {
+  readonly prefix: string;        // Full key prefix
+  readonly contributorId: string; // plugin:{pluginId}:{subPrefix}
 
-// URL references
-'url:https://docs.example.com/api'
+  // Add/update a node
+  contribute(node: ContributeNodeInput): Promise<void>;
 
-// Function references
-'function:createUser'
-'function:src/api.ts:createUser'
+  // Add multiple nodes efficiently
+  contributeBatch(nodes: ContributeNodeInput[]): Promise<void>;
 
-// Commit references
-'commit:abc123'
+  // Remove a node
+  remove(key: string): Promise<boolean>;
 
-// Issue/PR references
-'issue:42'
-'pr:123'
+  // Get node count
+  getNodeCount(): Promise<number>;
+
+  // List all keys
+  listKeys(options?: { status?: 'active' | 'pending' | 'all' }): Promise<string[]>;
+
+  // Re-sync contribution (deactivates nodes not re-contributed)
+  reconcile(): Promise<{ added: number; updated: number; deactivated: number }>;
+
+  // Unregister (marks all nodes inactive)
+  unregister(): Promise<void>;
+}
 ```
 
-## Common Patterns
+### contribute(node)
 
-### Organize by Topic
+Add or update a single node.
 
 ```typescript
-// Add to different trees
-await ctx.majk.knowledge.add({
-  conversationId,
-  title: 'Auth Flow Decision',
-  content: 'Using JWT tokens...',
-  tree: 'architecture'
+await handle.contribute({
+  key: 'customer-123',           // Relative to prefix
+  title: 'ACME Corporation',     // Human-readable title
+  content: 'Enterprise customer, 500 employees, using premium tier...',
+  children: ['contact-1', 'contact-2'],  // Optional child keys
+  refs: {                        // Optional metadata
+    source: 'crm-sync',
+    lastUpdated: Date.now()
+  }
 });
+// Full key: /plugins/my-plugin/contacts/customer-123
+```
 
-await ctx.majk.knowledge.add({
-  conversationId,
-  title: 'JWT Security Concern',
-  content: 'Token expiry handling...',
-  tree: 'security'
-});
+### contributeBatch(nodes)
 
-// Query specific tree
-const archNodes = await ctx.majk.knowledge.getTree(conversationId, 'architecture');
+Add multiple nodes efficiently.
+
+```typescript
+await handle.contributeBatch([
+  { key: 'c1', title: 'Contact 1', content: 'Details...' },
+  { key: 'c2', title: 'Contact 2', content: 'Details...' },
+  { key: 'c3', title: 'Contact 3', content: 'Details...' }
+]);
 ```
 
-### Track Code References
+### remove(key)
+
+Remove a node by key.
 
 ```typescript
-// Add knowledge about code
-await ctx.majk.knowledge.add({
-  conversationId,
-  title: 'Rate Limiter Implementation',
-  content: 'The rate limiter uses a sliding window algorithm...',
-  references: [
-    'file:src/middleware/rateLimiter.ts',
-    'file:src/config/limits.ts#L5-20',
-    'url:https://en.wikipedia.org/wiki/Sliding_window_protocol'
-  ],
-  tree: 'implementation'
+const removed = await handle.remove('customer-123');
+// Also works with full key:
+const removed = await handle.remove('/plugins/my-plugin/contacts/customer-123');
+```
+
+### listContributions()
+
+List all contributions registered by your plugin.
+
+```typescript
+const contributions = await ctx.majk.knowledge.fabric.listContributions();
+// Returns: ContributionSummary[]
+// {
+//   prefix: '/plugins/my-plugin/contacts',
+//   contributorId: 'plugin:my-plugin:contacts',
+//   nodeCount: 42,
+//   pendingCount: 3,
+//   status: 'ready' | 'indexing' | 'unregistered'
+// }
+```
+
+### onNodeChange(handler, options?)
+
+Subscribe to node change events.
+
+```typescript
+const unsubscribe = ctx.majk.knowledge.fabric.onNodeChange((event) => {
+  console.log(`Node ${event.key} was ${event.type}`);
+  // event.type: 'indexed' | 'activated' | 'deactivated' | 'reindexed'
 });
 
-// Later, find all knowledge about a file
-const related = await ctx.majk.knowledge.getByReference('file:src/middleware/rateLimiter.ts');
+// Later: unsubscribe();
 ```
 
-### Build Context for Analysis
+## Types
 
 ```typescript
-async function getProjectContext(conversationId: string) {
-  const trees = await ctx.majk.knowledge.listTrees(conversationId);
+interface KnowledgeFabricNode {
+  key: string;                    // Hierarchical path
+  title: string;                  // Human-readable title
+  content: string;                // Searchable content
+  children: string[];             // Child node keys
+  refs: Record<string, unknown>;  // Metadata/references
+  contributorId: string;          // Who contributed this
+  status: 'active' | 'pending' | 'inactive';
+  createdAt: number;              // Timestamp
+  lastContributed: number;        // Timestamp
+}
 
-  const context = {};
-  for (const treeName of trees) {
-    const tree = await ctx.majk.knowledge.getTree(conversationId, treeName);
-    context[treeName] = tree.nodes.map(n => ({
-      title: n.title,
-      summary: n.content.slice(0, 200)
-    }));
-  }
+interface ContributeNodeInput {
+  key: string;                    // Relative to prefix
+  title: string;
+  content: string;
+  children?: string[];            // Optional child keys
+  refs?: Record<string, unknown>; // Optional metadata
+}
+
+interface FabricSearchOptions {
+  under?: string;     // Scope to key prefix
+  limit?: number;     // Max results (1-100)
+  minScore?: number;  // Min similarity (0-1)
+}
+
+interface FabricSearchResponse {
+  results: Array<{
+    node: KnowledgeFabricNode;
+    score: number;
+  }>;
+  indexingStatus: {
+    complete: boolean;
+    pendingCount: number;
+  };
+}
+```
+
+## Common Patterns
 
-  return context;
+### CRM Integration
+
+```typescript
+// Sync contacts from external CRM
+async function syncContacts(ctx, crmClient) {
+  const handle = await ctx.majk.knowledge.fabric.registerContribution('contacts');
+
+  const contacts = await crmClient.getAllContacts();
+
+  await handle.contributeBatch(contacts.map(c => ({
+    key: c.id,
+    title: c.name,
+    content: `
+      ${c.name} - ${c.company}
+      Role: ${c.title}
+      Email: ${c.email}
+      Notes: ${c.notes}
+      Tags: ${c.tags.join(', ')}
+    `,
+    refs: {
+      crmId: c.id,
+      company: c.company,
+      lastSync: Date.now()
+    }
+  })));
 }
+
+// Search contacts with natural language
+const results = await ctx.majk.knowledge.fabric.search(
+  'decision maker at enterprise companies'
+);
 ```
 
-### Search Across Everything
+### Documentation Indexer
 
 ```typescript
-// Broad search
-const results = await ctx.majk.knowledge.search('authentication');
+// Index documentation files
+async function indexDocs(ctx, docsPath) {
+  const handle = await ctx.majk.knowledge.fabric.registerContribution('docs');
+
+  const files = await glob(`${docsPath}/**/*.md`);
+
+  for (const file of files) {
+    const content = await fs.readFile(file, 'utf-8');
+    const title = extractTitle(content);
+
+    await handle.contribute({
+      key: path.relative(docsPath, file),
+      title,
+      content,
+      refs: {
+        filePath: file,
+        indexed: Date.now()
+      }
+    });
+  }
+}
+
+// Search documentation
+const results = await ctx.majk.knowledge.fabric.search('how to configure authentication');
+```
+
+### Scoped Search
 
-// Filtered search
-const securityResults = await ctx.majk.knowledge.search('authentication', {
-  conversationId,
-  tree: 'security',
+```typescript
+// Search only within your plugin's knowledge
+const handle = await ctx.majk.knowledge.fabric.registerContribution('data');
+
+const results = await ctx.majk.knowledge.fabric.search('important topic', {
+  under: handle.prefix,  // Only search your contribution
   limit: 5
 });
+
+// Search within a specific project
+const projectResults = await ctx.majk.knowledge.fabric.search('auth', {
+  under: '/projects/proj-abc123'
+});
+```
+
+### Reconciliation
+
+```typescript
+// Re-sync when source data changes
+async function resyncData(ctx) {
+  const handle = await ctx.majk.knowledge.fabric.registerContribution('data');
+
+  // Contribute current data
+  const currentData = await fetchCurrentData();
+  await handle.contributeBatch(currentData.map(d => ({
+    key: d.id,
+    title: d.name,
+    content: d.description
+  })));
+
+  // Reconcile - deactivates nodes not in currentData
+  const stats = await handle.reconcile();
+  console.log(`Added: ${stats.added}, Updated: ${stats.updated}, Removed: ${stats.deactivated}`);
+}
+```
+
+### Browse Hierarchy
+
+```typescript
+// Build a tree view of knowledge
+async function browseKnowledge(ctx) {
+  const fabric = ctx.majk.knowledge.fabric;
+
+  // Get root nodes
+  const roots = await fabric.getRoots();
+
+  // Expand to see children
+  for (const root of roots) {
+    const expanded = await fabric.expand(root.key);
+    console.log(`${root.title}:`);
+    for (const child of expanded?.children || []) {
+      console.log(`  - ${child.title}`);
+    }
+  }
+}
 ```
 
+## Reserved Prefixes
+
+These prefixes are reserved for system use and cannot be used for custom contributions:
+
+- `/projects` - Project notes and knowledge trees
+- `/teammates` - Teammate configurations
+- `/tools` - Tool definitions
+- `/skills` - Skill definitions
+- `/reports` - Report storage
+- `/characterizations` - System characterizations
+
+Use the default `/plugins/{pluginId}/...` prefix or choose a custom unreserved prefix.
+
 ## Best Practices
 
-1. **Use meaningful titles** - Clear, searchable node titles
-2. **Consistent tree names** - Standardize tree names across conversations
-3. **Rich references** - Link to files, URLs, commits for traceability
-4. **Structured content** - Use markdown for formatted content
-5. **Search-friendly content** - Include keywords that users might search for
+1. **Use meaningful content** - Include keywords and context for better search results
+2. **Structure with hierarchy** - Use parent/child relationships for organized browsing
+3. **Add metadata in refs** - Store source information, timestamps, IDs for traceability
+4. **Reconcile periodically** - Keep knowledge in sync with external sources
+5. **Scope searches** - Use `under` option to search specific namespaces
+6. **Handle indexing status** - Check `indexingStatus.complete` for real-time accuracy
+
+---
+
+## Legacy Knowledge Trees API (Deprecated)
+
+> **Note:** The following methods work with conversation-scoped knowledge trees. They are deprecated in favor of the Knowledge Fabric which provides unified search across all knowledge. Use `ctx.majk.knowledge.fabric` for new development.
+
+### add(input) [DEPRECATED]
+
+```typescript
+// DEPRECATED - use fabric.registerContribution() instead
+await ctx.majk.knowledge.add({
+  conversationId: string,
+  title: string,
+  content: string,
+  references?: string[],
+  tree?: string
+});
+```
+
+### search(query, options?) [DEPRECATED]
+
+```typescript
+// DEPRECATED - use fabric.search() instead
+const nodes = await ctx.majk.knowledge.search(query, {
+  conversationId?: string,
+  tree?: string,
+  limit?: number
+});
+```
 
-## Tree Organization Examples
+### getByReference(ref) [DEPRECATED]
 
+```typescript
+// DEPRECATED - use fabric for knowledge operations
+const nodes = await ctx.majk.knowledge.getByReference('file:src/api.ts');
 ```
-decisions/       - Architecture and design decisions
-security/        - Security considerations and findings
-implementation/  - How things are implemented
-bugs/           - Known issues and their context
-research/       - Background research and findings
+
+### getTree(conversationId, treeName) [DEPRECATED]
+
+```typescript
+// DEPRECATED - use fabric for knowledge operations
+const tree = await ctx.majk.knowledge.getTree('conv-123', 'decisions');
+```
+
+### listTrees(conversationId) [DEPRECATED]
+
+```typescript
+// DEPRECATED - use fabric for knowledge operations
+const trees = await ctx.majk.knowledge.listTrees('conv-123');
 ```
 
 ## Next Steps
 
-Run `npx @majkapp/plugin-kit --reports` - Reports and knowledge base
-Run `npx @majkapp/plugin-kit --context` - Context API overview
+Run `npx @majkapp/plugin-kit --llm:reports` - Reports and knowledge management
+Run `npx @majkapp/plugin-kit --llm:context` - Context API overview
+Run `npx @majkapp/plugin-kit --llm:skills` - Skills system

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@majkapp/plugin-kit",
-  "version": "3.7.9",
+  "version": "3.7.11",
   "description": "Pure plugin definition library for MAJK - outputs plugin definitions, not HTTP servers",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",