regen-koi-mcp 1.0.6 → 1.2.0

package/README.md

@@ -12,6 +12,8 @@ curl -fsSL https://raw.githubusercontent.com/gaiaaiagent/regen-koi-mcp/main/inst
 
 This automatically configures Claude Desktop and Claude Code CLI. Just restart and you're done! 🎉
 
+**Quick Test:** After restarting, open Claude and ask: _"What repositories are indexed in KOI?"_ to verify the tools are working.
+
 ---
 
 ### Option 1: NPM (Recommended - Auto-Updates)
@@ -45,6 +47,8 @@ Config file locations:
 
 Then restart Claude Desktop and you're done! 🎉
 
+**Quick Test:** Ask Claude: _"What repositories are indexed in KOI?"_ to verify the tools are working.
+
 **For existing git users**: See the migration section below for a simple one-line script to switch to npx.
 
 ---
@@ -101,6 +105,93 @@ Want complete control including data collection? You'll need:
 
 See [ARCHITECTURE.md](ARCHITECTURE.md) for detailed setup instructions and architecture overview.
 
+## 🔐 Authentication (Optional - Team Members Only)
+
+### Public Data (No Auth Required)
+KOI provides access to extensive public Regen Network knowledge:
+- ✅ Regen Ledger code and documentation
+- ✅ Regen Network forum discussions
+- ✅ Public GitHub repositories
+- ✅ Website content and blogs
+
+**Most users don't need authentication** - just install and start querying!
+
+### Internal Documentation (Regen Network Team)
+Regen Network team members with `@regen.network` emails can optionally authenticate to access additional internal documentation.
+
+**To authenticate:**
+```
+User: "Please use the regen_koi_authenticate tool"
+
+MCP: Opening browser for authentication...
+     [Browser opens to Google OAuth]
+
+User: [Log in with yourname@regen.network]
+      [Grant permissions: email, profile]
+
+MCP: ✅ Authentication successful!
+     Authenticated as: yourname@regen.network
+```
+
+**After authentication:**
+- Queries automatically include internal documentation
+- Token is saved on the server (not your machine)
+- No need to re-authenticate unless token expires (~7 days)
+
+### What Permissions Are Requested
+
+The OAuth flow requests **minimal, identity-only permissions**:
+- ✅ `openid` - Verify your identity
+- ✅ `email` - Confirm you're @regen.network
+- ✅ `profile` - Basic profile info
+
+**NOT requested:**
+- ❌ Access to your personal Drive files
+- ❌ Access to your Gmail
+- ❌ Any write/modification permissions
+
+The service account (not your OAuth token) handles all data access.
+
+### Example: Team Member Workflow
+
+```
+# First time - authenticate
+User: "Use regen_koi_authenticate to authenticate me"
+
+MCP: Opening browser...
+     ✅ Authenticated as: alice@regen.network
+
+# Now all queries include internal docs
+User: "How do I register a carbon project?"
+
+MCP: Found 23 results from multiple sources:
+     - Project Registration Guide (2024)
+     - Registry Standards Documentation
+     - Carbon Methodology Overview
+     ...
+```
+
+### Token Security
+
+Your authentication tokens are:
+- 🔒 Stored server-side only (never on your machine)
+- 🔒 Encrypted in PostgreSQL database
+- 🔒 User-specific (never shared between users)
+- 🔒 Automatically refreshed when needed
+
+### Troubleshooting
+
+**"Authentication window opened but nothing happened"**
+- Check if your browser blocked the popup
+- Try manually clicking the URL shown in the terminal
+
+**"Access denied" after logging in**
+- Ensure you're using your `@regen.network` email (not personal Gmail)
+- Contact your Workspace admin if you don't have access
+
+**Still having issues?**
+- Open an issue: https://github.com/gaiaaiagent/regen-koi-mcp/issues
+
 ## 🎯 What This Does
 
 This MCP server gives AI assistants access to Regen Network's comprehensive knowledge base with 15,000+ documents about:
@@ -112,6 +203,66 @@ This MCP server gives AI assistants access to Regen Network's comprehensive know
 
 **Note:** This MCP server connects to our hosted KOI API at `https://regen.gaiaai.xyz/api/koi` (behind HTTPS via Nginx), so you don't need to run any infrastructure locally.
 
+## 🎓 Quick Start: Example Queries
+
+Once you've installed the MCP server, try these queries in Claude to explore what's available:
+
+### 🔍 Discovery: What's Indexed?
+
+```
+"What repositories are indexed in the KOI knowledge base?"
+"What types of code entities exist?"
+"Show me statistics about what's in the knowledge base"
+```
+
+**What you'll get:** The system has indexed 5 repositories (regen-ledger, regen-web, koi-sensors, and more) with 26,768 code entities including Methods, Functions, Structs, and Interfaces.
+
+### 💻 Code Exploration: Find Specific Code
+
+```
+"Search for functions containing 'keeper' in regen-ledger"
+"Find all code related to MsgCreateBatch"
+"What Structs exist in the ecocredit module?"
+"Show me all Interface types in regen-ledger"
+"Search for retirement-related code"
+```
+
+**What you'll get:** Direct links to code entities with file paths and signatures. The graph contains Methods, Functions, Structs, and Interfaces extracted via tree-sitter AST parsing.
+
+### 📚 Documentation: Understanding the System
+
+```
+"Explain the ecocredit module architecture"
+"What's the tech stack for regen-web?"
+"Search GitHub docs for information about credit classes"
+"Get an overview of the regen-ledger repository"
+```
+
+**What you'll get:** Architectural explanations, technology choices, and relevant documentation from GitHub repositories.
+
+### 🔬 Advanced: Graph Traversal
+
+```
+"What functions call CreateBatch?"
+"What does the NewKeeper function call?"
+"Find orphaned code that's never called"
+"Show me the call graph for MsgRetire"
+```
+
+**What you'll get:** Call graph traversal showing function relationships (CALLS edges), orphan detection, and code dependency analysis.
+
+### 📊 Knowledge Base: Recent Activity
+
+```
+"Generate a weekly digest of Regen Network discussions"
+"Search for recent discussions about carbon credits from the past week"
+"Find documentation about basket creation from 2024"
+```
+
+**What you'll get:** Summaries of community discussions, forum posts, and activity with source citations.
+
+---
+
 ## 📦 Available Tools
 
 ### Knowledge Base Search
@@ -120,7 +271,8 @@ This MCP server gives AI assistants access to Regen Network's comprehensive know
 | `search_knowledge` | Hybrid search (vectors + graph with RRF) | `query` (string), `limit` (1–20, default 5), `published_from` (YYYY‑MM‑DD), `published_to` (YYYY‑MM‑DD), `include_undated` (bool, default false) |
 | `hybrid_search` | Intelligent search routing (auto-detects entity vs conceptual queries) | `query` (string), `limit` (1–50, default 10) |
 | `get_stats` | Knowledge base statistics | `detailed` (boolean) |
-| `generate_weekly_digest` | Generate weekly digest of Regen Network activity | `start_date` (YYYY-MM-DD, default: 7 days ago), `end_date` (YYYY-MM-DD, default: today), `save_to_file` (bool, default false), `output_path` (string), `format` ('markdown' or 'json', default: 'markdown') |
+| `generate_weekly_digest` | Generate weekly digest SUMMARY of Regen Network activity | `start_date` (YYYY-MM-DD, default: 7 days ago), `end_date` (YYYY-MM-DD, default: today), `save_to_file` (bool, default false), `output_path` (string), `format` ('markdown' or 'json', default: 'markdown') |
+| `get_notebooklm_export` | Get FULL NotebookLM export with complete forum posts, Notion pages, and source material | `save_to_file` (bool, default false), `output_path` (string) |
 
 ### Code Knowledge Graph
 | Tool | Description | Key Inputs |
@@ -134,6 +286,88 @@ This MCP server gives AI assistants access to Regen Network's comprehensive know
 | `get_repo_overview` | Get structured overview of a Regen repository | `repository` (enum: regen-ledger, regen-web, regen-data-standards, regenie-corpus) |
 | `get_tech_stack` | Get technical stack information for Regen repositories | `repository` (optional, omit to show all repos) |
 
+---
+
+## 🤔 What Can I Ask? User Guide
+
+This table helps you understand which tool to use for different tasks. Just ask Claude in natural language - it will automatically use the right tool.
+
+| **When You Want To...**                                  | **Ask Claude**                                           | **Tool Used**            |
+|----------------------------------------------------------|----------------------------------------------------------|--------------------------|
+| **Discover what's indexed**                              |                                                          |                          |
+| See what repositories are available                      | "What repositories are indexed?"                         | `list_repos`             |
+| See what types of code entities exist                    | "What entity types are available?"                       | `list_entity_types`      |
+| Get comprehensive statistics                             | "Show me statistics about the knowledge base"            | `get_entity_stats`       |
+| **Find specific code**                                   |                                                          |                          |
+| Find all entities of a type                              | "Show me all Keepers in regen-ledger"                    | `find_by_type`           |
+| Search for entities by name                              | "Find entities related to MsgCreateBatch"                | `search_entities`        |
+| Find related code entities                               | "What's related to the ecocredit Keeper?"                | `related_entities`       |
+| **Understand relationships**                             |                                                          |                          |
+| Find which Keeper handles a Message                      | "Which Keeper handles MsgCreateBatch?"                   | `keeper_for_msg`         |
+| Find what Messages a Keeper handles                      | "What messages does the ecocredit Keeper handle?"        | `msgs_for_keeper`        |
+| Find what documentation mentions an entity               | "What docs mention MsgRetire?"                           | `docs_mentioning`        |
+| Find entities in a file                                  | "What entities are in keeper.go?"                        | `entities_in_doc`        |
+| **Navigate by modules**                                  |                                                          |                          |
+| List all modules                                         | "What modules exist in regen-ledger?"                    | `list_modules`           |
+| Get details about a module                               | "Tell me about the ecocredit module"                     | `get_module`             |
+| Search for modules                                       | "Find modules related to baskets"                        | `search_modules`         |
+| Find entities in a module                                | "What entities are in the ecocredit module?"             | `module_entities`        |
+| **Search documentation**                                 |                                                          |                          |
+| Search GitHub docs                                       | "Search for documentation about credit classes"          | `search_github_docs`     |
+| Get repository overview                                  | "Give me an overview of regen-web"                       | `get_repo_overview`      |
+| Understand tech stack                                    | "What's the tech stack for regen-ledger?"                | `get_tech_stack`         |
+| **Search everything (hybrid)**                           |                                                          |                          |
+| Semantic search across code and docs                     | "How does credit retirement work?"                       | `hybrid_search`          |
+| Advanced filtering by date                               | "Find discussions about tokens from last week"           | `search_knowledge`       |
+| **Get activity summaries**                               |                                                          |                          |
+| Generate weekly digest summary                           | "Create a weekly digest of Regen activity"               | `generate_weekly_digest` |
+| Get full content for NotebookLM                          | "Get the full NotebookLM export with all forum posts"    | `get_notebooklm_export`  |
+
+### Query Type Reference
+
+The `query_code_graph` tool supports these query types:
+
+#### Discovery
+- **`list_repos`** - Show all indexed repositories with entity counts
+- **`list_entity_types`** - Show all entity types (Function, Class, Keeper, Message, etc.) with counts
+- **`get_entity_stats`** - Comprehensive statistics: entities by type, language, repository
+- **`list_modules`** - Show all modules across repositories
+
+#### Entity Queries
+- **`find_by_type`** - Find all entities of a specific type (requires `entity_type` parameter)
+- **`search_entities`** - Search entities by name pattern (requires `entity_name` parameter)
+- **`related_entities`** - Find entities related to a given entity (requires `entity_name` parameter)
+
+#### Relationship Queries
+- **`keeper_for_msg`** - Find which Keeper handles a Message (requires `entity_name` parameter)
+- **`msgs_for_keeper`** - Find what Messages a Keeper handles (requires `entity_name` parameter)
+- **`docs_mentioning`** - Find documentation mentioning an entity (requires `entity_name` parameter)
+- **`entities_in_doc`** - Find entities defined in a document (requires `doc_path` parameter)
+
+#### Module Queries
+- **`get_module`** - Get details about a specific module (requires `module_name` parameter)
+- **`search_modules`** - Search modules by keyword (requires `entity_name` parameter)
+- **`module_entities`** - Get entities in a module (requires `module_name` parameter)
+- **`module_for_entity`** - Find which module contains an entity (requires `entity_name` parameter)
+
+### What's Currently Indexed
+
+**Repositories:** 5 total
+- `regen-ledger`: 18,619 entities (Go, Cosmos SDK modules)
+- `regen-web`: 3,164 entities (TypeScript/React frontend)
+- `koi-sensors`: 1,250 entities (Python data collection)
+- `regen-koi-mcp`: 688 entities (TypeScript MCP server)
+- `regen-data-standards`: 6 entities (JSON schemas)
+
+**Entity Types:** 10 types
+- Function, Class, Interface, Method, Type
+- Keeper, Message, Query, Event (Cosmos SDK specific)
+- Module, Repository
+
+**Total:** 23,728 code entities
+
+---
+
 ## 🏗️ Architecture
 
 This repo contains everything you need to run a complete KOI MCP setup:
@@ -184,16 +418,35 @@ Or manually add to config (`~/Library/Application Support/Claude/claude_desktop_
 
 ### Claude Code CLI
 
-**One-line install:**
+**Option 1: Use the automated installer (recommended)**
 ```bash
-claude mcp add regen-koi npx -y regen-koi-mcp@latest
+curl -fsSL https://raw.githubusercontent.com/gaiaaiagent/regen-koi-mcp/main/install.sh | bash
 ```
+This configures both Claude Desktop and Claude Code CLI with the correct environment variables.
 
-Then set environment variable:
+**Option 2: Manual installation**
+1. Add the MCP server:
 ```bash
-export KOI_API_ENDPOINT=https://regen.gaiaai.xyz/api/koi
+claude mcp add regen-koi npx -y regen-koi-mcp@latest
 ```
 
+2. Configure the environment variable in your Claude Code settings file (`~/.claude/settings.json` or similar):
+```json
+{
+  "mcpServers": {
+    "regen-koi": {
+      "command": "npx",
+      "args": ["-y", "regen-koi-mcp@latest"],
+      "env": {
+        "KOI_API_ENDPOINT": "https://regen.gaiaai.xyz/api/koi"
+      }
+    }
+  }
+}
+```
+
+**Verification:** After installation, restart Claude Code and ask: "What repositories are indexed?" to verify the tools are working.
+
 ---
 
 ### VS Code / VS Code Insiders
@@ -545,6 +798,27 @@ The setup connects to our hosted KOI API at `https://regen.gaiaai.xyz/api/koi`.
 ### "Command not found"
 Make sure Node.js is installed and in your PATH. The setup script uses `node` command.
 
+### "Connection error: localhost:8301" or "localhost:8910"
+
+**Cause:** Claude caches MCP configurations in `~/.claude.json`. If you previously ran a local dev server, it may have cached `localhost` URLs that override your current config.
+
+**Fix:**
+```bash
+# 1. Check for cached localhost configs
+grep -i "localhost.*8301\|localhost.*8910" ~/.claude.json
+
+# 2. Backup your config
+cp ~/.claude.json ~/.claude.json.backup
+
+# 3. Replace localhost with public API
+sed -i '' 's|http://localhost:8301/api/koi|https://regen.gaiaai.xyz/api/koi|g' ~/.claude.json
+sed -i '' 's|http://localhost:8910/api/koi|https://regen.gaiaai.xyz/api/koi|g' ~/.claude.json
+
+# 4. Restart Claude Code/Desktop
+```
+
+**Prevention:** Always use `https://regen.gaiaai.xyz/api/koi` in MCP configs unless you're actively developing the API server locally.
+
 ## 📚 Example Usage
 
 Once configured, you can ask Claude:
@@ -556,30 +830,49 @@ Once configured, you can ask Claude:
 - "Generate a weekly digest of Regen Network activity from the past week"
 - "Create a digest of discussions from January 1 to January 7, 2025"
 
-### Weekly Digest Tool
+### Weekly Digest Tools
+
+There are two tools for getting weekly activity information, each designed for different use cases:
 
-The `generate_weekly_digest` tool creates comprehensive markdown summaries of Regen Network activity:
+#### `generate_weekly_digest` - Curated Summary
+
+Creates a condensed, curated markdown summary of Regen Network activity. Best for quick overview and reading in Claude.
 
 **Features:**
+- Executive summary with key highlights
+- Curated selection of important discussions and updates
+- Governance analysis and community insights
+- On-chain metrics summary
 - Automatically aggregates content from the past 7 days (or custom date range)
-- Returns markdown content that can be used directly in Claude Desktop as an artifact
-- Optionally saves to a file for use with NotebookLM or other tools
-- Includes proper source citations and statistics
 
 **Examples:**
-
-```javascript
-// In Claude Desktop or Claude Code CLI:
+```
 "Generate a weekly digest of Regen Network activity"
-
-// With custom date range:
 "Create a digest from December 1 to December 8, 2024"
-
-// Save to file:
 "Generate a weekly digest and save it to weekly_summary.md"
 ```
 
-**Note:** The digest content is returned in the response, so in Claude Desktop it will be displayed inline (and may be created as an artifact). The `save_to_file` option is useful when you want a persistent copy on disk.
+#### `get_notebooklm_export` - Full Content Export
+
+Returns the complete, unabridged source material. Best for deep analysis or loading into NotebookLM.
+
+**Features:**
+- **Full forum thread posts** - Complete text of all posts, not just summaries
+- **Complete Notion page content** - All chunks combined into full documents
+- **Enriched URLs** - Metadata and context for linked resources
+- **Detailed source material** - Everything you need for thorough analysis
+- Generates on demand if no recent cached version exists (takes 2-3 minutes)
+
+**Examples:**
+```
+"Get the full NotebookLM export"
+"Get the complete weekly content with all forum posts"
+"Export the full source material and save it to notebooklm_export.md"
+```
+
+**When to use which:**
+- Use `generate_weekly_digest` for a quick, readable summary
+- Use `get_notebooklm_export` when you need raw source content for deep analysis, fact-checking, or loading into external tools like NotebookLM
 
 ## 🤝 Contributing
 

package/dist/cache.d.ts

@@ -0,0 +1,70 @@
+/**
+ * Cache Module - LRU Caching for Query Results
+ *
+ * Provides in-memory caching with TTL for query results to reduce
+ * database load and improve response times.
+ */
+/**
+ * Query Cache class
+ */
+declare class QueryCache {
+    private caches;
+    constructor();
+    /**
+     * Get cache for a specific category
+     */
+    private getCache;
+    /**
+     * Get the category for a query type
+     */
+    private getCategory;
+    /**
+     * Get a cached result
+     */
+    get<T>(tool: string, queryType: string, params: Record<string, any>): T | undefined;
+    /**
+     * Set a cached result
+     */
+    set<T>(tool: string, queryType: string, params: Record<string, any>, value: T): void;
+    /**
+     * Check if a key exists in cache
+     */
+    has(tool: string, queryType: string, params: Record<string, any>): boolean;
+    /**
+     * Delete a specific cached result
+     */
+    delete(tool: string, queryType: string, params: Record<string, any>): void;
+    /**
+     * Clear all caches
+     */
+    clear(): void;
+    /**
+     * Clear a specific category's cache
+     */
+    clearCategory(category: string): void;
+    /**
+     * Get cache statistics
+     */
+    getStats(): {
+        totalSize: number;
+        byCategory: Record<string, {
+            size: number;
+            maxSize: number;
+        }>;
+    };
+}
+export declare const queryCache: QueryCache;
+/**
+ * Wrapper function to cache async query results
+ */
+export declare function cachedQuery<T>(tool: string, queryType: string, params: Record<string, any>, fetchFn: () => Promise<T>): Promise<T>;
+/**
+ * Check if a query type should be cached
+ */
+export declare function shouldCache(queryType: string): boolean;
+/**
+ * Get TTL for a query type
+ */
+export declare function getTTL(queryType: string): number;
+export default queryCache;
+//# sourceMappingURL=cache.d.ts.map

package/dist/cache.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"cache.d.ts","sourceRoot":"","sources":["../src/cache.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAgGH;;GAEG;AACH,cAAM,UAAU;IACd,OAAO,CAAC,MAAM,CAAiD;;IAc/D;;OAEG;IACH,OAAO,CAAC,QAAQ;IAShB;;OAEG;IACH,OAAO,CAAC,WAAW;IAInB;;OAEG;IACH,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,GAAG,CAAC,GAAG,SAAS;IAkBnF;;OAEG;IACH,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,EAAE,KAAK,EAAE,CAAC,GAAG,IAAI;IAUpF;;OAEG;IACH,GAAG,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,GAAG,OAAO;IAO1E;;OAEG;IACH,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,GAAG,IAAI;IAQ1E;;OAEG;IACH,KAAK,IAAI,IAAI;IAMb;;OAEG;IACH,aAAa,CAAC,QAAQ,EAAE,MAAM,GAAG,IAAI;IAOrC;;OAEG;IACH,QAAQ,IAAI;QACV,SAAS,EAAE,MAAM,CAAC;QAClB,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE;YAAE,IAAI,EAAE,MAAM,CAAC;YAAC,OAAO,EAAE,MAAM,CAAA;SAAE,CAAC,CAAC;KAC/D;CAeF;AAGD,eAAO,MAAM,UAAU,YAAmB,CAAC;AAE3C;;GAEG;AACH,wBAAsB,WAAW,CAAC,CAAC,EACjC,IAAI,EAAE,MAAM,EACZ,SAAS,EAAE,MAAM,EACjB,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,EAC3B,OAAO,EAAE,MAAM,OAAO,CAAC,CAAC,CAAC,GACxB,OAAO,CAAC,CAAC,CAAC,CAcZ;AAED;;GAEG;AACH,wBAAgB,WAAW,CAAC,SAAS,EAAE,MAAM,GAAG,OAAO,CAGtD;AAED;;GAEG;AACH,wBAAgB,MAAM,CAAC,SAAS,EAAE,MAAM,GAAG,MAAM,CAGhD;AAED,eAAe,UAAU,CAAC"}