midnight-mcp 0.0.8 → 0.1.0

package/README.md

@@ -91,6 +91,8 @@ By default, the MCP uses a **hosted API** for semantic search:
 - ✅ **Semantic search** works immediately
 - ✅ **No API keys** needed
 
+> **📊 Quality Metrics**: To ensure the MCP stays accurate as Midnight's codebase evolves rapidly, we collect anonymous usage metrics (query counts, relevance scores) to monitor search quality. No query content or personal data is stored. This helps us identify when re-indexing is needed and improve results over time.
+
 ### Local Mode (Optional)
 
 Run everything locally for privacy or offline use:
@@ -121,26 +123,42 @@ Add `"GITHUB_TOKEN": "ghp_..."` for higher GitHub API rate limits (60 → 5000 r
 
 ## Features
 
-### Tools (16)
-
-| Tool                              | Description                             |
-| --------------------------------- | --------------------------------------- |
-| `midnight-search-compact`         | Search Compact contract code            |
-| `midnight-search-typescript`      | Search TypeScript SDK                   |
-| `midnight-search-docs`            | Search documentation                    |
-| `midnight-analyze-contract`       | Analyze contract structure and security |
-| `midnight-explain-circuit`        | Explain circuits in plain language      |
-| `midnight-get-file`               | Get files from Midnight repos           |
-| `midnight-list-examples`          | List example contracts                  |
-| `midnight-get-latest-updates`     | Recent repo changes                     |
-| `midnight-get-version-info`       | Get version and release info            |
-| `midnight-check-breaking-changes` | Check for breaking changes              |
-| `midnight-get-migration-guide`    | Migration guides between versions       |
-| `midnight-get-file-at-version`    | Get file at specific version            |
-| `midnight-compare-syntax`         | Compare files between versions          |
-| `midnight-get-latest-syntax`      | Latest syntax reference                 |
-| `midnight-health-check`           | Check server health status              |
-| `midnight-get-status`             | Get rate limits and cache stats         |
+### Tools (19)
+
+| Tool                              | Description                                 |
+| --------------------------------- | ------------------------------------------- |
+| `midnight-search-compact`         | Search Compact contract code                |
+| `midnight-search-typescript`      | Search TypeScript SDK                       |
+| `midnight-search-docs`            | Search documentation                        |
+| `midnight-analyze-contract`       | Analyze contract structure and security     |
+| `midnight-explain-circuit`        | Explain circuits in plain language          |
+| `midnight-get-file`               | Get files from Midnight repos               |
+| `midnight-list-examples`          | List example contracts                      |
+| `midnight-get-latest-updates`     | Recent repo changes                         |
+| `midnight-get-version-info`       | Get version and release info                |
+| `midnight-check-breaking-changes` | Check for breaking changes                  |
+| `midnight-get-migration-guide`    | Migration guides between versions           |
+| `midnight-get-file-at-version`    | Get file at specific version                |
+| `midnight-compare-syntax`         | Compare files between versions              |
+| `midnight-get-latest-syntax`      | Latest syntax reference                     |
+| `midnight-health-check`           | Check server health status                  |
+| `midnight-get-status`             | Get rate limits and cache stats             |
+| `midnight-generate-contract` ✨   | AI-generate contracts from natural language |
+| `midnight-review-contract` ✨     | AI-powered security review of contracts     |
+| `midnight-document-contract` ✨   | AI-generate documentation for contracts     |
+
+> **✨ New AI-Powered Tools** require a client with [MCP Sampling](https://spec.modelcontextprotocol.io/specification/client/sampling/) support (e.g., Claude Desktop). Without sampling, these tools will return a helpful message instead.
+
+### Resource Templates (4)
+
+Dynamic access to any resource using URI templates:
+
+| Template                                | Description                    |
+| --------------------------------------- | ------------------------------ |
+| `midnight://code/{owner}/{repo}/{path}` | Any code file from any repo    |
+| `midnight://docs/{section}/{topic}`     | Documentation by section/topic |
+| `midnight://examples/{category}/{name}` | Example contracts by category  |
+| `midnight://schema/{type}`              | JSON schemas (AST, tx, proofs) |
 
 ### Resources (20)
 
@@ -188,6 +206,18 @@ MIDNIGHT_API_URL=http://localhost:8787 npm start
 
 The hosted API runs on Cloudflare Workers + Vectorize. See [api/README.md](./api/README.md) for deployment and development instructions.
 
+### Search Quality
+
+The API indexes **25 Midnight repositories** including core infrastructure, SDK, examples, ZK libraries, and developer tools.
+
+Search quality techniques:
+
+- **Optimized chunking** — 1000-char chunks with 200-char overlap for precise, contextual results
+- **Hybrid search** — Combines vector similarity with keyword boosting (up to 20% boost for exact matches)
+- **Incremental indexing** — Daily updates via tarball download + batch embeddings (~5 min)
+
+View live metrics at the [Dashboard](https://midnight-mcp-api.midnightmcp.workers.dev/dashboard).
+
 ## License
 
 MIT

package/dist/server.d.ts

@@ -3,6 +3,15 @@ import { Server } from "@modelcontextprotocol/sdk/server/index.js";
  * Create and configure the MCP server
  */
 export declare function createServer(): Server;
+/**
+ * Notify subscribers when a resource changes
+ * Call this when re-indexing or when docs are updated
+ */
+export declare function notifyResourceUpdate(server: Server, uri: string): void;
+/**
+ * Get the list of active subscriptions
+ */
+export declare function getActiveSubscriptions(): string[];
 /**
  * Initialize the server and vector store
  */

package/dist/server.js

@@ -1,24 +1,60 @@
 import { Server } from "@modelcontextprotocol/sdk/server/index.js";
 import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
-import { CallToolRequestSchema, ListToolsRequestSchema, ListResourcesRequestSchema, ReadResourceRequestSchema, ListPromptsRequestSchema, GetPromptRequestSchema, } from "@modelcontextprotocol/sdk/types.js";
+import { CallToolRequestSchema, ListToolsRequestSchema, ListResourcesRequestSchema, ReadResourceRequestSchema, ListPromptsRequestSchema, GetPromptRequestSchema, ListResourceTemplatesRequestSchema, SubscribeRequestSchema, UnsubscribeRequestSchema, } from "@modelcontextprotocol/sdk/types.js";
 import { logger, formatErrorResponse } from "./utils/index.js";
 import { vectorStore } from "./db/index.js";
 import { allTools } from "./tools/index.js";
 import { allResources, getDocumentation, getCode, getSchema, } from "./resources/index.js";
 import { promptDefinitions, generatePrompt } from "./prompts/index.js";
+import { registerSamplingCallback } from "./services/index.js";
 // Server information
 const SERVER_INFO = {
     name: "midnight-mcp",
     version: "1.0.0",
     description: "MCP Server for Midnight Blockchain Development",
 };
+// Resource subscriptions tracking
+const resourceSubscriptions = new Set();
+// Resource templates for parameterized resources (RFC 6570 URI Templates)
+const resourceTemplates = [
+    {
+        uriTemplate: "midnight://code/{owner}/{repo}/{path}",
+        name: "Repository Code",
+        title: "📄 Repository Code Files",
+        description: "Access code files from any Midnight repository by specifying owner, repo, and file path",
+        mimeType: "text/plain",
+    },
+    {
+        uriTemplate: "midnight://docs/{section}/{topic}",
+        name: "Documentation",
+        title: "📚 Documentation Sections",
+        description: "Access documentation by section (guides, api, concepts) and topic",
+        mimeType: "text/markdown",
+    },
+    {
+        uriTemplate: "midnight://examples/{category}/{name}",
+        name: "Example Contracts",
+        title: "📝 Example Contracts",
+        description: "Access example contracts by category (counter, bboard, token, voting) and name",
+        mimeType: "text/x-compact",
+    },
+    {
+        uriTemplate: "midnight://schema/{type}",
+        name: "Schema Definitions",
+        title: "🔧 Schema Definitions",
+        description: "Access JSON schemas for contract AST, transactions, and proofs",
+        mimeType: "application/json",
+    },
+];
 /**
  * Create and configure the MCP server
  */
 export function createServer() {
     const server = new Server(SERVER_INFO, {
         capabilities: {
-            tools: {},
+            tools: {
+                listChanged: true,
+            },
             resources: {
                 subscribe: true,
                 listChanged: true,
@@ -34,13 +70,17 @@ export function createServer() {
     registerResourceHandlers(server);
     // Register prompt handlers
     registerPromptHandlers(server);
+    // Register subscription handlers
+    registerSubscriptionHandlers(server);
+    // Setup sampling callback if available
+    setupSampling(server);
     return server;
 }
 /**
  * Register tool handlers
  */
 function registerToolHandlers(server) {
-    // List available tools
+    // List available tools with annotations and output schemas
     server.setRequestHandler(ListToolsRequestSchema, async () => {
         logger.debug("Listing tools");
         return {
@@ -48,6 +88,10 @@ function registerToolHandlers(server) {
                 name: tool.name,
                 description: tool.description,
                 inputSchema: tool.inputSchema,
+                // Include output schema if defined
+                ...(tool.outputSchema && { outputSchema: tool.outputSchema }),
+                // Include annotations if defined
+                ...(tool.annotations && { annotations: tool.annotations }),
             })),
         };
     });
@@ -117,6 +161,19 @@ function registerResourceHandlers(server) {
             })),
         };
     });
+    // List resource templates (RFC 6570 URI Templates)
+    server.setRequestHandler(ListResourceTemplatesRequestSchema, async () => {
+        logger.debug("Listing resource templates");
+        return {
+            resourceTemplates: resourceTemplates.map((template) => ({
+                uriTemplate: template.uriTemplate,
+                name: template.name,
+                title: template.title,
+                description: template.description,
+                mimeType: template.mimeType,
+            })),
+        };
+    });
     // Read resource content
     server.setRequestHandler(ReadResourceRequestSchema, async (request) => {
         const { uri } = request.params;
@@ -221,6 +278,97 @@ function registerPromptHandlers(server) {
         };
     });
 }
+/**
+ * Register resource subscription handlers
+ */
+function registerSubscriptionHandlers(server) {
+    // Handle subscribe requests
+    server.setRequestHandler(SubscribeRequestSchema, async (request) => {
+        const { uri } = request.params;
+        logger.info(`Subscribing to resource: ${uri}`);
+        // Validate that the URI is a valid resource
+        const validPrefixes = [
+            "midnight://docs/",
+            "midnight://code/",
+            "midnight://schema/",
+        ];
+        const isValid = validPrefixes.some((prefix) => uri.startsWith(prefix));
+        if (!isValid) {
+            logger.warn(`Invalid subscription URI: ${uri}`);
+            return {};
+        }
+        resourceSubscriptions.add(uri);
+        logger.debug(`Active subscriptions: ${resourceSubscriptions.size}`);
+        return {};
+    });
+    // Handle unsubscribe requests
+    server.setRequestHandler(UnsubscribeRequestSchema, async (request) => {
+        const { uri } = request.params;
+        logger.info(`Unsubscribing from resource: ${uri}`);
+        resourceSubscriptions.delete(uri);
+        logger.debug(`Active subscriptions: ${resourceSubscriptions.size}`);
+        return {};
+    });
+}
+/**
+ * Notify subscribers when a resource changes
+ * Call this when re-indexing or when docs are updated
+ */
+export function notifyResourceUpdate(server, uri) {
+    if (resourceSubscriptions.has(uri)) {
+        logger.info(`Notifying subscribers of update: ${uri}`);
+        // Send notification via the server
+        server.notification({
+            method: "notifications/resources/updated",
+            params: { uri },
+        });
+    }
+}
+/**
+ * Get the list of active subscriptions
+ */
+export function getActiveSubscriptions() {
+    return Array.from(resourceSubscriptions);
+}
+/**
+ * Setup sampling capability
+ * Registers a callback that allows the server to request LLM completions
+ */
+function setupSampling(server) {
+    // Create a sampling callback that uses the server's request method
+    const samplingCallback = async (request) => {
+        logger.debug("Requesting sampling from client", {
+            messageCount: request.messages.length,
+            maxTokens: request.maxTokens,
+        });
+        try {
+            // Request completion from the client
+            const response = await server.request({
+                method: "sampling/createMessage",
+                params: {
+                    messages: request.messages,
+                    systemPrompt: request.systemPrompt,
+                    maxTokens: request.maxTokens || 2048,
+                    temperature: request.temperature,
+                    modelPreferences: request.modelPreferences,
+                },
+            }, 
+            // Use a schema that matches the expected response
+            {
+                parse: (data) => data,
+                _def: { typeName: "SamplingResponse" },
+            });
+            return response;
+        }
+        catch (error) {
+            logger.error("Sampling request failed", { error: String(error) });
+            throw error;
+        }
+    };
+    // Register the callback
+    registerSamplingCallback(samplingCallback);
+    logger.info("Sampling capability configured");
+}
 /**
  * Initialize the server and vector store
  */

package/dist/services/index.d.ts

@@ -0,0 +1,6 @@
+/**
+ * Services index
+ * Export all service modules
+ */
+export { isSamplingAvailable, registerSamplingCallback, requestCompletion, generateContract, reviewContract, generateDocumentation, } from "./sampling.js";
+//# sourceMappingURL=index.d.ts.map

package/dist/services/index.js

@@ -0,0 +1,6 @@
+/**
+ * Services index
+ * Export all service modules
+ */
+export { isSamplingAvailable, registerSamplingCallback, requestCompletion, generateContract, reviewContract, generateDocumentation, } from "./sampling.js";
+//# sourceMappingURL=index.js.map

package/dist/services/sampling.d.ts

@@ -0,0 +1,62 @@
+/**
+ * Sampling service for agentic workflows
+ *
+ * Enables the MCP server to request LLM completions from the client,
+ * allowing for sophisticated multi-step workflows like:
+ * - Auto-generating contracts from examples
+ * - Code review with suggested fixes
+ * - Documentation generation
+ */
+import type { SamplingRequest, SamplingResponse, ModelPreferences } from "../types/index.js";
+type SamplingCallback = (request: SamplingRequest) => Promise<SamplingResponse>;
+/**
+ * Check if sampling is available
+ */
+export declare function isSamplingAvailable(): boolean;
+/**
+ * Register the sampling callback from the client
+ * This is called during server initialization when client supports sampling
+ */
+export declare function registerSamplingCallback(callback: SamplingCallback): void;
+/**
+ * Request a completion from the LLM via the client
+ */
+export declare function requestCompletion(messages: {
+    role: "user" | "assistant";
+    content: string;
+}[], options?: {
+    systemPrompt?: string;
+    maxTokens?: number;
+    temperature?: number;
+    modelPreferences?: ModelPreferences;
+}): Promise<string>;
+/**
+ * Generate a Compact contract based on requirements
+ */
+export declare function generateContract(requirements: string, options?: {
+    baseExample?: string;
+    contractType?: "counter" | "token" | "voting" | "custom";
+}): Promise<{
+    code: string;
+    explanation: string;
+    warnings: string[];
+}>;
+/**
+ * Review contract code and suggest improvements
+ */
+export declare function reviewContract(code: string): Promise<{
+    summary: string;
+    issues: Array<{
+        severity: "error" | "warning" | "info";
+        line?: number;
+        message: string;
+        suggestion?: string;
+    }>;
+    improvedCode?: string;
+}>;
+/**
+ * Generate documentation for a contract
+ */
+export declare function generateDocumentation(code: string, format?: "markdown" | "jsdoc"): Promise<string>;
+export {};
+//# sourceMappingURL=sampling.d.ts.map

package/dist/services/sampling.js

@@ -0,0 +1,256 @@
+/**
+ * Sampling service for agentic workflows
+ *
+ * Enables the MCP server to request LLM completions from the client,
+ * allowing for sophisticated multi-step workflows like:
+ * - Auto-generating contracts from examples
+ * - Code review with suggested fixes
+ * - Documentation generation
+ */
+import { logger } from "../utils/index.js";
+// Store for the sampling callback
+let samplingCallback = null;
+/**
+ * Check if sampling is available
+ */
+export function isSamplingAvailable() {
+    return samplingCallback !== null;
+}
+/**
+ * Register the sampling callback from the client
+ * This is called during server initialization when client supports sampling
+ */
+export function registerSamplingCallback(callback) {
+    samplingCallback = callback;
+    logger.info("Sampling capability registered");
+}
+/**
+ * Request a completion from the LLM via the client
+ */
+export async function requestCompletion(messages, options = {}) {
+    if (!samplingCallback) {
+        throw new Error("Sampling not available - client does not support this capability");
+    }
+    const request = {
+        messages: messages.map((m) => ({
+            role: m.role,
+            content: { type: "text", text: m.content },
+        })),
+        systemPrompt: options.systemPrompt,
+        maxTokens: options.maxTokens ?? 2048,
+        temperature: options.temperature ?? 0.7,
+        modelPreferences: options.modelPreferences ?? {
+            hints: [{ name: "claude-3-sonnet" }, { name: "gpt-4" }],
+            intelligencePriority: 0.8,
+            speedPriority: 0.5,
+        },
+    };
+    logger.debug("Requesting LLM completion", {
+        messageCount: messages.length,
+        maxTokens: request.maxTokens,
+    });
+    const response = await samplingCallback(request);
+    if (response.content.type !== "text") {
+        throw new Error("Unexpected response content type");
+    }
+    return response.content.text;
+}
+/**
+ * Generate a Compact contract based on requirements
+ */
+export async function generateContract(requirements, options = {}) {
+    if (!isSamplingAvailable()) {
+        return {
+            code: "",
+            explanation: "Sampling not available - this feature requires a client that supports the sampling capability (like Claude Desktop)",
+            warnings: ["Feature unavailable without sampling support"],
+        };
+    }
+    const systemPrompt = `You are an expert Compact smart contract developer for the Midnight blockchain.
+Your task is to generate secure, well-documented Compact contracts based on user requirements.
+
+Key Compact concepts:
+- \`ledger { }\` - Defines on-chain state (public and private)
+- \`@private\` - Marks state as private/shielded
+- \`export circuit\` - Public functions that generate ZK proofs
+- \`witness\` - Off-chain computation functions
+- \`assert()\` - Creates ZK constraints
+- \`Counter\`, \`Map<K,V>\`, \`Set<T>\` - Built-in collection types
+- \`Field\`, \`Boolean\`, \`Uint<N>\`, \`Bytes<N>\` - Primitive types
+
+Always include:
+1. Proper imports (include "std")
+2. Clear ledger state definitions
+3. Access control where appropriate
+4. Comprehensive inline comments
+
+Return ONLY the Compact code, no explanations.`;
+    const userPrompt = options.baseExample
+        ? `Based on this example contract:
+\`\`\`compact
+${options.baseExample}
+\`\`\`
+
+Generate a new contract with these requirements:
+${requirements}`
+        : `Generate a Compact smart contract with these requirements:
+${requirements}
+
+Contract type: ${options.contractType || "custom"}`;
+    try {
+        const code = await requestCompletion([{ role: "user", content: userPrompt }], {
+            systemPrompt,
+            maxTokens: 4096,
+            temperature: 0.3, // Lower temperature for code generation
+            modelPreferences: {
+                hints: [{ name: "claude-3-sonnet" }],
+                intelligencePriority: 0.9,
+                speedPriority: 0.3,
+            },
+        });
+        // Extract code from markdown if wrapped
+        const extractedCode = code.includes("```")
+            ? code
+                .replace(/```compact?\n?/g, "")
+                .replace(/```/g, "")
+                .trim()
+            : code.trim();
+        // Generate explanation
+        const explanation = await requestCompletion([
+            {
+                role: "user",
+                content: `Briefly explain what this Compact contract does (2-3 sentences):
+\`\`\`compact
+${extractedCode}
+\`\`\``,
+            },
+        ], {
+            systemPrompt: "You are a Compact contract documentation expert. Be concise.",
+            maxTokens: 256,
+            temperature: 0.5,
+        });
+        return {
+            code: extractedCode,
+            explanation: explanation.trim(),
+            warnings: [],
+        };
+    }
+    catch (error) {
+        logger.error("Contract generation failed", { error: String(error) });
+        return {
+            code: "",
+            explanation: `Contract generation failed: ${String(error)}`,
+            warnings: ["Generation failed - check logs for details"],
+        };
+    }
+}
+/**
+ * Review contract code and suggest improvements
+ */
+export async function reviewContract(code) {
+    if (!isSamplingAvailable()) {
+        return {
+            summary: "Code review requires sampling capability",
+            issues: [
+                {
+                    severity: "info",
+                    message: "This feature requires a client that supports the sampling capability",
+                },
+            ],
+        };
+    }
+    const systemPrompt = `You are a Compact smart contract security auditor.
+Review the provided contract for:
+1. Security vulnerabilities
+2. Privacy concerns (improper handling of shielded state)
+3. Logic errors
+4. Best practice violations
+5. Gas/performance issues
+
+Respond in JSON format:
+{
+  "summary": "Brief summary of the contract and overall quality",
+  "issues": [
+    {
+      "severity": "error|warning|info",
+      "line": optional_line_number,
+      "message": "Description of the issue",
+      "suggestion": "How to fix it"
+    }
+  ],
+  "improvedCode": "Full improved contract code if changes are needed"
+}`;
+    try {
+        const response = await requestCompletion([
+            {
+                role: "user",
+                content: `Review this Compact contract:\n\`\`\`compact\n${code}\n\`\`\``,
+            },
+        ], {
+            systemPrompt,
+            maxTokens: 4096,
+            temperature: 0.2,
+        });
+        // Parse JSON response
+        const jsonMatch = response.match(/\{[\s\S]*\}/);
+        if (jsonMatch) {
+            return JSON.parse(jsonMatch[0]);
+        }
+        return {
+            summary: response,
+            issues: [],
+        };
+    }
+    catch (error) {
+        logger.error("Contract review failed", { error: String(error) });
+        return {
+            summary: `Review failed: ${String(error)}`,
+            issues: [
+                {
+                    severity: "error",
+                    message: "Review failed - check logs for details",
+                },
+            ],
+        };
+    }
+}
+/**
+ * Generate documentation for a contract
+ */
+export async function generateDocumentation(code, format = "markdown") {
+    if (!isSamplingAvailable()) {
+        return "Documentation generation requires sampling capability";
+    }
+    const systemPrompt = format === "markdown"
+        ? `Generate comprehensive Markdown documentation for this Compact contract.
+Include:
+- Overview and purpose
+- State variables (with privacy annotations)
+- Circuit functions with parameters and effects
+- Witness functions
+- Usage examples
+- Security considerations`
+        : `Generate JSDoc-style documentation comments for this Compact contract.
+Add documentation comments above each:
+- Ledger field
+- Circuit function
+- Witness function
+- Type definition`;
+    try {
+        return await requestCompletion([
+            {
+                role: "user",
+                content: `Generate ${format} documentation for:\n\`\`\`compact\n${code}\n\`\`\``,
+            },
+        ], {
+            systemPrompt,
+            maxTokens: 4096,
+            temperature: 0.5,
+        });
+    }
+    catch (error) {
+        logger.error("Documentation generation failed", { error: String(error) });
+        return `Documentation generation failed: ${String(error)}`;
+    }
+}
+//# sourceMappingURL=sampling.js.map