subgraph-registry-mcp 0.7.2 → 0.8.0

package/README.md

@@ -8,6 +8,11 @@ Agent-friendly semantic classification of all subgraphs on [The Graph Network](h
 
 Pre-computed index of **14,700+ subgraphs** with domain classification, protocol type detection, schema fingerprinting, canonical entity mapping, and composite reliability scoring.
 
+> **What's new in 0.8.0** — three agent-discovery upgrades:
+> - **[Semantic search](#semantic-search)** via 384-dim embeddings (`semantic_search_subgraphs`)
+> - **[Schema evolution tracking](#schema-evolution)** with stability days surfaced on every result (`get_schema_changes`)
+> - **[OpenAPI 3.1 spec](#openapi)** auto-generated for MCP tools + REST routes, served at `/.well-known/openapi.json`
+
 ## The Problem
 
 Agents querying The Graph need to discover and select the right subgraph before they can query data. Today this requires 3-4 tool calls (search, check volumes, fetch schema, infer structure) before any real work happens. This registry flips that: agents start with structured knowledge, not a blank slate.
@@ -162,11 +167,13 @@ The registry is available as an MCP server with **dual transport** — stdio for
 
 > The shipped server is the Node implementation in [`src/index.js`](src/index.js); that's what `npx subgraph-registry-mcp` runs and what's published to npm. A Python equivalent in [`python/mcp_server.py`](python/mcp_server.py) is kept for local development against the same SQLite database — bug fixes and new tools should land in the Node version first.
 
-**4 tools:**
+**6 tools:**
 - **search_subgraphs** — filter by domain, network, protocol type, entity, or keyword
-- **recommend_subgraph** — natural language goal to best subgraphs
-- **get_subgraph_detail** — full classification for a specific subgraph
+- **recommend_subgraph** — natural language goal to best subgraphs (includes `schema_stable_days`)
+- **get_subgraph_detail** — full classification for a specific subgraph (includes `schema_changed_at`)
 - **list_registry_stats** — registry overview (domains, networks, counts)
+- **semantic_search_subgraphs** — vector-similarity search over precomputed embeddings (sentence-transformers/all-MiniLM-L6-v2, 384-dim). Use for fuzzy/paraphrased goals where literal keyword match would miss.
+- **get_schema_changes** — chronological schema-fingerprint history for a subgraph (one row per detected change). Helps agents prefer mature subgraphs whose data contract has been stable.
 
 ### Install
 
@@ -215,6 +222,64 @@ curl http://localhost:3848/.well-known/subgraph/5zvR82QoaXYFyDEKLZ9t6v9adgnptxYp
 
 ---
 
+## Semantic Search
+
+Every subgraph has a precomputed 384-dim embedding from `sentence-transformers/all-MiniLM-L6-v2`, built from its display name, description, canonical entities, top schema entity names, and protocol metadata. At MCP-tool-call time the Node server embeds the query string with the same model (via [@xenova/transformers](https://github.com/xenova/transformers.js), quantized ONNX bundled in the npm package — no first-call download) and ranks rows by cosine similarity.
+
+```js
+const { subgraphs } = await mcp.call("semantic_search_subgraphs", {
+  query: "lending positions near liquidation on a Layer 2",
+  limit: 5,
+});
+// subgraphs[i].semantic_score is cosine similarity in [0, 1]; >0.5 ~= strong match.
+```
+
+Use it when:
+- The goal is paraphrased or use-case-shaped (`search_subgraphs` is keyword-only).
+- You're exploring "what data exists for X?" rather than fetching a specific protocol's subgraph.
+
+Same model is shared between Python crawl-time (`fastembed`) and JS runtime (`@xenova/transformers`) — vectors are bitwise-comparable so cosine math gives consistent rankings across runtimes.
+
+Embeddings add ~22 MB to `registry.db` (14k × 384 × 4 bytes); model bundle adds ~23 MB to the npm package.
+
+---
+
+## Schema Evolution
+
+Each crawl computes a `schema_fingerprint` (MD5 of sorted `entity:field_count` pairs) per subgraph. Whenever the fingerprint changes from the previous sync, an immutable row is written to `schema_history`. The table is append-only and survives full DB rebuilds.
+
+```js
+const history = await mcp.call("get_schema_changes", {
+  subgraph_id: "5zvR82QoaXYFyDEKLZ9t6v9adgnptxYpKpSbxtgVENFV",
+});
+// {
+//   total_changes: 3,
+//   stable_days: 47.2,
+//   changed_within_24h: false,
+//   changed_within_7d: false,
+//   changes: [
+//     { fingerprint: "abc123...", prev_fingerprint: "def456...", detected_at: 1717... },
+//     ...
+//   ]
+// }
+```
+
+`recommend_subgraph` and `get_subgraph_detail` results now also include `schema_changed_at` (unix seconds of last detected change) and `schema_stable_days` so agents can prefer subgraphs whose data contract has been stable longer — useful when a query needs to keep working across the agent's planning horizon.
+
+---
+
+## OpenAPI
+
+The full API surface (MCP tools + REST routes) is published as OpenAPI 3.1:
+
+- `openapi.yaml` — checked into the repo, single source of truth
+- `data/openapi.json` — bundled with the npm tarball
+- `GET /.well-known/openapi.json` — served by the HTTP transport for live discovery
+
+The spec is regenerated on every release from the declarative `TOOLS[]` + `REST_ROUTES[]` exports in [`src/index.js`](src/index.js) via [`scripts/gen-openapi.js`](scripts/gen-openapi.js). CI fails any PR that touches `src/index.js` without regenerating the spec.
+
+---
+
 ## REST API
 
 ```

package/data/openapi.json

@@ -0,0 +1,600 @@
+{
+  "openapi": "3.1.0",
+  "info": {
+    "title": "Subgraph Registry",
+    "description": "Agent-friendly subgraph discovery on The Graph Network. 14,700+ classified subgraphs with semantic search, reliability scoring, schema-evolution tracking, and x402 query URLs ($0.01 USDC on Base, no API key).",
+    "version": "0.7.2",
+    "license": {
+      "name": "MIT"
+    },
+    "contact": {
+      "name": "PaulieB14",
+      "url": "https://github.com/PaulieB14/subgraph-registry"
+    }
+  },
+  "servers": [
+    {
+      "url": "http://localhost:3848",
+      "description": "Local MCP HTTP transport (default port)"
+    }
+  ],
+  "paths": {
+    "/mcp/tools/search_subgraphs": {
+      "post": {
+        "operationId": "tool_search_subgraphs",
+        "summary": "Search and filter the classified subgraph registry (15,500+ subgraphs).",
+        "description": "Search and filter the classified subgraph registry (15,500+ subgraphs). Filter by domain (defi, nfts, dao, gaming, identity, infrastructure, social, analytics), network (mainnet, arbitrum-one, base, matic, bsc, optimism, avalanche), protocol_type (dex, lending, bridge, staking, options, perpetuals, nft-marketplace, yield-aggregator, governance, name-service), canonical entity type (liquidity_pool, trade, token, position, vault, loan, collateral, liquidation, nft_collection, nft_item, nft_sale, proposal, delegate, domain_name, account, transaction, daily_snapshot, hourly_snapshot), or free-text keyword. Returns subgraphs ranked by reliability score. Each result includes query_url_x402 (POST GraphQL and pay $0.01 USDC on Base per query — no API key needed) and a legacy query_url (Studio API key required).",
+        "tags": [
+          "mcp-tools"
+        ],
+        "requestBody": {
+          "required": false,
+          "content": {
+            "application/json": {
+              "schema": {
+                "type": "object",
+                "additionalProperties": false,
+                "properties": {
+                  "query": {
+                    "type": "string",
+                    "description": "Free-text search across names and descriptions"
+                  },
+                  "domain": {
+                    "type": "string",
+                    "description": "Filter by domain: defi, nfts, dao, gaming, identity, infrastructure, social, analytics"
+                  },
+                  "network": {
+                    "type": "string",
+                    "description": "Filter by chain: mainnet, arbitrum-one, base, matic, bsc, optimism, avalanche, etc."
+                  },
+                  "protocol_type": {
+                    "type": "string",
+                    "description": "Filter by protocol type: dex, lending, bridge, staking, options, perpetuals, etc."
+                  },
+                  "entity": {
+                    "type": "string",
+                    "description": "Filter by canonical entity: liquidity_pool, trade, token, position, vault, loan, etc."
+                  },
+                  "min_reliability": {
+                    "type": "number",
+                    "description": "Minimum reliability score (0-1). Higher = more query fees, volume, curation signal, and indexer allocation."
+                  },
+                  "limit": {
+                    "type": "integer",
+                    "description": "Max results to return (default: 20)",
+                    "default": 20
+                  }
+                }
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "Tool result (JSON-encoded)",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object"
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Invalid input",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/mcp/tools/recommend_subgraph": {
+      "post": {
+        "operationId": "tool_recommend_subgraph",
+        "summary": "Given a natural-language goal like 'find DEX trades on Arbitrum' or 'get lending liquidation data', returns the best…",
+        "description": "Given a natural-language goal like 'find DEX trades on Arbitrum' or 'get lending liquidation data', returns the best matching subgraphs with reliability scores. Automatically infers domain and protocol type from the goal. Each result includes query_url_x402 (preferred — POST GraphQL, pay $0.01 USDC on Base per query, no API key) and a legacy query_url for Studio-key flows.",
+        "tags": [
+          "mcp-tools"
+        ],
+        "requestBody": {
+          "required": true,
+          "content": {
+            "application/json": {
+              "schema": {
+                "type": "object",
+                "additionalProperties": false,
+                "properties": {
+                  "goal": {
+                    "type": "string",
+                    "description": "What you want to do, e.g. 'query Uniswap pool data on Base'"
+                  },
+                  "chain": {
+                    "type": "string",
+                    "description": "Optional chain filter: mainnet, arbitrum-one, base, matic, etc."
+                  }
+                },
+                "required": [
+                  "goal"
+                ]
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "Tool result (JSON-encoded)",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object"
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Invalid input",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/mcp/tools/get_subgraph_detail": {
+      "post": {
+        "operationId": "tool_get_subgraph_detail",
+        "summary": "Get full classification detail for a specific subgraph by its subgraph ID or IPFS hash.",
+        "description": "Get full classification detail for a specific subgraph by its subgraph ID or IPFS hash. Returns domain, protocol type, canonical entities, all entity names with field counts, reliability score, signal data, both query URLs (x402 and legacy), the x402 pricing manifest ($0.01 USDC on Base), and step-by-step instructions for both query paths.",
+        "tags": [
+          "mcp-tools"
+        ],
+        "requestBody": {
+          "required": true,
+          "content": {
+            "application/json": {
+              "schema": {
+                "type": "object",
+                "additionalProperties": false,
+                "properties": {
+                  "subgraph_id": {
+                    "type": "string",
+                    "description": "Subgraph ID or IPFS hash (Qm...)"
+                  }
+                },
+                "required": [
+                  "subgraph_id"
+                ]
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "Tool result (JSON-encoded)",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object"
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Invalid input",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/mcp/tools/list_registry_stats": {
+      "post": {
+        "operationId": "tool_list_registry_stats",
+        "summary": "Get an overview of the subgraph registry: total count, available domains, networks, and protocol types with counts.",
+        "description": "Get an overview of the subgraph registry: total count, available domains, networks, and protocol types with counts. Use this to understand what data is available before searching.",
+        "tags": [
+          "mcp-tools"
+        ],
+        "requestBody": {
+          "required": false,
+          "content": {
+            "application/json": {
+              "schema": {
+                "type": "object",
+                "additionalProperties": false,
+                "properties": {}
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "Tool result (JSON-encoded)",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object"
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Invalid input",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/mcp/tools/semantic_search_subgraphs": {
+      "post": {
+        "operationId": "tool_semantic_search_subgraphs",
+        "summary": "Semantic vector search over the subgraph registry.",
+        "description": "Semantic vector search over the subgraph registry. Embeds the query string with sentence-transformers/all-MiniLM-L6-v2 (the same model architecture used at crawl time; the runtime uses an INT8-quantized ONNX build so absolute scores can drift ~0.01-0.03 from the float32 reference but top-K rankings are stable) and ranks subgraphs by cosine similarity against the precomputed 384-dim embedding of each subgraph's description + entities + protocol metadata. Prefer this over search_subgraphs when the goal is fuzzy, paraphrased, or describes a use-case rather than a literal protocol/entity name. Supports the same domain/network/protocol_type/min_reliability pre-filters as search_subgraphs (applied as SQL WHERE before cosine scoring for performance). Returns the same shape as search_subgraphs plus a `semantic_score` in [0,1] (>0.5 is typically a strong match).",
+        "tags": [
+          "mcp-tools"
+        ],
+        "requestBody": {
+          "required": true,
+          "content": {
+            "application/json": {
+              "schema": {
+                "type": "object",
+                "additionalProperties": false,
+                "properties": {
+                  "query": {
+                    "type": "string",
+                    "description": "Natural-language description of the data the agent wants to query, e.g. 'on-chain options market activity' or 'lending positions near liquidation'"
+                  },
+                  "limit": {
+                    "type": "integer",
+                    "description": "Max results to return (default 10, max 50)",
+                    "default": 10,
+                    "minimum": 1,
+                    "maximum": 50
+                  },
+                  "min_score": {
+                    "type": "number",
+                    "description": "Minimum cosine similarity to include (0-1). Default 0.3.",
+                    "default": 0.3
+                  },
+                  "include_unserved": {
+                    "type": "boolean",
+                    "description": "Include subgraphs with 0 active indexer allocations (returns 'no allocations' on query). Default false.",
+                    "default": false
+                  },
+                  "domain": {
+                    "type": "string",
+                    "description": "Pre-filter by domain (defi, nfts, dao, gaming, identity, infrastructure, social, analytics)"
+                  },
+                  "network": {
+                    "type": "string",
+                    "description": "Pre-filter by chain (mainnet, arbitrum-one, base, matic, bsc, optimism, avalanche, etc.)"
+                  },
+                  "protocol_type": {
+                    "type": "string",
+                    "description": "Pre-filter by protocol type (dex, lending, bridge, staking, options, perpetuals, etc.)"
+                  },
+                  "min_reliability": {
+                    "type": "number",
+                    "description": "Pre-filter: minimum reliability score (0-1)."
+                  }
+                },
+                "required": [
+                  "query"
+                ]
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "Tool result (JSON-encoded)",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object"
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Invalid input",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/mcp/tools/get_schema_changes": {
+      "post": {
+        "operationId": "tool_get_schema_changes",
+        "summary": "Return chronological schema-fingerprint changes for a subgraph.",
+        "description": "Return chronological schema-fingerprint changes for a subgraph. Each row is one detected fingerprint change with prev_fingerprint, fingerprint, and detected_at (unix seconds). Use to assess schema stability before depending on a subgraph: a long stable_days value means the schema contract is mature; a recent changed_within_24h means the upstream protocol just shipped a schema update and queries may need to be revisited.",
+        "tags": [
+          "mcp-tools"
+        ],
+        "requestBody": {
+          "required": true,
+          "content": {
+            "application/json": {
+              "schema": {
+                "type": "object",
+                "additionalProperties": false,
+                "properties": {
+                  "subgraph_id": {
+                    "type": "string",
+                    "description": "Subgraph ID (the same id used by other tools)"
+                  },
+                  "since_timestamp": {
+                    "type": "integer",
+                    "description": "Only return changes detected at or after this unix timestamp (seconds). Default 0 (full history).",
+                    "default": 0
+                  }
+                },
+                "required": [
+                  "subgraph_id"
+                ]
+              }
+            }
+          }
+        },
+        "responses": {
+          "200": {
+            "description": "Tool result (JSON-encoded)",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object"
+                }
+              }
+            }
+          },
+          "400": {
+            "description": "Invalid input",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/health": {
+      "get": {
+        "operationId": "rest_get_health",
+        "summary": "Liveness probe + total subgraph count",
+        "tags": [
+          "rest"
+        ],
+        "responses": {
+          "200": {
+            "description": "Success",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object",
+                  "properties": {
+                    "status": {
+                      "type": "string",
+                      "enum": [
+                        "ok"
+                      ]
+                    },
+                    "subgraphs": {
+                      "type": "integer"
+                    }
+                  },
+                  "required": [
+                    "status",
+                    "subgraphs"
+                  ]
+                }
+              }
+            }
+          },
+          "404": {
+            "description": "Not found",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/.well-known/subgraph/{id}.jsonld": {
+      "get": {
+        "operationId": "rest_get_well_known_subgraph_id_jsonld",
+        "summary": "JSON-LD manifest for a single subgraph (canonical location)",
+        "tags": [
+          "rest"
+        ],
+        "parameters": [
+          {
+            "name": "id",
+            "in": "path",
+            "required": true,
+            "schema": {
+              "type": "string"
+            }
+          }
+        ],
+        "responses": {
+          "200": {
+            "description": "JSON-LD SubgraphDeployment manifest",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object",
+                  "description": "JSON-LD SubgraphDeployment manifest"
+                }
+              }
+            }
+          },
+          "404": {
+            "description": "Not found",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/subgraphs/{id}.jsonld": {
+      "get": {
+        "operationId": "rest_get_subgraphs_id_jsonld",
+        "summary": "JSON-LD manifest alias (same payload as /.well-known/...)",
+        "tags": [
+          "rest"
+        ],
+        "parameters": [
+          {
+            "name": "id",
+            "in": "path",
+            "required": true,
+            "schema": {
+              "type": "string"
+            }
+          }
+        ],
+        "responses": {
+          "200": {
+            "description": "JSON-LD SubgraphDeployment manifest",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object",
+                  "description": "JSON-LD SubgraphDeployment manifest"
+                }
+              }
+            }
+          },
+          "404": {
+            "description": "Not found",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/.well-known/subgraph-index.jsonld": {
+      "get": {
+        "operationId": "rest_get_well_known_subgraph_index_jsonld",
+        "summary": "Top-100 subgraphs by reliability with their manifest URLs",
+        "tags": [
+          "rest"
+        ],
+        "responses": {
+          "200": {
+            "description": "JSON-LD SubgraphIndex",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object",
+                  "description": "JSON-LD SubgraphIndex"
+                }
+              }
+            }
+          },
+          "404": {
+            "description": "Not found",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    },
+    "/.well-known/openapi.json": {
+      "get": {
+        "operationId": "rest_get_well_known_openapi_json",
+        "summary": "This OpenAPI spec, self-served for discovery",
+        "tags": [
+          "rest"
+        ],
+        "responses": {
+          "200": {
+            "description": "OpenAPI 3.1 document",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "type": "object",
+                  "description": "OpenAPI 3.1 document"
+                }
+              }
+            }
+          },
+          "404": {
+            "description": "Not found",
+            "content": {
+              "application/json": {
+                "schema": {
+                  "$ref": "#/components/schemas/ToolError"
+                }
+              }
+            }
+          }
+        }
+      }
+    }
+  },
+  "components": {
+    "schemas": {
+      "ToolError": {
+        "type": "object",
+        "properties": {
+          "error": {
+            "type": "string"
+          }
+        },
+        "required": [
+          "error"
+        ]
+      }
+    }
+  }
+}