spice-mcp 0.1.4__tar.gz → 0.1.5__tar.gz

{spice_mcp-0.1.4 → spice_mcp-0.1.5}/.gitignore

@@ -24,3 +24,5 @@ debug_api.py
 
 # Local testing environment
 test_spice_mcp/
+scripts/
+test_mcp_cursor/

{spice_mcp-0.1.4 → spice_mcp-0.1.5}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: spice-mcp
-Version: 0.1.4
-Summary: MCP server for Dune Analytics data access
+Version: 0.1.5
+Summary: mcp server built ontop of dune api endpoint
 Author-email: Evan-Kim2028 <ekcopersonal@gmail.com>
 License-File: LICENSE
 Classifier: Operating System :: OS Independent
@@ -28,11 +28,14 @@ Description-Content-Type: text/markdown
 
 An MCP server that provides AI agents with direct access to [Dune Analytics](https://dune.com/) data. Execute queries, discover schemas and tables, and manage saved queries—all through a clean, type-safe interface optimized for AI workflows.
 
+**Discover High-Quality Tables**: Leverages [Dune Spellbook](https://github.com/duneanalytics/spellbook), Dune's official GitHub repository of curated dbt models, to surface verified, production-ready tables with rich metadata.
+
 ## Why spice-mcp?
 
 - **Agent-friendly**: Designed for AI agents using the Model Context Protocol (MCP)
+- **High-Quality Discovery**: Leverages Dune Spellbook's GitHub repository to find verified, production-ready tables with rich metadata
 - **Efficient**: Polars-first pipeline keeps data lazy until needed, reducing memory usage
-- **Discovery**: Built-in tools to explore Dune's extensive blockchain datasets
+- **Discovery**: Built-in tools to explore Dune's extensive blockchain datasets from both Dune API and Spellbook
 - **Type-safe**: Fully typed parameters and responses with FastMCP
 - **Reproducible**: Automatic query history logging and SQL artifact storage
 
@@ -73,10 +76,8 @@ An MCP server that provides AI agents with direct access to [Dune Analytics](htt
 |------|-------------|----------------|
 | `dune_query` | Execute queries by ID, URL, or raw SQL | `query` (str), `parameters` (object), `limit` (int), `offset` (int), `format` (`preview\|raw\|metadata\|poll`), `refresh` (bool), `timeout_seconds` (float) |
 | `dune_query_info` | Get metadata for a saved query | `query` (str - ID or URL) |
-| `dune_discover` | Unified discovery across Dune API and Spellbook | `keyword` (str\|list), `schema` (str), `limit` (int), `source` (`dune\|spellbook\|both`), `include_columns` (bool) |
-| `dune_find_tables` | Search schemas and list tables | `keyword` (str), `schema` (str), `limit` (int) |
+| `dune_discover` | Unified discovery across Dune API and Spellbook (returns verified tables only). **Leverages Dune Spellbook GitHub repository** for high-quality, curated tables. | `keyword` (str\|list), `schema` (str), `limit` (int), `source` (`dune\|spellbook\|both`), `include_columns` (bool) |
 | `dune_describe_table` | Get column metadata for a table | `schema` (str), `table` (str) |
-| `spellbook_find_models` | Search Spellbook dbt models | `keyword` (str\|list), `schema` (str), `limit` (int), `include_columns` (bool) |
 | `dune_health_check` | Verify API key and configuration | (no parameters) |
 | `dune_query_create` | Create a new saved query | `name` (str), `query_sql` (str), `description` (str), `tags` (list), `parameters` (list) |
 | `dune_query_update` | Update an existing saved query | `query_id` (int), `name` (str), `query_sql` (str), `description` (str), `tags` (list), `parameters` (list) |
@@ -91,6 +92,17 @@ An MCP server that provides AI agents with direct access to [Dune Analytics](htt
 
 [Dune](https://dune.com/) is a crypto data platform providing curated blockchain datasets and a public API. It aggregates on-chain data from Ethereum, Solana, Polygon, and other chains into queryable SQL tables. See the [Dune Docs](https://dune.com/docs) for more information.
 
+## What is Dune Spellbook?
+
+[Dune Spellbook](https://github.com/duneanalytics/spellbook) is Dune's official GitHub repository containing thousands of curated dbt models. These models represent high-quality, production-ready tables that are:
+
+- **Verified**: All tables are verified to exist in Dune before being returned
+- **Well-documented**: Rich metadata including column descriptions and types
+- **Maintained**: Regularly updated by the Dune community and team
+- **Production-ready**: Used by analysts and dashboards across the ecosystem
+
+spice-mcp automatically clones and parses the Spellbook repository to discover these high-quality tables, parsing dbt config blocks to resolve actual Dune table names and verifying their existence before returning them to you.
+
 ## Installation
 
 **From PyPI** (recommended):

{spice_mcp-0.1.4 → spice_mcp-0.1.5}/README.md

@@ -7,11 +7,14 @@
 
 An MCP server that provides AI agents with direct access to [Dune Analytics](https://dune.com/) data. Execute queries, discover schemas and tables, and manage saved queries—all through a clean, type-safe interface optimized for AI workflows.
 
+**Discover High-Quality Tables**: Leverages [Dune Spellbook](https://github.com/duneanalytics/spellbook), Dune's official GitHub repository of curated dbt models, to surface verified, production-ready tables with rich metadata.
+
 ## Why spice-mcp?
 
 - **Agent-friendly**: Designed for AI agents using the Model Context Protocol (MCP)
+- **High-Quality Discovery**: Leverages Dune Spellbook's GitHub repository to find verified, production-ready tables with rich metadata
 - **Efficient**: Polars-first pipeline keeps data lazy until needed, reducing memory usage
-- **Discovery**: Built-in tools to explore Dune's extensive blockchain datasets
+- **Discovery**: Built-in tools to explore Dune's extensive blockchain datasets from both Dune API and Spellbook
 - **Type-safe**: Fully typed parameters and responses with FastMCP
 - **Reproducible**: Automatic query history logging and SQL artifact storage
 
@@ -52,10 +55,8 @@ An MCP server that provides AI agents with direct access to [Dune Analytics](htt
 |------|-------------|----------------|
 | `dune_query` | Execute queries by ID, URL, or raw SQL | `query` (str), `parameters` (object), `limit` (int), `offset` (int), `format` (`preview\|raw\|metadata\|poll`), `refresh` (bool), `timeout_seconds` (float) |
 | `dune_query_info` | Get metadata for a saved query | `query` (str - ID or URL) |
-| `dune_discover` | Unified discovery across Dune API and Spellbook | `keyword` (str\|list), `schema` (str), `limit` (int), `source` (`dune\|spellbook\|both`), `include_columns` (bool) |
-| `dune_find_tables` | Search schemas and list tables | `keyword` (str), `schema` (str), `limit` (int) |
+| `dune_discover` | Unified discovery across Dune API and Spellbook (returns verified tables only). **Leverages Dune Spellbook GitHub repository** for high-quality, curated tables. | `keyword` (str\|list), `schema` (str), `limit` (int), `source` (`dune\|spellbook\|both`), `include_columns` (bool) |
 | `dune_describe_table` | Get column metadata for a table | `schema` (str), `table` (str) |
-| `spellbook_find_models` | Search Spellbook dbt models | `keyword` (str\|list), `schema` (str), `limit` (int), `include_columns` (bool) |
 | `dune_health_check` | Verify API key and configuration | (no parameters) |
 | `dune_query_create` | Create a new saved query | `name` (str), `query_sql` (str), `description` (str), `tags` (list), `parameters` (list) |
 | `dune_query_update` | Update an existing saved query | `query_id` (int), `name` (str), `query_sql` (str), `description` (str), `tags` (list), `parameters` (list) |
@@ -70,6 +71,17 @@ An MCP server that provides AI agents with direct access to [Dune Analytics](htt
 
 [Dune](https://dune.com/) is a crypto data platform providing curated blockchain datasets and a public API. It aggregates on-chain data from Ethereum, Solana, Polygon, and other chains into queryable SQL tables. See the [Dune Docs](https://dune.com/docs) for more information.
 
+## What is Dune Spellbook?
+
+[Dune Spellbook](https://github.com/duneanalytics/spellbook) is Dune's official GitHub repository containing thousands of curated dbt models. These models represent high-quality, production-ready tables that are:
+
+- **Verified**: All tables are verified to exist in Dune before being returned
+- **Well-documented**: Rich metadata including column descriptions and types
+- **Maintained**: Regularly updated by the Dune community and team
+- **Production-ready**: Used by analysts and dashboards across the ecosystem
+
+spice-mcp automatically clones and parses the Spellbook repository to discover these high-quality tables, parsing dbt config blocks to resolve actual Dune table names and verifying their existence before returning them to you.
+
 ## Installation
 
 **From PyPI** (recommended):

{spice_mcp-0.1.4 → spice_mcp-0.1.5}/docs/codex_cli.md

@@ -54,8 +54,8 @@ Verify configuration
 - `codex mcp list` should list `spice_mcp_beta` with the python command and no secrets in Env.
 
 Try some tools
-- Find schemas
-  - `mcp__spice_mcp_beta__dune_find_tables {"keyword": "dex"}`
+- Find schemas and tables
+  - `mcp__spice_mcp_beta__dune_discover {"keyword": "dex", "source": "dune"}`
 - Describe a table
   - `mcp__spice_mcp_beta__dune_describe_table {"schema": "dex", "table": "trades"}`
 - Query preview (with metadata/pagination)

{spice_mcp-0.1.4 → spice_mcp-0.1.5}/docs/discovery.md

@@ -2,20 +2,24 @@ Catalog Discovery
 
 Summary
 - There is no public REST endpoint to browse the full catalog. Discovery is best achieved using Dune SQL primitives and fallback probes.
+- **Native SHOW statements are preferred** - they're faster than information_schema queries. See issue #10 for details.
 
 Approach
 - Schemas
   - SHOW SCHEMAS
   - SHOW SCHEMAS LIKE '%keyword%'
+  - ⚠️ Avoid: information_schema.schemata (slower, causes lag)
 - Tables
   - SHOW TABLES FROM <schema>
   - If SHOW is blocked, probe candidate names via SELECT 1 FROM <schema>.<table> LIMIT 1
+  - ⚠️ Avoid: information_schema.tables (slower, causes lag)
 - Columns
   - SHOW COLUMNS FROM <schema>.<table>
   - Fallback: SELECT * FROM <schema>.<table> LIMIT 1, infer columns and Polars dtypes client-side
-- INFORMATION_SCHEMA
-  - Some deployments allow: information_schema.schemata/tables/columns
-  - If blocked, use SHOW + probes
+- INFORMATION_SCHEMA (Deprecated)
+  - Previously used for portability, but causes performance issues
+  - Native SHOW statements are now used directly (faster, no lag)
+  - Kept as fallback only if SHOW is blocked
 
 
 Helpers in this repo
@@ -23,5 +27,8 @@ Helpers in this repo
   - `find_schemas(keyword)`, `list_tables(schema, limit)`, `describe_table(schema, table)`
 
 MCP Tools
-- dune_find_tables: search schemas and list tables
+- dune_discover: **PRIMARY discovery tool** - unified search across Dune API and Spellbook, returns verified tables only
+  - Automatically parses dbt configs from Spellbook models to resolve actual Dune table names
+  - Verifies tables exist in Dune before returning (uses persistent cache)
+  - Filters out non-existent tables
 - dune_describe_table: describe columns with SHOW + fallback

spice_mcp-0.1.5/docs/tools.md

@@ -0,0 +1,118 @@
+Tools Reference
+
+All tools are exposed by the MCP server started with `spice-mcp`.
+
+1) dune_discover
+- Purpose: **PRIMARY discovery tool** for finding tables in Dune. Searches both Dune API and Spellbook repository. Returns ONLY verified, queryable tables.
+- ⚠️ **IMPORTANT**: Always use this tool instead of querying `information_schema` directly (which is slow and causes lag).
+- Input schema:
+  - keyword?: string | string[] — search term(s) to find schemas/tables (e.g., "walrus", ["layerzero", "dex"])
+  - schema?: string — schema name to list tables from (e.g., "dex", "sui_walrus")
+  - limit?: integer (default 50) — maximum number of tables to return
+  - source?: 'dune' | 'spellbook' | 'both' (default 'both') — where to search
+  - include_columns?: boolean (default true) — include column details for Spellbook models
+- Output fields:
+  - schemas: string[] — matching schema names
+  - tables: array of table objects, each with:
+    - schema: string — Spellbook subproject name (for Spellbook models) or Dune schema name
+    - table: string — Spellbook model name (for Spellbook models) or Dune table name
+    - fully_qualified_name: string — schema.table format
+    - source: 'dune' | 'spellbook'
+    - dune_schema?: string — actual Dune schema name (for Spellbook models, parsed from dbt config)
+    - dune_alias?: string — actual Dune table alias (for Spellbook models, parsed from dbt config)
+    - dune_table?: string — verified, queryable Dune table name (e.g., "sui_walrus.base_table")
+    - verified?: boolean — true (all returned tables are verified to exist)
+    - columns?: array — column details (if include_columns=true)
+  - source: string — the source parameter used
+  - message?: string — helpful message if no tables found
+- Features:
+  - Automatically parses dbt configs from Spellbook models to resolve actual Dune table names
+  - Verifies tables exist in Dune before returning (uses persistent cache)
+  - Filters out non-existent tables
+- Examples:
+  - Search for walrus tables (returns verified tables only):
+    - `dune_discover {"keyword":"walrus"}`
+    - → Returns tables with `dune_table` field like "sui_walrus.base_table"
+  - Use discovered table to query:
+    - `dune_query {"query":"SELECT * FROM sui_walrus.base_table LIMIT 10"}`
+  - Search only Spellbook:
+    - `dune_discover {"keyword":["layerzero","bridge"],"source":"spellbook"}`
+  - List tables in a schema:
+    - `dune_discover {"schema":"dex"}`
+
+2) dune_query
+- Purpose: Execute Dune queries (ID, URL, raw SQL) and return a compact preview plus Dune metadata/pagination hints.
+- ⚠️ **IMPORTANT**: Always use `dune_discover` FIRST to find verified table names. Do not guess table names or query `information_schema` directly.
+- Input schema:
+  - query: string (required) — Query ID, URL, or raw SQL using tables from `dune_discover`
+  - parameters?: object
+  - performance?: 'medium' | 'large'
+  - limit?: integer, offset?: integer, sort_by?: string, columns?: string[], sample_count?: integer
+  - refresh?: boolean, max_age?: number, timeout_seconds?: number
+  - format?: 'preview' | 'raw' | 'metadata' | 'poll' (preview by default)
+  - extras?: object (e.g., allow_partial_results, ignore_max_datapoints_per_request)
+- Output fields:
+  - type: 'preview' | 'metadata' | 'raw' | 'execution'
+  - rowcount: number, columns: string[]
+  - data_preview: object[] (first rows)
+  - execution_id: string, duration_ms: number
+  - metadata?: structured Dune metadata / execution state / error hints
+  - next_uri?: string, next_offset?: number
+  - Errors: `{ "ok": false, "error": { code, message, data: { suggestions }, context? } }`
+- Examples:
+  - Workflow: discover → query:
+    - `dune_discover {"keyword":"walrus"}` → get `dune_table="sui_walrus.base_table"`
+    - `dune_query {"query":"SELECT * FROM sui_walrus.base_table LIMIT 10"}`
+  - Preview latest metadata without rows:
+    - `dune_query {"query":"4388","format":"metadata"}`
+  - Preview first rows and metadata:
+    - `dune_query {"query":"4388","limit":10}`
+
+Logging & Artifacts
+- Successful calls are written to a JSONL audit log (see `docs/config.md` for path configuration via `QueryHistory`).
+- The canonical SQL is stored as a deduplicated artefact keyed by SHA‑256 (for raw SQL and query IDs/URLs), enabling reproducibility and offline review.
+- Result caching is handled by `adapters.dune.cache` (parquet files) and can be tuned via `SPICE_CACHE_*` environment variables.
+ 
+
+3) dune_describe_table
+- Purpose: Describe columns for a schema.table (SHOW + fallback to 1-row SELECT inference).
+- Input schema:
+  - schema: string
+  - table: string
+- Output fields:
+  - columns: [{ name, dune_type?, polars_dtype?, extra?, comment? }]
+  - Errors follow the standard MCP envelope.
+
+3) dune_health_check
+- Purpose: Basic environment and logging readiness check.
+- Output fields: ok, api_key_present, status
+
+4) dune_query_info
+- Purpose: Fetch Dune query object metadata (name, description, tags, parameter schema, SQL).
+- Input schema:
+  - query: string — ID or URL
+- Output fields:
+  - ok: boolean, status: number, query_id: number
+  - name?: string, description?: string, tags?: string[], parameters?: object[], version?: number, query_sql?: string
+
+5) dune_query_create
+- Purpose: Create a saved Dune query.
+- Input schema:
+  - name: string (required)
+  - query_sql: string (required)
+  - description?: string, tags?: string[], parameters?: object[]
+- Output: Dune query object
+
+6) dune_query_update
+- Purpose: Update a saved Dune query.
+- Input schema:
+  - query_id: integer (required)
+  - name?: string, query_sql?: string, description?: string, tags?: string[], parameters?: object[]
+- Output: Dune query object
+
+7) dune_query_fork
+- Purpose: Fork an existing saved Dune query.
+- Input schema:
+  - source_query_id: integer (required)
+  - name?: string (new name)
+- Output: Dune query object

{spice_mcp-0.1.4 → spice_mcp-0.1.5}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "spice-mcp"
-version = "0.1.4"
-description = "MCP server for Dune Analytics data access"
+version = "0.1.5"
+description = "mcp server built ontop of dune api endpoint"
 authors = [
     { name = "Evan-Kim2028", email = "ekcopersonal@gmail.com" }
 ]

{spice_mcp-0.1.4 → spice_mcp-0.1.5}/src/spice_mcp/adapters/dune/client.py

@@ -1,7 +1,6 @@
 from __future__ import annotations
 
 import os
-import re
 import time
 from collections.abc import Mapping, Sequence
 from typing import Any
@@ -47,10 +46,9 @@ class DuneAdapter(QueryExecutor, CatalogExplorer):
         self._ensure_api_key()
         start = time.time()
         q = request.query
-        if isinstance(q, str):
-            q_rewritten = _maybe_rewrite_show_sql(q)
-            if q_rewritten is not None:
-                q = q_rewritten
+        # Use native SHOW statements directly - they're faster than information_schema queries
+        # See issue #10: https://github.com/Evan-Kim2028/spice-mcp/issues/10
+        # Removed rewrite to avoid performance issues with information_schema queries
         result = _execute_dune_query(
             query_or_execution=q,
             verbose=False,
@@ -200,7 +198,9 @@ class DuneAdapter(QueryExecutor, CatalogExplorer):
     # Internal helpers --------------------------------------------------------------
     def _run_sql(self, sql: str, *, limit: int | None = None) -> pl.DataFrame:
         self._ensure_api_key()
-        sql_eff = _maybe_rewrite_show_sql(sql) or sql
+        # Use native SHOW statements directly - they're faster than information_schema queries
+        # See issue #10: https://github.com/Evan-Kim2028/spice-mcp/issues/10
+        sql_eff = sql
         df = _execute_dune_query(
             query_or_execution=sql_eff,
             verbose=False,
@@ -233,28 +233,12 @@ def _build_preview(lf: pl.LazyFrame, columns: list[str], rowcount: int) -> Resul
 
 
 def _maybe_rewrite_show_sql(sql: str) -> str | None:
-    """Rewrite certain SHOW statements to information_schema SELECTs for portability.
-
-    This allows running discovery-style commands through the parameterized raw SQL
-    template which expects SELECT statements.
+    """DEPRECATED: This function is no longer used.
+    
+    Native SHOW statements are now used directly as they're faster than
+    information_schema queries in Dune. See issue #10 for details.
+    
+    This function is kept for backward compatibility but is not called.
     """
-    s = sql.strip()
-    m = re.match(r"^SHOW\s+SCHEMAS\s+LIKE\s+'([^']+)'\s*;?$", s, flags=re.IGNORECASE)
-    if m:
-        pat = m.group(1)
-        return (
-            "SELECT schema_name AS Schema FROM information_schema.schemata "
-            f"WHERE schema_name LIKE '{pat}'"
-        )
-    if re.match(r"^SHOW\s+SCHEMAS\s*;?$", s, flags=re.IGNORECASE):
-        return "SELECT schema_name AS Schema FROM information_schema.schemata"
-
-    m = re.match(r"^SHOW\s+TABLES\s+FROM\s+([A-Za-z0-9_\.]+)\s*;?$", s, flags=re.IGNORECASE)
-    if m:
-        schema = m.group(1)
-        return (
-            "SELECT table_name AS Table FROM information_schema.tables "
-            f"WHERE table_schema = '{schema}'"
-        )
-
+    # Function body kept for reference but not executed
     return None

{spice_mcp-0.1.4 → spice_mcp-0.1.5}/src/spice_mcp/adapters/spellbook/explorer.py

@@ -133,11 +133,43 @@ class SpellbookExplorer(CatalogExplorer):
                     if not schema_yml.exists():
                         schema_yml = sql_file.parent.parent / "schema.yml"
                     
+                    # Parse dbt config to get actual Dune table name
+                    config = self._parse_dbt_config(sql_file)
+
+                    # Ignore templated dbt config values like "{{ target.schema }}"
+                    def _is_templated(val: Any) -> bool:
+                        try:
+                            s = str(val)
+                        except Exception:
+                            return False
+                        return "{{" in s and "}}" in s
+
+                    raw_schema = config.get("schema")
+                    raw_alias = config.get("alias")
+
+                    dune_schema = (
+                        raw_schema.strip() if isinstance(raw_schema, str) else raw_schema
+                    )
+                    dune_alias = (
+                        raw_alias.strip() if isinstance(raw_alias, str) else raw_alias
+                    )
+
+                    # Fall back to original names when values are templated or empty
+                    if not dune_schema or _is_templated(dune_schema):
+                        dune_schema = schema_name
+                    if not dune_alias or _is_templated(dune_alias):
+                        dune_alias = model_name
+
+                    dune_table = f"{dune_schema}.{dune_alias}"
+                    
                     models[schema_name].append({
                         "name": model_name,
                         "file": sql_file,
                         "schema_yml": schema_yml if schema_yml.exists() else None,
                         "schema": schema_name,
+                        "dune_schema": dune_schema,
+                        "dune_alias": dune_alias,
+                        "dune_table": dune_table,
                     })
         
         self._models_cache = models
@@ -264,6 +296,58 @@ class SpellbookExplorer(CatalogExplorer):
         
         return []
 
+    def _parse_dbt_config(self, sql_file: Path) -> dict[str, str]:
+        """
+        Parse dbt config block from SQL file to extract schema and alias.
+        
+        Looks for patterns like:
+        {{ config(schema='sui_walrus', alias='base_table') }}
+        {{ config(schema="sui_walrus", alias="base_table") }}
+        
+        Returns dict with 'schema' and 'alias' keys, or empty dict if not found.
+        """
+        try:
+            with open(sql_file, encoding="utf-8") as f:
+                sql = f.read()
+            
+            # Match dbt config block: {{ config(...) }}
+            # Use non-greedy match to get first config block
+            config_match = re.search(
+                r"{{\s*config\s*\((.*?)\)\s*}}",
+                sql,
+                re.IGNORECASE | re.DOTALL,
+            )
+            
+            if not config_match:
+                return {}
+            
+            config_content = config_match.group(1)
+            result: dict[str, str] = {}
+            
+            # Extract schema parameter (supports single and double quotes)
+            schema_match = re.search(
+                r"schema\s*=\s*['\"]([^'\"]+)['\"]",
+                config_content,
+                re.IGNORECASE,
+            )
+            if schema_match:
+                result["schema"] = schema_match.group(1)
+            
+            # Extract alias parameter (supports single and double quotes)
+            alias_match = re.search(
+                r"alias\s*=\s*['\"]([^'\"]+)['\"]",
+                config_content,
+                re.IGNORECASE,
+            )
+            if alias_match:
+                result["alias"] = alias_match.group(1)
+            
+            return result
+        except Exception:
+            # On any error (file read, parsing, etc.), return empty dict
+            # This allows fallback to using schema_name and model_name
+            return {}
+
     def _parse_sql_columns(self, sql_file: Path) -> list[TableColumn]:
         """Parse SQL file to extract column names from SELECT statements."""
         try:
@@ -310,4 +394,3 @@ class SpellbookExplorer(CatalogExplorer):
             pass
         
         return []
-