@aborruso/ckan-mcp-server 0.4.8 → 0.4.10

package/.devin/wiki.json

@@ -0,0 +1,273 @@
+{
+  "repo_notes": [
+    {
+      "content": ""
+    }
+  ],
+  "pages": [
+    {
+      "title": "Overview",
+      "purpose": "Introduce the CKAN MCP Server system, explaining its purpose as a Model Context Protocol server that enables AI agents to interact with 500+ CKAN open data portals worldwide through natural language and structured queries",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Key Features",
+      "purpose": "Highlight the main capabilities: 15+ CKAN tools, dual runtime support, MCP Resource Templates, and global edge deployment",
+      "parent": "Overview",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Architecture Overview",
+      "purpose": "Provide a high-level architectural diagram showing the two-tier system: MCP Server Core and CKAN Portals, plus the dual deployment model",
+      "parent": "Overview",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Getting Started",
+      "purpose": "Guide users through the fastest path to using the system based on their needs and technical expertise",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Claude Desktop Setup",
+      "purpose": "Configure Claude Desktop to use the MCP server via stdio (local) or HTTP (remote/Workers) transport modes",
+      "parent": "Getting Started",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Installation Options",
+      "purpose": "Compare the three deployment options: npm global install, Cloudflare Workers, and self-hosted HTTP server",
+      "parent": "Getting Started",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Architecture",
+      "purpose": "Deep dive into the system architecture, component relationships, and design decisions",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "System Components",
+      "purpose": "Explain the modular architecture: entry points, transport layer, server core, tool handlers, resource templates, and utilities",
+      "parent": "Architecture",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "MCP Server Core",
+      "purpose": "Explain the MCP server implementation: tool registration system, request routing, Zod validation, and error handling",
+      "parent": "Architecture",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Transport Layer",
+      "purpose": "Document the three transport modes: stdio (12 lines), HTTP (27 lines), and Cloudflare Workers WebStandardStreamable, explaining how the same tools run across different runtimes",
+      "parent": "Architecture",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Tool Layer",
+      "purpose": "Describe the tool implementation pattern: Zod schemas, handler functions, makeCkanRequest abstraction, and output formatting (1,465+ lines of business logic)",
+      "parent": "Architecture",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Dual Runtime Design",
+      "purpose": "Explain how the same codebase runs on Node.js and Cloudflare Workers V8 isolates, including build targets, bundle sizes, and runtime differences",
+      "parent": "Architecture",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "User Guides",
+      "purpose": "Practical guides for end users interacting with CKAN data through the system",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Querying CKAN Data",
+      "purpose": "Guide to common CKAN queries: searching datasets, exploring organizations, querying DataStore, using facets, and understanding Solr syntax",
+      "parent": "User Guides",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Advanced Search Techniques",
+      "purpose": "Document Solr query syntax: fuzzy search (~N), proximity search (\"phrase\"~N), boosting (^N), range queries, date math, wildcards, and boolean operators",
+      "parent": "User Guides",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "DataStore Queries",
+      "purpose": "Explain DataStore functionality: basic key-value filtering, SQL queries, field selection, sorting, pagination, and the 32,000 record limit",
+      "parent": "User Guides",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Working with Organizations",
+      "purpose": "Guide to organization tools: listing with package counts, searching by pattern, retrieving details, and understanding organizational hierarchies",
+      "parent": "User Guides",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Tool Reference",
+      "purpose": "Complete reference documentation for all 15+ MCP tools with parameters, return types, and usage examples",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Dataset Tools",
+      "purpose": "Document ckan_package_search, ckan_find_relevant_datasets, ckan_package_show, ckan_package_list with detailed parameter descriptions and examples",
+      "parent": "Tool Reference",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Organization Tools",
+      "purpose": "Document ckan_organization_list, ckan_organization_show, ckan_organization_search with sorting, pagination, and filtering options",
+      "parent": "Tool Reference",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "DataStore Tools",
+      "purpose": "Document ckan_datastore_search and ckan_datastore_search_sql with filtering syntax, SQL query examples, and limit constraints",
+      "parent": "Tool Reference",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Metadata Tools",
+      "purpose": "Document ckan_tag_list, ckan_group_list, ckan_group_show, ckan_group_search, and ckan_status_show for portal exploration",
+      "parent": "Tool Reference",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "MCP Resource Templates",
+      "purpose": "Explain the ckan:// URI scheme for accessing datasets, resources, and organizations via MCP Resource Templates",
+      "parent": "Tool Reference",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Developer Guide",
+      "purpose": "Guide for developers who want to modify, extend, or contribute to the codebase",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Project Structure",
+      "purpose": "Document the codebase organization: src/ structure, tool modules (1,465+ lines), utilities (138+ lines), resources (50+ lines), and test organization (2,340 lines)",
+      "parent": "Developer Guide",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Adding New Tools",
+      "purpose": "Step-by-step guide to implementing a new MCP tool: defining Zod schemas, creating handlers, using makeCkanRequest, formatting outputs, and registering with the server",
+      "parent": "Developer Guide",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    },
+    {
+      "title": "Utility Functions",
+      "purpose": "Document the utility layer: makeCkanRequest (51 lines, 98.59% coverage), formatting functions (37 lines, 98% coverage), query resolution, and URL generation",
+      "parent": "Developer Guide",
+      "page_notes": [
+        {
+          "content": ""
+        }
+      ]
+    }
+  ]
+}

package/AGENTS.md

@@ -4,15 +4,97 @@
 These instructions are for AI assistants working in this project.
 
 Always open `@/openspec/AGENTS.md` when the request:
+
 - Mentions planning or proposals (words like proposal, spec, change, plan)
 - Introduces new capabilities, breaking changes, architecture shifts, or big performance/security work
 - Sounds ambiguous and you need the authoritative spec before coding
 
 Use `@/openspec/AGENTS.md` to learn:
+
 - How to create and apply change proposals
 - Spec format and conventions
 - Project structure and guidelines
 
 Keep this managed block so 'openspec update' can refresh the instructions.
 
-<!-- OPENSPEC:END -->
+<!-- OPENSPEC:END -->
+
+# Agent Guide
+
+## Scope
+
+- Repo: `ckan-mcp-server`
+- Stack: Node.js + TypeScript (ESM)
+- Tests: Vitest
+- Build: esbuild
+- Cloudflare Workers: `wrangler`
+
+## Cursor and Copilot Rules
+
+- No `.cursor/rules`, `.cursorrules`, or `.github/copilot-instructions.md` found
+
+## Commands
+
+**Install**: `npm install`
+
+**Build**: `npm run build` | `npm run build:tsc` | `npm run build:worker`
+
+**Run**: `npm start` | `npm run dev` | `npm run watch` | `npm run dev:worker` | `npm run deploy`
+
+**Test**: `npm test` | `npm run test:watch` | `npm run test:coverage`
+
+**Single test**: `npm test -- tests/unit/http.test.ts` | `npm test -- -t "testName"`
+
+## TypeScript Style
+
+Use strict typing, avoid `any` unless from CKAN payloads. Prefer explicit return types, `type` aliases, ESM imports with `.js` extensions. Keep `noUnusedLocals` and `noUnusedParameters` clean.
+
+## Import Conventions
+
+Use `import type` for type-only imports. Group by kind: external, internal, types. Double quotes, relative paths.
+
+## Naming
+
+`camelCase` vars/functions, `PascalCase` types/classes, `UPPER_SNAKE_CASE` constants. Tool names match MCP tool ids.
+
+## Error Handling
+
+Wrap tool handlers in `try/catch`. Return `{ isError: true }` for MCP errors. Include context, map HTTP errors to readable messages, preserve cause when rethrowing.
+
+## Tool Responses
+
+Use `ResponseFormat` for markdown vs JSON. `truncateText` for large payloads. Pretty-print JSON, include `structuredContent` for JSON mode.
+
+## Testing Guidelines
+
+Vitest with `globals: true`. Place tests in `tests/unit` or `tests/integration`. AAA pattern, mock via fixtures in `tests/fixtures`, descriptive names.
+
+## Configuration
+
+Node `>=18`. Worker build in `wrangler.toml`. Vitest coverage thresholds enforced.
+
+## Change Hygiene
+
+Minimal focused diffs. No unrelated refactors. Update tests for behavior changes. Avoid editing `dist/`.
+
+## Project Layout
+
+`src/index.ts` entry, `src/server.ts` wiring, `src/tools/` handlers, `src/utils/` helpers, `src/resources/` templates, `src/transport/` stdio/HTTP. `tests/unit/` utilities, `tests/integration/` behavior, `tests/fixtures/` mocks.
+
+## CSV Data Exploration
+
+For exploring CSV resources from datasets, use duckdb CLI (already installed) with direct HTTP URL:
+
+```bash
+duckdb -jsonlines -c "DESCRIBE SELECT * FROM read_csv('http://url/file.csv')"
+duckdb -jsonlines -c "SUMMARIZE SELECT * FROM read_csv('http://url/file.csv')"
+duckdb -jsonlines -c "SELECT * FROM read_csv('http://url/file.csv') USING SAMPLE 10"
+```
+
+Use direct resource URLs (http/https), not GitHub view/blob URLs. The `-jsonlines` parameter outputs in JSONL format, easier for AI to parse.
+
+For random sampling, use `USING SAMPLE N` syntax (where N is the number of rows):
+
+```bash
+duckdb -jsonlines -c "SELECT * FROM read_csv('http://url/file.csv') USING SAMPLE 10"
+```

package/CLAUDE.md

@@ -41,7 +41,7 @@ The server exposes MCP tools for:
 # Build project (uses esbuild - fast and lightweight)
 npm run build
 
-# Run test suite (113 tests - unit + integration)
+# Run test suite (179 tests - unit + integration)
 npm test
 
 # Watch mode for tests during development
@@ -73,8 +73,8 @@ npm run deploy            # Deploy to Cloudflare Workers
 The project uses **esbuild** for compilation and **vitest** for testing:
 
 - **Build**: Ultra-fast builds (milliseconds instead of minutes)
-- **Tests**: 101 tests (unit + integration) with 100% success rate
-- **Coverage**: Available via vitest with v8 coverage engine
+- **Tests**: 179 tests (unit + integration) with 100% success rate
+- **Coverage**: ~39% overall (utils: 98%, tools: 15-20%) - available via vitest with v8 coverage engine
 
 The `build:tsc` script is available as a fallback but can cause memory issues in some environments (particularly WSL). Always use `npm run build` which uses esbuild.
 
@@ -101,7 +101,7 @@ tests/
     └── errors/                # Error scenario mocks
 ```
 
-**Test Coverage**: 101 tests total (36 unit + 65 integration)
+**Test Coverage**: 179 tests total (85 unit + 94 integration)
 
 When making changes:
 1. Run tests before committing: `npm test`
@@ -292,11 +292,14 @@ npm run test:watch
 npm run test:coverage
 ```
 
-Test coverage target is 80%. Current test suite includes:
-- Unit tests for utility functions (formatting, HTTP)
+Current test coverage: ~39% overall (utility modules: 98%, tool handlers: 15-20%).
+
+Test suite includes:
+- Unit tests for utility functions (formatting, HTTP, URI parsing, URL generation, search)
 - Integration tests for MCP tools with mocked CKAN API responses
 - Mock fixtures for CKAN API success and error scenarios
 
+Coverage is strong for utility modules but needs improvement for tool handlers.
 See `tests/README.md` for detailed testing guidelines and fixture structure.
 
 ### Manual Testing
@@ -338,8 +341,8 @@ To test with Claude Desktop, add the MCP configuration to the config file.
 - Direct data access for datasets, resources, organizations
 
 **v0.2.0 (2026-01-08)**: Comprehensive test suite
-- 113 tests (unit + integration)
-- 97%+ code coverage
+- 179 tests (unit + integration)
+- ~39% code coverage (utils well-tested, tools improving)
 
 **v0.1.0 (2026-01-08)**: Modular refactoring
 - Restructured from monolithic file to 11 modules
@@ -375,3 +378,21 @@ When releasing a new version:
 7. **Deploy to Cloudflare** (if code changed): `npm run deploy`
 
 See `docs/DEPLOYMENT.md` for detailed Cloudflare deployment instructions.
+
+## CSV Data Exploration
+
+For exploring CSV resources from datasets, use duckdb CLI (already installed) with direct HTTP URL:
+
+```bash
+duckdb -jsonlines -c "DESCRIBE SELECT * FROM read_csv('http://url/file.csv')"
+duckdb -jsonlines -c "SUMMARIZE SELECT * FROM read_csv('http://url/file.csv')"
+duckdb -jsonlines -c "SELECT * FROM read_csv('http://url/file.csv') USING SAMPLE 10"
+```
+
+Use direct resource URLs (http/https), not GitHub view/blob URLs. The `-jsonlines` parameter outputs in JSONL format, easier for AI to parse.
+
+For random sampling, use `USING SAMPLE N` syntax (where N is the number of rows):
+
+```bash
+duckdb -jsonlines -c "SELECT * FROM read_csv('http://url/file.csv') USING SAMPLE 10"
+```

package/LOG.md

@@ -1,5 +1,61 @@
 # LOG
 
+## 2026-01-15
+
+### Version 0.4.10 - Guided MCP prompts
+- **Feature**: Added 5 guided MCP prompts (theme, organization, format, recent datasets, dataset analysis)
+- **Docs**: README and new `docs/prompts.md` updated with usage examples
+- **Tests**: Added prompt unit tests; total now 184 tests (all passing)
+- **Files**: New `src/prompts/*`, updates in `src/server.ts`, `src/worker.ts`, README.md
+
+## 2026-01-11
+
+### Version 0.4.9 - Security, Testing & Documentation
+- **Security**: Updated @modelcontextprotocol/sdk from 1.25.1 to 1.25.2 (fixes HIGH severity ReDoS vulnerability)
+- **Testing**: Added 49 new unit tests for package.ts scoring functions
+- **Coverage**: Improved from 37.33% to 38.63% (package.ts: 12.5% to 15%)
+- **Total tests**: 179 tests (all passing, +49 from 130)
+- **Documentation**: Corrected test coverage claims (was "113 tests, 97%+" now accurate "179 tests, ~39%")
+- **Deployment**: Added npm audit check to DEPLOYMENT.md
+- **Files modified**: package.json, src/server.ts, src/worker.ts, README.md, CLAUDE.md, docs/DEPLOYMENT.md
+- **New file**: tests/unit/package-scoring.test.ts
+- **No breaking changes**: All existing functionality preserved
+
+### Test improvements - package scoring functions
+- **Added**: 49 new unit tests for package.ts scoring functions
+- **Coverage improvement**: package.ts from 12.5% to 15%
+- **Overall coverage**: 37.33% to 38.63%
+- **Total tests**: 130 to 179 tests (all passing)
+- **New test file**: tests/unit/package-scoring.test.ts
+- **Functions tested**:
+  - extractQueryTerms (10 tests)
+  - escapeRegExp (6 tests)
+  - textMatchesTerms (10 tests)
+  - scoreTextField (6 tests)
+  - scoreDatasetRelevance (17 tests with edge cases)
+- **Exports**: Made internal functions testable (extractQueryTerms, escapeRegExp, textMatchesTerms, scoreTextField)
+- **Impact**: Better coverage of dataset relevance scoring logic
+
+### Documentation corrections - test coverage accuracy
+- **Fix**: Corrected test coverage claims in README.md and CLAUDE.md
+- **Previous claim**: "113 tests, 97%+ coverage"
+- **Actual values**: 130 tests passing, ~37% overall coverage
+  - Utility modules: 98% coverage (excellent)
+  - Tool handlers: 12-20% coverage (needs improvement)
+- **Impact**: Documentation now accurately reflects project state
+- **Files modified**: README.md, CLAUDE.md
+
+### Documentation enhancement - deployment security
+- **Added**: npm audit check to DEPLOYMENT.md (Step 4.5)
+- **Added**: Security audit to pre-release checklist
+- **Recommendation**: Always run `npm audit` before production deployment
+
+### Security fix - MCP SDK update
+- **Fix**: Update @modelcontextprotocol/sdk from 1.25.1 to 1.25.2
+- **Reason**: Resolves HIGH severity ReDoS vulnerability (GHSA-8r9q-7v3j-jr4g)
+- **Tests**: All 130 tests passing
+- **Audit**: 0 vulnerabilities
+
 ## 2026-01-10
 
 ### Version 0.4.8 - Organization list fallback