@aborruso/ckan-mcp-server 0.3.1

package/README.md

@@ -0,0 +1,468 @@
+# CKAN MCP Server
+
+MCP (Model Context Protocol) server for interacting with CKAN-based open data portals.
+
+## Features
+
+- ✅ Support for any CKAN server (dati.gov.it, data.gov, demo.ckan.org, etc.)
+- 🔍 Advanced search with Solr syntax
+- 📊 DataStore queries for tabular data analysis
+- 🏢 Organization and group exploration
+- 📦 Complete dataset and resource metadata
+- 🎨 Output in Markdown or JSON format
+- ⚡ Pagination and faceting support
+- 📄 MCP Resource Templates for direct data access
+- 🧪 Comprehensive test suite (101 tests, 100% passing)
+
+## Installation
+
+### From npm (recommended)
+
+```bash
+npm install ckan-mcp-server
+```
+
+### From source
+
+```bash
+# Clone or copy the project
+cd ckan-mcp-server
+
+# Install dependencies
+npm install
+
+# Build with esbuild (fast, ~4ms)
+npm run build
+
+# Run tests (101 tests)
+npm test
+```
+
+## Usage
+
+### Start with stdio (for local integration)
+
+```bash
+npm start
+```
+
+### Start with HTTP (for remote access)
+
+```bash
+TRANSPORT=http PORT=3000 npm start
+```
+
+The server will be available at `http://localhost:3000/mcp`
+
+## Claude Desktop Configuration
+
+Add to Claude Desktop configuration file (`claude_desktop_config.json`):
+
+```json
+{
+  "mcpServers": {
+    "ckan": {
+      "command": "node",
+      "args": ["/path/to/ckan-mcp-server/dist/index.js"]
+    }
+  }
+}
+```
+
+Su macOS: `~/Library/Application Support/Claude/claude_desktop_config.json`
+Su Windows: `%APPDATA%\Claude\claude_desktop_config.json`
+Su Linux: `~/.config/Claude/claude_desktop_config.json`
+
+## Available Tools
+
+### Search and Discovery
+
+- **ckan_package_search**: Search datasets with Solr queries
+- **ckan_package_show**: Complete details of a dataset
+- **ckan_package_list**: List all datasets
+
+### Organizations
+
+- **ckan_organization_list**: List all organizations
+- **ckan_organization_show**: Details of an organization
+
+### DataStore
+
+- **ckan_datastore_search**: Query tabular data
+- **ckan_datastore_search_sql**: SQL queries (in development)
+
+### Utilities
+
+- **ckan_status_show**: Verify server status
+
+## MCP Resource Templates
+
+Direct data access via `ckan://` URI scheme:
+
+- `ckan://{server}/dataset/{id}` - Dataset metadata
+- `ckan://{server}/resource/{id}` - Resource metadata and download URL
+- `ckan://{server}/organization/{name}` - Organization details
+
+Examples:
+
+```
+ckan://dati.gov.it/dataset/vaccini-covid
+ckan://demo.ckan.org/resource/abc-123
+ckan://data.gov/organization/sample-org
+```
+
+## Usage Examples
+
+### Search datasets on dati.gov.it
+
+```typescript
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  q: "popolazione",
+  rows: 20
+})
+```
+
+### Filter by organization
+
+```typescript
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  fq: "organization:regione-siciliana",
+  sort: "metadata_modified desc"
+})
+```
+
+### Search organizations with wildcard
+
+```typescript
+// Find all organizations containing "salute" in the name
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  q: "organization:*salute*",
+  rows: 0,
+  facet_field: ["organization"],
+  facet_limit: 100
+})
+```
+
+### Get statistics with faceting
+
+```typescript
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  facet_field: ["organization", "tags", "res_format"],
+  rows: 0
+})
+```
+
+### DataStore Query
+
+```typescript
+ckan_datastore_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  resource_id: "abc-123-def",
+  filters: { "regione": "Sicilia", "anno": 2023 },
+  sort: "popolazione desc",
+  limit: 50
+})
+```
+
+## Supported CKAN Portals
+
+Some of the main compatible portals:
+
+- 🇮🇹 **www.dati.gov.it/opendata** - Italy
+- 🇺🇸 **data.gov** - United States
+- 🇨🇦 **open.canada.ca/data** - Canada
+- 🇬🇧 **data.gov.uk** - United Kingdom
+- 🇪🇺 **data.europa.eu** - European Union
+- 🌍 **demo.ckan.org** - CKAN Demo
+- And 500+ more portals worldwide
+
+### Portal View URL Templates
+
+Some CKAN portals expose non-standard web URLs for viewing datasets or organizations. To support those cases, this project ships with `src/portals.json`, which maps known portal API URLs (and aliases) to custom view URL templates.
+
+When generating a dataset or organization view link, the server:
+- matches the `server_url` against `api_url` and `api_url_aliases` in `src/portals.json`
+- uses the portal-specific `dataset_view_url` / `organization_view_url` template when available
+- falls back to the generic defaults (`{server_url}/dataset/{name}` and `{server_url}/organization/{name}`)
+
+You can extend `src/portals.json` by adding new entries under `portals` if a portal uses different web URL patterns.
+
+## Advanced Solr Queries
+
+CKAN uses Apache Solr for search. Examples:
+
+```
+# Basic search
+q: "popolazione"
+
+# Field search
+q: "title:popolazione"
+q: "notes:sanità"
+
+# Boolean operators
+q: "popolazione AND sicilia"
+q: "popolazione OR abitanti"
+q: "popolazione NOT censimento"
+
+# Filters (fq)
+fq: "organization:comune-palermo"
+fq: "tags:sanità"
+fq: "res_format:CSV"
+
+# Wildcard
+q: "popolaz*"
+
+# Date range
+fq: "metadata_modified:[2023-01-01T00:00:00Z TO *]"
+```
+
+### Advanced Query Examples
+
+These real-world examples demonstrate powerful Solr query combinations tested on the Italian open data portal (dati.gov.it):
+
+#### 1. Fuzzy Search + Date Math + Boosting
+
+Find healthcare datasets (tolerating spelling errors) modified in the last 6 months, prioritizing title matches:
+
+```typescript
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  q: "(title:sanità~2^3 OR title:salute~2^3 OR notes:sanità~1) AND metadata_modified:[NOW-6MONTHS TO *]",
+  sort: "score desc, metadata_modified desc",
+  rows: 30
+})
+```
+
+**Techniques used**:
+- `sanità~2` - Fuzzy search with edit distance 2 (finds "sanita", "sanitá", minor typos)
+- `^3` - Boosts title matches 3x higher in relevance scoring
+- `NOW-6MONTHS` - Dynamic date math for rolling time windows
+- Combined boolean logic with multiple field searches
+
+**Results**: 871 datasets including hospital units, healthcare organizations, medical services
+
+#### 2. Proximity Search + Complex Boolean
+
+Environmental datasets where "inquinamento" and "aria" (air pollution) appear close together, excluding water-related datasets:
+
+```typescript
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  q: "(notes:\"inquinamento aria\"~5 OR title:\"qualità aria\"~3) AND NOT (title:acqua OR title:mare)",
+  facet_field: ["organization", "res_format"],
+  rows: 25
+})
+```
+
+**Techniques used**:
+- `"inquinamento aria"~5` - Proximity search (words within 5 positions)
+- `~3` - Tighter proximity for title matches
+- `NOT (title:acqua OR title:mare)` - Exclude water/sea datasets
+- Faceting for statistical breakdown
+
+**Results**: 306 datasets, primarily air quality monitoring from Milan (44) and Palermo (161), formats: XML (150), CSV (124), JSON (76)
+
+#### 3. Wildcard + Field Existence + Range Queries
+
+Regional datasets with at least 5 resources, published in the last year:
+
+```typescript
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  q: "organization:regione* AND num_resources:[5 TO *] AND metadata_created:[NOW-1YEAR TO *] AND res_format:*",
+  sort: "num_resources desc, metadata_modified desc",
+  facet_field: ["organization"],
+  rows: 40
+})
+```
+
+**Techniques used**:
+- `regione*` - Wildcard matches all regional organizations
+- `[5 TO *]` - Inclusive range (5 or more resources)
+- `res_format:*` - Field existence check (has at least one resource format)
+- `NOW-1YEAR` - Rolling 12-month window
+
+**Results**: 5,318 datasets, top contributors: Lombardy (3,012), Tuscany (1,151), Puglia (460)
+
+#### 4. Date Ranges + Exclusive Bounds
+
+ISTAT datasets with moderate resource count (10-50), modified in specific date range:
+
+```typescript
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  q: "(title:istat OR organization:*istat*) AND num_resources:{9 TO 51} AND metadata_modified:[2025-07-01T00:00:00Z TO 2025-12-31T23:59:59Z]",
+  sort: "metadata_modified desc",
+  facet_field: ["res_format", "tags"],
+  rows: 30
+})
+```
+
+**Techniques used**:
+- `{9 TO 51}` - Exclusive bounds (10-50 resources, excluding 9 and 51)
+- `[2025-07-01T00:00:00Z TO 2025-12-31T23:59:59Z]` - Explicit date range
+- Combined organization wildcard with title search
+- Multiple facets for content analysis
+
+**Note**: This specific query returned 0 results due to the narrow time window, demonstrating how precise constraints work.
+
+### Solr Query Syntax Reference
+
+**Boolean Operators**: `AND`, `OR`, `NOT`, `+required`, `-excluded`
+**Wildcards**: `*` (multiple chars), `?` (single char) - Note: left truncation not supported
+**Fuzzy**: `~N` (edit distance), e.g., `health~2`
+**Proximity**: `"phrase"~N` (words within N positions)
+**Boosting**: `^N` (relevance multiplier), e.g., `title:water^2`
+**Ranges**:
+  - Inclusive: `[a TO b]`, e.g., `num_resources:[5 TO 10]`
+  - Exclusive: `{a TO b}`, e.g., `num_resources:{0 TO 100}`
+  - Open-ended: `[2024-01-01T00:00:00Z TO *]`
+**Date Math**: `NOW`, `NOW-1YEAR`, `NOW-6MONTHS`, `NOW-7DAYS`, `NOW/DAY`
+**Field Existence**: `field:*` (field exists), `NOT field:*` (field missing)
+
+## Project Structure
+
+```
+ckan-mcp-server/
+├── src/
+│   ├── index.ts          # Entry point
+│   ├── server.ts         # MCP server setup
+│   ├── types.ts          # Types & schemas
+│   ├── utils/
+│   │   ├── http.ts       # CKAN API client
+│   │   └── formatting.ts # Output formatting
+│   ├── tools/
+│   │   ├── package.ts    # Package search/show
+│   │   ├── organization.ts # Organization tools
+│   │   ├── datastore.ts  # DataStore queries
+│   │   └── status.ts     # Server status
+│   ├── resources/        # MCP Resource Templates
+│   │   ├── index.ts
+│   │   ├── uri.ts        # URI parsing
+│   │   ├── dataset.ts
+│   │   ├── resource.ts
+│   │   └── organization.ts
+│   └── transport/
+│       ├── stdio.ts      # Stdio transport
+│       └── http.ts       # HTTP transport
+├── tests/                # Test suite (101 tests)
+├── dist/                 # Compiled files (generated)
+├── package.json
+└── README.md
+```
+
+## Development
+
+### Testing
+
+The project uses **Vitest** for automated testing:
+
+```bash
+# Run all tests
+npm test
+
+# Run tests in watch mode
+npm run test:watch
+
+# Run tests with coverage report
+npm run test:coverage
+```
+
+Test coverage target is 80%. Current test suite includes:
+- Unit tests for utility functions (formatting, HTTP)
+- Integration tests for MCP tools with mocked CKAN API responses
+- Mock fixtures for CKAN API success and error scenarios
+
+See `tests/README.md` for detailed testing guidelines.
+
+### Build
+
+The project uses **esbuild** for ultra-fast compilation (~4ms):
+
+```bash
+# Build with esbuild (default)
+npm run build
+
+# Watch mode for development
+npm run watch
+```
+
+### Manual Testing
+
+```bash
+# Start server in HTTP mode
+TRANSPORT=http PORT=3000 npm start
+
+# In another terminal, test available tools
+curl -X POST http://localhost:3000/mcp \
+  -H 'Content-Type: application/json' \
+  -H 'Accept: application/json, text/event-stream' \
+  -d '{"jsonrpc":"2.0","method":"tools/list","id":1}'
+```
+
+## Troubleshooting
+
+### dati.gov.it URL
+
+**Important**: The correct URL for the Italian portal is `https://www.dati.gov.it/opendata` (not `https://dati.gov.it`).
+
+### Connection error
+
+```
+Error: Server not found: https://esempio.gov.it
+```
+
+**Solution**: Verify the URL is correct and the server is online. Use `ckan_status_show` to verify.
+
+### No results
+
+```typescript
+// Use a more generic query
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  q: "*:*"
+})
+
+// Check contents with faceting
+ckan_package_search({
+  server_url: "https://www.dati.gov.it/opendata",
+  facet_field: ["tags", "organization"],
+  rows: 0
+})
+```
+
+## Contributing
+
+Contributions are welcome! Please:
+
+1. Fork the project
+2. Create a branch for the feature (`git checkout -b feature/AmazingFeature`)
+3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)
+4. Push to the branch (`git push origin feature/AmazingFeature`)
+5. Open a Pull Request
+
+## License
+
+MIT License - See LICENSE.txt file for complete details.
+
+## Useful Links
+
+- **CKAN**: https://ckan.org/
+- **CKAN API Documentation**: https://docs.ckan.org/en/latest/api/
+- **MCP Protocol**: https://modelcontextprotocol.io/
+- **onData**: https://www.ondata.it/
+- **dati.gov.it**: https://www.dati.gov.it/opendata/
+
+## Support
+
+For issues or questions:
+- Open an issue on GitHub
+- Contact onData: https://www.ondata.it/
+
+---
+
+Created with ❤️ by onData for the open data community

package/REFACTORING.md

@@ -0,0 +1,237 @@
+# Refactoring Documentation
+
+## Overview
+
+Il codebase è stato refactorizzato da un singolo file monolitico di 1021 righe a una struttura modulare di 11 file.
+
+## Motivazione
+
+**Problemi del file unico**:
+- 1021 righe difficili da navigare
+- Modifiche rischiose (alta probabilità di errori)
+- Testing complesso
+- Code review difficili
+- Merge conflicts probabili in collaborazione
+
+**Vantaggi della struttura modulare**:
+- File più piccoli (max 350 righe)
+- Modifiche localizzate e sicure
+- Testing isolato per tool
+- Manutenzione semplificata
+- Collaborazione efficiente
+
+## Nuova Struttura
+
+```
+src/
+├── index.ts              # Entry point (39 lines)
+├── server.ts             # MCP server setup (12 lines)
+├── types.ts              # Types & schemas (16 lines)
+├── utils/
+│   ├── http.ts           # CKAN API client (51 lines)
+│   └── formatting.ts     # Output formatting (37 lines)
+├── tools/
+│   ├── package.ts        # Package tools (350 lines)
+│   ├── organization.ts   # Organization tools (341 lines)
+│   ├── datastore.ts      # DataStore tools (146 lines)
+│   └── status.ts         # Status tools (66 lines)
+└── transport/
+    ├── stdio.ts          # Stdio transport (12 lines)
+    └── http.ts           # HTTP transport (27 lines)
+```
+
+**Total**: 1097 lines (vs 1021 original, +76 lines for better organization)
+
+## File Descriptions
+
+### Core Files
+
+**`index.ts`** (Entry Point)
+- Importa e registra tutti i tool
+- Sceglie transport (stdio/http)
+- Gestisce startup e error handling
+
+**`server.ts`** (Server Configuration)
+- Crea istanza MCP server
+- Configurazione base (name, version)
+
+**`types.ts`** (Type Definitions)
+- `ResponseFormat` enum
+- `ResponseFormatSchema` Zod validator
+- `CHARACTER_LIMIT` constant
+
+### Utils
+
+**`utils/http.ts`** (HTTP Client)
+- `makeCkanRequest<T>()` - HTTP client per CKAN API
+- Normalizzazione URL
+- Gestione errori (timeout, 404, network)
+- User-Agent header
+
+**`utils/formatting.ts`** (Output Formatting)
+- `truncateText()` - Limita output a CHARACTER_LIMIT
+- `formatDate()` - Format date in Italian locale
+- `formatBytes()` - Human-readable file sizes
+
+### Tools
+
+**`tools/package.ts`** (Package/Dataset Tools)
+- `ckan_package_search` - Search datasets (182 lines handler)
+- `ckan_package_show` - Dataset details (138 lines handler)
+
+**`tools/organization.ts`** (Organization Tools)
+- `ckan_organization_list` - List organizations (118 lines)
+- `ckan_organization_show` - Org details (95 lines)
+- `ckan_organization_search` - Search orgs (102 lines)
+
+**`tools/datastore.ts`** (DataStore Tools)
+- `ckan_datastore_search` - Query tabular data (130 lines)
+
+**`tools/status.ts`** (Status Tools)
+- `ckan_status_show` - Check server status (51 lines)
+
+### Transport
+
+**`transport/stdio.ts`** (Stdio Transport)
+- `runStdio()` - Standard input/output transport
+- For Claude Desktop and local MCP clients
+
+**`transport/http.ts`** (HTTP Transport)
+- `runHTTP()` - HTTP server on configurable port
+- For remote access via HTTP POST
+
+## Build Configuration
+
+**No changes needed** - esbuild continua a funzionare:
+```bash
+npm run build  # 15ms build time
+```
+
+esbuild automaticamente:
+- Bundle tutti i moduli interni
+- Tree-shake codice non usato
+- Mantiene external dependencies separate
+- Produce singolo file `dist/index.js`
+
+## Testing Results
+
+✅ **Build**: Successful (15ms)
+✅ **Tool Registration**: All 7 tools listed
+✅ **Tool Execution**: `ckan_status_show` tested successfully
+✅ **Backward Compatibility**: 100% - stessa API MCP
+
+## Benefits Achieved
+
+### Maintainability
+- Ogni file < 350 righe (easy to understand)
+- Moduli isolati (modifiche localizzate)
+- Clear separation of concerns
+
+### Testing
+- Tool handlers isolati e testabili
+- Utilities testabili indipendentemente
+- Mock più facili per unit tests
+
+### Collaboration
+- Merge conflicts ridotti
+- Code review più veloci
+- Feature development parallelo possibile
+
+### Performance
+- Build time invariato: ~15ms
+- Bundle size invariato: ~30KB
+- Runtime performance identico
+- Tree-shaking preservato
+
+## Migration Notes
+
+### Old Structure
+```typescript
+// src/index.ts (1021 lines)
+// Everything in one file
+```
+
+### New Structure
+```typescript
+// src/index.ts (39 lines)
+import { createServer } from "./server.js";
+import { registerPackageTools } from "./tools/package.js";
+// ...
+
+const server = createServer();
+registerPackageTools(server);
+// ...
+```
+
+### Backward Compatibility
+
+✅ **Zero breaking changes**:
+- Stessi tool MCP esposti
+- Stessi parametri input
+- Stesso formato output
+- Stesso comportamento
+
+### Old File Preserved
+
+`src/index-old.ts` - Backup del file originale (per riferimento)
+
+## Future Enhancements Enabled
+
+Con la nuova struttura diventa più facile:
+
+1. **Unit Testing**: Test singoli tool in isolamento
+2. **New Tools**: Aggiungere tool in file separati
+3. **Shared Logic**: Utilities riutilizzabili
+4. **Documentation**: JSDoc per ogni modulo
+5. **Type Safety**: Types centralizzati
+6. **Optimization**: Ottimizzare singoli moduli
+
+## Maintenance Guide
+
+### Adding a New Tool
+
+1. Create `src/tools/newtool.ts`:
+```typescript
+import type { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
+
+export function registerNewTools(server: McpServer) {
+  server.registerTool("tool_name", { ... }, async (params) => { ... });
+}
+```
+
+2. Import in `src/index.ts`:
+```typescript
+import { registerNewTools } from "./tools/newtool.js";
+registerNewTools(server);
+```
+
+3. Build and test:
+```bash
+npm run build
+npm start
+```
+
+### Modifying a Tool
+
+1. Edit relevant file in `src/tools/`
+2. Changes are isolated
+3. Build and test
+4. No risk to other tools
+
+### Adding Utilities
+
+1. Create in `src/utils/`
+2. Export function
+3. Import where needed
+4. Automatic tree-shaking if unused
+
+## Conclusion
+
+Il refactoring è stato completato con successo:
+- ✅ Struttura modulare
+- ✅ Build funzionante
+- ✅ Backward compatible
+- ✅ Testing verificato
+- ✅ Performance preserved
+
+Il codice è ora più manutenibile, testabile e scalabile, mantenendo tutta la funzionalità originale.