@aborruso/ckan-mcp-server 0.4.119 → 0.4.121

package/LOG.md

@@ -1,7 +1,101 @@
 # LOG
 
+## 2026-09-05
+
+### v0.4.121 - clearer 404 on datastore_search_sql
+
+Patch release for the fix in #532: a 404 on `datastore_search_sql` now says the portal does not expose the SQL endpoint, instead of sending the caller to `ckan_package_show` for a resource_id that was already valid. Also ships the telemetry pipeline fixes and the DEPLOYMENT.md/CLAUDE.md realignment from earlier today.
+
+### DEPLOYMENT.md and CLAUDE.md realigned with the actual deployment
+
+Six things the deployment guide got wrong, all verified against the live system:
+
+- tool count stuck at 7; `/health` reports 20 tools, 7 resources, 6 prompts on v0.4.120
+- bundle "~400KB" in two places, while the doc's own sample output said 541 KiB; rebuilt: 533 KB
+- the free tier presented as the configuration in use. Workers Observability keeps 7 days of telemetry on this account (measured: events at 7 days back, none at 8), which is the paid retention — the free-tier request and CPU limits are not the ones this deployment operates under
+- `demo.ckan.org` in three curl examples, which CLAUDE.md forbids for tests
+- the release workflow told you to commit and push straight to `main`, in both files. Now branch, PR, squash-merge, then tag on the merged commit
+- a stale `Co-Authored-By` line in the sample commit
+
+Also corrected the troubleshooting entry calling CPU-limit errors "rare": 22 in the first four days of September against 11 in all of August. Left flagged as unexplained.
+
+Left alone on purpose: archived OpenSpec proposals, historical LOG entries, and `demo.ckan.org` where it appears as an example for end users (README, EXAMPLES, SKILL) rather than as a test target.
+
+### 404 on datastore_search_sql: hint sent callers round a loop
+
+A 404 on `datastore_search_sql` fell into the generic `datastore_search` branch of
+`formatCkanError`, which answers "get a valid resource_id first: call `ckan_package_show`".
+On a portal that does not expose SQL the resource_id is fine and `ckan_package_show`
+hands back the same one, so a client following the hint retries into the same 404.
+
+SQL access is optional in CKAN and widely disabled: `datastore_search_sql?sql=SELECT 1`
+returns 404 on Toronto's portal and 200 on dati.comune.messina.it, while
+`datastore_search` answers 200 on both. A bad table name comes back as a 400, not a 404,
+so a 404 on that action means the endpoint itself is missing. The hint now says so and
+points at `ckan_datastore_search`.
+
+Verified e2e: Toronto returns the new hint; Messina counts 202.906 rows in a single SQL
+call. 498 tests pass, one added.
+
+### Telemetry pipeline: cross-worker contamination, missing outcome, unattributable errors
+
+A usage review of `worker_events_flat.jsonl` surfaced three defects in the measurement itself, all confirmed against the live Observability API:
+
+- The query filtered only on `$metadata.type = cf-worker`, with no `service` filter, so events from `opensdmx-mcp` (another Worker on the same account) were archived here: 7 out of 31 events in a 48h probe, 589 `sdmx_*` records in the May-July history. Added `$metadata.service = ckan-mcp-server` (env override `CF_SCRIPT_NAME`); the probe then returned 24/24 ckan events. The existing records cannot be cleaned: retention is 3 days.
+- `$workers.outcome` disappeared from the API on 2026-07-12 (the object now carries `spanId`/`traceId` instead). Not a regression of ours: the archiver's only refactor is 7e78b32, March. The replacement was already there and unused: `src/worker.ts` logs its own `status` per call, plus `cache_hit`, `duration_ms` and `limit`, and the archiver was throwing all four away. `resolve_outcome()` now reads `$workers.outcome`, then `source.status`, then `$metadata.level`, and writes `unknown` rather than assuming `ok`. On a 47-event live sample it resolved every record (40 ok, 7 error, no unknown), CPU-limit crashes included. July-September stays null and cannot be backfilled: retention is 3 days.
+- `worker_daily_stats.sh` had been writing `ok:0, errors:0` on every row since that date. It now derives the error flag from `error IS NOT NULL OR outcome IN (...)`, which stays valid across the schema change. March figures are unchanged; August and September are populated again (2026-09-03: 240 calls, 220 ok, 20 errors).
+
+Flat schema gains `script_version`, `request_id`, `trigger`, `cache_hit`, `duration_ms`, `limit`. Without the first, a CPU-limit event cannot be tied to a release, and those events carry no `tool` or `server` either; they now do carry the version, so September's 22 CPU errors in 4 days (against 11 in the whole of August) become attributable from here on, not retroactively. `cache_hit` and `limit` answer the two questions this review could only guess at: cache fragmentation from URL spelling, and page size on the long datastore loops. Older records are padded to the same field set on rewrite.
+
+Usage, excluding March (evaluation traffic: 2.953 calls, 242 portals) and the sdmx events: ~470-1.450 calls a month, median 13,5 a day, very spiky. `ckan_datastore_search` leads (1.142 since June), and on 2026-08-24 a single client generated 392 calls with 6 distinct queries paging Toronto's datastore. Callers spell the same portal in three or four ways (`dati.gov.it/opendata`, `www.` variant, trailing slash; `open.canada.ca` in three forms), which fragments the cache key.
+
+### v0.4.120 - SSRF hardening release + dependency audit
+
+Releases PR #531 (env-proxy refusal, multicast/reserved/site-local ranges, see 2026-09-04 entry). Also:
+
+- `npm audit fix`: axios 1.13.2 -> 1.20.0 (29 open advisories, including NO_PROXY/SSRF bypasses), follow-redirects -> 1.16.0, wrangler -> 4.129.0, vitest -> 4.1.11. Remaining findings (body-parser, express, qs) need express 5, a major: left for a dedicated change.
+- `src/server.ts` and `src/worker.ts` were still at 0.4.118: the 0.4.119 bump skipped them because CLAUDE.md's release step lists only the three JSON files. Fixed here.
+- Verified: build, worker build, 497 tests, real HTTP e2e (search on dati.gov.it, datastore on Messina, `169.254.169.254` still refused).
+
+## 2026-09-04
+
+### SSRF guard aligned with datagouv-mcp v1.0.0
+
+Compared our `isBlockedIp` and request chain against the SSRF hardening shipped in [datagouv-mcp v1.0.0](https://github.com/datagouv/datagouv-mcp/releases/tag/v1.0.0) (PR #126, external report). Connect-time pinning, redirect re-checks and IPv4-mapped/6to4/NAT64 unwrapping were already in place; their specific vector (fetching producer-supplied URLs from catalog metadata) does not apply here. Two gaps closed:
+
+- `proxy: false` on the axios path: with `HTTP_PROXY`/`HTTPS_PROXY` set, axios connected to the proxy and the SSRF-safe lookup validated only the proxy's IP, not the target's.
+- `isBlockedIp` now also rejects 224.0.0.0/4 (multicast), 240.0.0.0/4 (reserved), ff00::/8 (IPv6 multicast) and fec0::/10 (site-local). Running their 37 test cases against our function surfaced exactly these.
+
+## 2026-08-31
+
+### Support #4682889 answered: the Advisory Database queue reaches back to June 2026
+
+Second reply from Tempy (25 August, seen on the portal - the notification was in the inbox but the question was already answered before I looked): the advisory curation team confirmed that publication of repository advisories to the Global Advisory Database **is queued**, reviewed manually, and that "the advisory publication queue stretches back to June 2026", with work under way to reduce it.
+
+So nothing is wrong with the records. The pattern we measured fits the explanation exactly: the only two advisories in the global database are the oldest ones (March and May), and CVE-2026-61612 (June) sits right at the edge of the backlog. Nothing on our side accelerates this, and there is no action left - the ticket auto-closed after seven days and is best left closed.
+
+Worth keeping: **Dependabot alerts are driven from the advisory in the database, not from the CVE assignment** (Support, 24 August). The thirteen CVEs this repository now holds are a public reference, not downstream protection - that arrives only when the advisory clears the queue.
+
+## 2026-08-24
+
+### Support #4682889: GitHub answers about CVEs, the question was about advisories
+
+First reply from GitHub Support (Tempy). It attributes the delay to an industry-wide surge in CVE publication and records being processed in order of arrival - which answers a question the ticket did not ask. The useful part is a statement that undercuts that very explanation:
+
+> The associated advisory in the GitHub Advisory Database is published independently of the CVE record, and Dependabot alerts are driven from the advisory - not the CVE assignment.
+
+If alerts are driven from the advisory, the CVE backlog cannot explain twelve published advisories being absent from the database. Replied restating the scope, with a new data point: `GHSA-x32r`, published 2026-08-20, is at its third working day and still not in the database either - so this is not confined to the older records.
+
+State unchanged: 14 published advisories, 2 in the global database, 13 carrying a CVE. Linked from the reply: "Inside the Advisory Database and what happens when vulnerability volume breaks records", which suggests the bottleneck is database curation rather than CVE issuance. If that is the answer, "queued, and the queue is long" would be enough - and is worth asking for explicitly.
+
 ## 2026-08-20
 
+### CVEs issued for the four narrowed advisories
+
+The 2026-08-15 narrowing round came back green: `GHSA-vqff` -> CVE-2026-76894, `GHSA-c499` -> CVE-2026-76895, `GHSA-3369` -> CVE-2026-76896, `GHSA-v3j5` -> CVE-2026-76897. Arguing dependency for `GHSA-c499` (one missing neutralization pass produces both the injection and the spoofing) worked as well as splitting did for the others - no advisory had to be re-published as a new one. Every published advisory now carries a CVE except `GHSA-x32r`, requested today.
+
+`SECURITY.md` updated. The propagation gap is unchanged and remains the subject of ticket #4682889: a CVE is necessary but not sufficient, and the global database still lists two of fourteen.
+
 ### v0.4.119 - classify IPv6 by value, not by prefix string (GHSA-x32r-mh7g-q2rf)
 
 `0xRomSec` reported that `isBlockedIp` misses the IPv6 ranges that embed an IPv4 address: NAT64 (`64:ff9b::/96`, `64:ff9b:1::/48`), 6to4 (`2002::/16`) and IPv4-compatible (`::/96`). Verified against the real function - all eight cases in the report return `false`. `64:ff9b::a9fe:a9fe` is 169.254.169.254 written in IPv6, and the function is the single decision point for both the literal guard and the DNS-resolution guard, so the gap applied to both halves.
@@ -12,6 +106,10 @@ The DNS half needed no change: `createSsrfSafeLookup` already passes `all: true`
 
 Tests: the eight advisory cases, four non-canonical spellings of the same address, four unparseable literals, `198.18.0.0/15` boundaries, plus three guard-level regressions that exercise the actual bypass chain (literal guard, `assertHostnameResolvesSafe`, connection-time lookup) rather than the classifier alone. 495 passing. Both build targets checked - `http.ts` is shared with the Workers bundle.
 
+Released the full round: npm 0.4.119 (CI, provenance), GitHub release with dxt/skill assets, Cloudflare deployed and verified in prod (`http://[64:ff9b::a9fe:a9fe]` rejected, normal search intact), MCP Registry 0.4.119 `isLatest=true`.
+
+API note, same class as the missing comments endpoint: **there is no `POST .../security-advisories/{ghsa}/publish`** - 404, and the OpenAPI description lists only `get`, `patch`, `cve` and `forks` for that path. Publishing an advisory is UI-only. `PATCH` (description, `patched_versions`) and `POST .../cve` do work from the API.
+
 ## 2026-08-20
 
 ### CVE-2026-76811/76812/76813 issued; advisory table added to SECURITY.md

package/dist/index.js

@@ -496,7 +496,9 @@ function formatCkanError(error, _toolName) {
   const { status, action, message } = error;
   let hint = "";
   if (status === 404) {
-    if (action.startsWith("datastore_search")) {
+    if (action === "datastore_search_sql") {
+      hint = "\u2192 This portal does not expose the SQL endpoint (optional in CKAN, often disabled). Use `ckan_datastore_search` with `filters`/`sort` instead; the resource_id is probably fine.";
+    } else if (action.startsWith("datastore_search")) {
       hint = "\u2192 Get a valid resource_id first: call `ckan_package_show` on a dataset, then pick a resource where `datastore_active` is true.";
     } else if (action === "package_show") {
       hint = "\u2192 Use `ckan_package_search` to find a valid dataset name or ID.";
@@ -704,7 +706,9 @@ function isBlockedIp(ip) {
     w[0] === 8194 || // 2002::/16 6to4
     w[0] === 8193 && w[1] === 0 || // 2001::/32 Teredo
     (w[0] & 65024) === 64512 || // fc00::/7 unique local
-    (w[0] & 65472) === 65152;
+    (w[0] & 65472) === 65152 || // fe80::/10 link-local
+    (w[0] & 65472) === 65216 || // fec0::/10 site-local (deprecated)
+    (w[0] & 65280) === 65280;
   }
   const m = v.match(/^(\d+)\.(\d+)\.(\d+)\.(\d+)$/);
   if (!m) return false;
@@ -718,7 +722,7 @@ function isBlockedIp(ip) {
   o1 === 172 && o2 >= 16 && o2 <= 31 || // 172.16.0.0/12 private
   o1 === 192 && o2 === 168 || // 192.168.0.0/16 private
   o1 === 198 && (o2 === 18 || o2 === 19) || // 198.18.0.0/15 benchmarking
-  o1 === 255;
+  o1 >= 224;
 }
 function validateServerUrl(serverUrl) {
   let parsed;
@@ -949,6 +953,9 @@ async function makeCkanRequest(serverUrl, action, params = {}, opts = {}) {
         timeout: 3e4,
         responseType: "arraybuffer",
         maxRedirects: 5,
+        // Never route through HTTP_PROXY/HTTPS_PROXY: a proxy would connect to the
+        // target itself, bypassing the SSRF-safe lookup pinned in the agents below.
+        proxy: false,
         maxContentLength: MAX_RESPONSE_BYTES,
         maxBodyLength: MAX_RESPONSE_BYTES,
         ...safeAgents ? { httpAgent: safeAgents.httpAgent, httpsAgent: safeAgents.httpsAgent } : {},
@@ -5795,7 +5802,7 @@ var registerAllPrompts = (server) => {
 function createServer() {
   return new McpServer({
     name: "ckan-mcp-server",
-    version: "0.4.118"
+    version: "0.4.121"
   });
 }
 function registerAll(server) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@aborruso/ckan-mcp-server",
-  "version": "0.4.119",
+  "version": "0.4.121",
   "mcpName": "io.github.aborruso/ckan-mcp-server",
   "description": "MCP server for interacting with CKAN open data portals",
   "repository": {