@aborruso/ckan-mcp-server 0.4.114 → 0.4.116

package/LOG.md

@@ -1,5 +1,59 @@
 # LOG
 
+## 2026-08-04
+
+### v0.4.116
+
+Ships the two DataStore output fixes below, plus `.greptile/rules.md`: the automated review on #43 produced one confident false positive, so the invariants a generic reviewer cannot know are now stated in the repo.
+
+### DataStore tables no longer hide columns silently
+
+Both markdown renderers in `src/tools/datastore.ts` cut the record table at 8 columns with no notice. On the Messina electoral-lists resource (14 columns) the table stopped right before `cognome`, `nome`, `sesso` and `voti`: a model reading it saw an election dataset with no votes and no candidate names, and nothing told it anything was missing. Row truncation was already handled properly (`... and N more records`, `Total Records`); columns were not.
+
+Both renderers now append a note naming the omitted columns and pointing at the way to retrieve them (`fields` parameter for search, an explicit SELECT list for SQL, or `response_format: "json"`). The JSON path never had the bug — `compactDatastoreResult` passes every column through.
+
+Found while checking what the GovInsider piece on the OKFN Brazil/Uruguay pilot added to `docs/future-ideas.md` (nothing new — it covers the same pilot already recorded on 2026-06-11), but its failure mode is exactly this: the model fills a gap it cannot see.
+
+### `_full_text` no longer eats a table slot
+
+Surfaced by the end-to-end check above: `datastore_search_sql` on `SELECT *` returns CKAN's internal `_full_text` column, which repeats the whole row as one concatenated string. It was taking the first table slot and pushing out a real column, and the same query reported 15 columns via SQL against 14 via `datastore_search`. `_id` was already filtered; `_full_text` now is too, in both renderers and in the JSON output (where it was pure token waste). The two tools now agree on the column count.
+
+465 tests pass; verified end to end against `dati.comune.messina.it`.
+
+## 2026-08-03
+
+### v0.4.115
+
+First release published from CI. No functional change: this exists to exercise the release workflow added earlier today end to end — tag guard, OIDC authentication, provenance attestation — rather than discovering whether it works during a release that actually matters.
+
+Worked on the first try: the job went green in 34s and the attestation binds the tarball to `ondata/ckan-mcp-server`, workflow `release.yml`, ref `refs/tags/v0.4.115`, on a GitHub-hosted runner. `npm view @aborruso/ckan-mcp-server@0.4.115 dist.attestations` returns an SLSA v1 provenance; the same query on the hand-published 0.4.114 returns nothing.
+
+One snag worth remembering: `mcp-publisher` failed with an expired JWT, and re-running `login github` then died with `incorrect_device_code` before the browser step. Neither error named the real cause — the local binary was **1.5.0 from 6 March**, and the device-auth flow changed by **1.8.0**. Updating the binary fixed it. When the registry token expires, check the publisher version too: both were installed the same day and go stale together.
+
+
+
+### npm provenance: release workflow on tags
+
+`npm view @aborruso/ckan-mcp-server@0.4.114 dist.attestations` came back empty — packages were published by hand from a local machine, with nothing binding a tarball to the commit that produced it. Adopters could check *which* version was current (see below) but not *where it came from*.
+
+- New `.github/workflows/release.yml`: triggers on `v*` tags, verifies the tag matches `package.json` **before** anything else (npm publishes are irreversible after 72h), then `npm ci` → build → tests → `npm publish --provenance`. Guard tested both ways; `npm test -- --run` confirmed against the current 461-test suite.
+- `package.json` had **no `repository` field** — a hard prerequisite for provenance, which would have failed the publish. Set to `ondata/ckan-mcp-server`.
+- Release workflow in `CLAUDE.md` rewritten rather than extended: step 5 now warns that pushing the tag *is* the publish, step 9 says explicitly not to run `npm publish` by hand (two paths would collide on `EPUBLISHCONFLICT`), and step 10 must wait for the CI run to go green because the MCP Registry validates that the npm version exists.
+- `.readme-full.md` added to `.gitignore`: the `prepack`/`postpack` pair swaps in the short npm README, so a local publish failing between the two hooks leaves the wrong `README.md` in the tree, one `git add .` away from being committed.
+- Authentication is **trusted publishing (OIDC)**, not a secret. The first draft used an `NPM_TOKEN`; `npm profile get` then surfaced npm's own warning that "tokens that bypass 2FA are being restricted for direct publishing", which pointed at the mechanism npm now recommends instead. Trusted publishing needs no stored credential, emits provenance by default, and — because the trusted publisher names the GitHub repo explicitly — also settles the open question about the npm scope (`@aborruso`) differing from the GitHub org (`ondata`). The workflow upgrades npm to ≥ 11.5.1 explicitly: Node 22 ships npm 10.x, which falls back to token auth *silently* and would publish unattested. `--provenance` is kept although implied, so that degradation fails the job instead of passing quietly.
+- **Not yet active**: the trusted publisher has to be registered by hand on npmjs.com (package → Settings → Trusted Publisher → GitHub Actions → `ondata` / `ckan-mcp-server` / `release.yml`). Until then the workflow runs and fails at the publish step. Note that the workflow filename is part of that identity.
+
+### MCP Registry entry realigned to 0.4.114
+
+Published and verified against the public endpoint: the registry now serves **0.4.114** with `isLatest: true` (updated 2026-08-03T06:01:07Z), and the old 0.4.83 record dropped to `isLatest: false`.
+
+The official MCP Registry entry for `io.github.aborruso/ckan-mcp-server` had been stuck at **0.4.83 since 2026-03-12** — 31 patch releases and almost five months behind npm, while still flagged `isLatest: true`. Clients resolving the server through the registry were pointed at a build predating the v0.4.108 SSRF remediation, and 0.4.83 is still installable from npm.
+
+- Root cause, not a one-off slip: the Release Workflow in `CLAUDE.md` listed the version bump for `package.json` and `manifest.json` but never `server.json`, and `npm publish` does not touch the registry. The drift was structural and would have kept growing.
+- Fixed `server.json` (both `version` and `packages[0].version` — two fields, easy to half-update) and rewrote the release workflow: `server.json` added to step 1, a new step 10 for `mcp-publisher publish` placed after `npm publish` since the registry validates that the npm version exists, plus a `curl` one-liner to verify the published entry.
+- Surfaced by an unsolicited vendor email selling a £395 "MCP Readiness Audit". The sales pitch was worthless — the remedy it offered has nothing to do with the defect — but the three technical claims all checked out under verification. Worth recording: the finding was real and cost the sender two `curl` calls, which is exactly how long it would have taken us to catch it ourselves with a check in the release procedure.
+- Registry tokens (`.mcpregistry_*`) verified: gitignored, never committed.
+
 ## 2026-07-31
 
 ### v0.4.114
@@ -11,7 +65,13 @@ The v0.4.113 cap applied to `content[].text` only, so any client reading the str
 - New `jsonToolResult()`: truncates once via `truncateJson` and parses the result back into `structuredContent`, so the two channels cannot disagree and `_truncated`/`_original_count` reach structured readers too. Applied to 13 call sites.
 - `ckan_status_show` and `ckan_find_portals` pair structured output with Markdown text, so there is no truncated JSON to derive it from: they cap it on its own via the new `cappedStructured()`. My first pass left them uncapped calling them "bounded by shape" — wrong, as review pointed out: `status_show` is echoed straight from the portal, and the 50-result limit on `find_portals` bounds the number of entries, not the length of their titles and URLs. Only the `all_fields=false` count branches of `ckan_organization_list`/`ckan_group_list` stay outside the cap: a single integer.
 - **Correction**: the deferral in v0.4.113 claimed capping would drop rows from `datastore-table-ui`. That was wrong and never verified — the UI resource is commented out in `src/resources/index.ts:18` and never registered, and `ckan_datastore_search` returns no `structuredContent` at all. No exception was needed. Same failure mode as the bug being fixed: a plausible claim about the code that nobody checked.
-- 4 new tests (458 total). Docs realigned: `README.md`, `docs/JSON-OUTPUT.md`, `docs/DECISIONS.md`, `CLAUDE.md` all asserted the uncapped behaviour.
+- 7 new tests (461 total), covering a payload long in a single field rather than in many, and a bounded-count list whose entries are individually oversized. Docs realigned: `README.md`, `docs/JSON-OUTPUT.md`, `docs/DECISIONS.md`, `CLAUDE.md` all asserted the uncapped behaviour.
+- Released end to end: tag, GitHub release with DXT and skill, npm `0.4.114`, Cloudflare version `36b611fd`. Verified on the public endpoint after waiting for edge propagation — `tag_list limit=1000`: 49,938 characters on both channels, byte-identical, `_truncated` set. `status_show` and `find_portals` legitimately differ across channels: Markdown text against JSON structured output, two renderings of the same data.
+- Process note: check edge propagation with an active probe, not a fixed wait. The v0.4.113 verification measured the old code because it ran seconds after deploy, and briefly looked like a broken release.
+
+### v0.4.113 — release addendum
+
+Released the same day: tag, GitHub release with DXT and skill, npm `0.4.113`, Cloudflare version `1cac40ea`. Also merged #38 (`docs: document user-facing limits`, contributed by `averyquinnhq`), which closed #37, and disabled the `claude-code-review` workflow: it cannot pass on fork PRs, since GitHub withholds secrets and the OIDC token for `pull_request` events from forks, so the job dies on token setup regardless of the diff. The file is kept in the repo. `CONTRIBUTING.md` now also states that `build:tsc` is not a gate — a contributor exhausted 8 GB of heap on it before reporting they could not pass it, because the warning lived only in `CLAUDE.md`.
 
 ### v0.4.113
 

package/dist/index.js

@@ -2936,6 +2936,20 @@ Typical workflow: ckan_organization_search \u2192 ckan_organization_show (get de
 
 // src/tools/datastore.ts
 import { z as z4 } from "zod";
+var MAX_TABLE_COLUMNS = 8;
+var INTERNAL_FIELDS = ["_id", "_full_text"];
+var isInternalField = (id) => INTERNAL_FIELDS.includes(id);
+var MAX_LISTED_OMITTED_COLUMNS = 15;
+var sanitizeFieldName = (id) => id.replace(/[\r\n]+/g, " ").replace(/\|/g, "\\|");
+function formatOmittedColumnsNote(allFields, hint) {
+  if (allFields.length <= MAX_TABLE_COLUMNS) return "";
+  const omitted = allFields.slice(MAX_TABLE_COLUMNS);
+  const listed = omitted.slice(0, MAX_LISTED_OMITTED_COLUMNS).map(sanitizeFieldName).join(", ");
+  const rest = omitted.length > MAX_LISTED_OMITTED_COLUMNS ? ` and ${omitted.length - MAX_LISTED_OMITTED_COLUMNS} more` : "";
+  return `
+> **Note**: the table above shows only the first ${MAX_TABLE_COLUMNS} of ${allFields.length} columns. Columns not shown: ${listed}${rest}. They do hold values \u2014 ${hint}
+`;
+}
 function formatDatastoreSearchMarkdown(result, serverUrl, resourceId, offset, limit) {
   let markdown = `# DataStore Query Results
 
@@ -2949,18 +2963,19 @@ function formatDatastoreSearchMarkdown(result, serverUrl, resourceId, offset, li
   markdown += `**Returned**: ${result.records ? result.records.length : 0} records
 
 `;
-  if (result.fields && result.fields.length > 0) {
+  const declaredFields = (result.fields || []).filter((f) => f.id !== "_full_text");
+  if (declaredFields.length > 0) {
     markdown += `## Fields
 
 `;
-    markdown += result.fields.map((f) => `- **${f.id}** (${f.type})`).join("\n") + "\n\n";
+    markdown += declaredFields.map((f) => `- **${f.id}** (${f.type})`).join("\n") + "\n\n";
   }
   if (result.records && result.records.length > 0) {
     markdown += `## Records
 
 `;
-    const fields = result.fields ? result.fields.map((f) => f.id).filter((id) => id !== "_id") : [];
-    const displayFields = fields.slice(0, 8);
+    const fields = result.fields ? result.fields.map((f) => f.id).filter((id) => !isInternalField(id)) : [];
+    const displayFields = fields.slice(0, MAX_TABLE_COLUMNS);
     markdown += `| ${displayFields.join(" | ")} |
 `;
     markdown += `| ${displayFields.map(() => "---").join(" | ")} |
@@ -2975,6 +2990,7 @@ function formatDatastoreSearchMarkdown(result, serverUrl, resourceId, offset, li
       markdown += `| ${values.join(" | ")} |
 `;
     }
+    markdown += formatOmittedColumnsNote(fields, 'use the `fields` parameter to select them, or `response_format: "json"` to get every column.');
     if (result.records.length > 50) {
       markdown += `
 ... and ${result.records.length - 50} more records
@@ -2994,7 +3010,7 @@ function formatDatastoreSearchMarkdown(result, serverUrl, resourceId, offset, li
 }
 function formatDatastoreSqlMarkdown(result, serverUrl, sql) {
   const records = result.records || [];
-  const fieldIds = (result.fields?.map((field) => field.id) || Object.keys(records[0] || {})).filter((id) => id !== "_id");
+  const fieldIds = (result.fields?.map((field) => field.id) || Object.keys(records[0] || {})).filter((id) => !isInternalField(id));
   let markdown = `# DataStore SQL Results
 
 `;
@@ -3005,17 +3021,18 @@ function formatDatastoreSqlMarkdown(result, serverUrl, sql) {
   markdown += `**Returned**: ${records.length} records
 
 `;
-  if (result.fields && result.fields.length > 0) {
+  const declaredFields = (result.fields || []).filter((field) => field.id !== "_full_text");
+  if (declaredFields.length > 0) {
     markdown += `## Fields
 
 `;
-    markdown += result.fields.map((field) => `- **${field.id}** (${field.type})`).join("\n") + "\n\n";
+    markdown += declaredFields.map((field) => `- **${field.id}** (${field.type})`).join("\n") + "\n\n";
   }
   if (records.length > 0 && fieldIds.length > 0) {
     markdown += `## Records
 
 `;
-    const displayFields = fieldIds.slice(0, 8);
+    const displayFields = fieldIds.slice(0, MAX_TABLE_COLUMNS);
     markdown += `| ${displayFields.join(" | ")} |
 `;
     markdown += `| ${displayFields.map(() => "---").join(" | ")} |
@@ -3030,6 +3047,7 @@ function formatDatastoreSqlMarkdown(result, serverUrl, sql) {
       markdown += `| ${values.join(" | ")} |
 `;
     }
+    markdown += formatOmittedColumnsNote(fieldIds, 'name them explicitly in the SELECT clause, or use `response_format: "json"` to get every column.');
     if (records.length > 50) {
       markdown += `
 ... and ${records.length - 50} more records
@@ -3043,11 +3061,10 @@ function formatDatastoreSqlMarkdown(result, serverUrl, sql) {
   return markdown;
 }
 function compactDatastoreResult(result) {
-  const fields = (result.fields || []).filter((f) => f.id !== "_id");
-  const records = (result.records || []).map((record) => {
-    const { _id, ...rest } = record;
-    return rest;
-  });
+  const fields = (result.fields || []).filter((f) => !isInternalField(f.id));
+  const records = (result.records || []).map(
+    (record) => Object.fromEntries(Object.entries(record).filter(([key]) => !isInternalField(key)))
+  );
   return {
     resource_id: result.resource_id || null,
     fields,

package/package.json

@@ -1,8 +1,12 @@
 {
   "name": "@aborruso/ckan-mcp-server",
-  "version": "0.4.114",
+  "version": "0.4.116",
   "mcpName": "io.github.aborruso/ckan-mcp-server",
   "description": "MCP server for interacting with CKAN open data portals",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/ondata/ckan-mcp-server.git"
+  },
   "main": "dist/index.js",
   "type": "module",
   "bin": {