@aborruso/ckan-mcp-server 0.4.115 → 0.4.116

package/LOG.md

@@ -1,11 +1,35 @@
 # LOG
 
+## 2026-08-04
+
+### v0.4.116
+
+Ships the two DataStore output fixes below, plus `.greptile/rules.md`: the automated review on #43 produced one confident false positive, so the invariants a generic reviewer cannot know are now stated in the repo.
+
+### DataStore tables no longer hide columns silently
+
+Both markdown renderers in `src/tools/datastore.ts` cut the record table at 8 columns with no notice. On the Messina electoral-lists resource (14 columns) the table stopped right before `cognome`, `nome`, `sesso` and `voti`: a model reading it saw an election dataset with no votes and no candidate names, and nothing told it anything was missing. Row truncation was already handled properly (`... and N more records`, `Total Records`); columns were not.
+
+Both renderers now append a note naming the omitted columns and pointing at the way to retrieve them (`fields` parameter for search, an explicit SELECT list for SQL, or `response_format: "json"`). The JSON path never had the bug — `compactDatastoreResult` passes every column through.
+
+Found while checking what the GovInsider piece on the OKFN Brazil/Uruguay pilot added to `docs/future-ideas.md` (nothing new — it covers the same pilot already recorded on 2026-06-11), but its failure mode is exactly this: the model fills a gap it cannot see.
+
+### `_full_text` no longer eats a table slot
+
+Surfaced by the end-to-end check above: `datastore_search_sql` on `SELECT *` returns CKAN's internal `_full_text` column, which repeats the whole row as one concatenated string. It was taking the first table slot and pushing out a real column, and the same query reported 15 columns via SQL against 14 via `datastore_search`. `_id` was already filtered; `_full_text` now is too, in both renderers and in the JSON output (where it was pure token waste). The two tools now agree on the column count.
+
+465 tests pass; verified end to end against `dati.comune.messina.it`.
+
 ## 2026-08-03
 
 ### v0.4.115
 
 First release published from CI. No functional change: this exists to exercise the release workflow added earlier today end to end — tag guard, OIDC authentication, provenance attestation — rather than discovering whether it works during a release that actually matters.
 
+Worked on the first try: the job went green in 34s and the attestation binds the tarball to `ondata/ckan-mcp-server`, workflow `release.yml`, ref `refs/tags/v0.4.115`, on a GitHub-hosted runner. `npm view @aborruso/ckan-mcp-server@0.4.115 dist.attestations` returns an SLSA v1 provenance; the same query on the hand-published 0.4.114 returns nothing.
+
+One snag worth remembering: `mcp-publisher` failed with an expired JWT, and re-running `login github` then died with `incorrect_device_code` before the browser step. Neither error named the real cause — the local binary was **1.5.0 from 6 March**, and the device-auth flow changed by **1.8.0**. Updating the binary fixed it. When the registry token expires, check the publisher version too: both were installed the same day and go stale together.
+
 
 
 ### npm provenance: release workflow on tags

package/dist/index.js

@@ -2936,6 +2936,20 @@ Typical workflow: ckan_organization_search \u2192 ckan_organization_show (get de
 
 // src/tools/datastore.ts
 import { z as z4 } from "zod";
+var MAX_TABLE_COLUMNS = 8;
+var INTERNAL_FIELDS = ["_id", "_full_text"];
+var isInternalField = (id) => INTERNAL_FIELDS.includes(id);
+var MAX_LISTED_OMITTED_COLUMNS = 15;
+var sanitizeFieldName = (id) => id.replace(/[\r\n]+/g, " ").replace(/\|/g, "\\|");
+function formatOmittedColumnsNote(allFields, hint) {
+  if (allFields.length <= MAX_TABLE_COLUMNS) return "";
+  const omitted = allFields.slice(MAX_TABLE_COLUMNS);
+  const listed = omitted.slice(0, MAX_LISTED_OMITTED_COLUMNS).map(sanitizeFieldName).join(", ");
+  const rest = omitted.length > MAX_LISTED_OMITTED_COLUMNS ? ` and ${omitted.length - MAX_LISTED_OMITTED_COLUMNS} more` : "";
+  return `
+> **Note**: the table above shows only the first ${MAX_TABLE_COLUMNS} of ${allFields.length} columns. Columns not shown: ${listed}${rest}. They do hold values \u2014 ${hint}
+`;
+}
 function formatDatastoreSearchMarkdown(result, serverUrl, resourceId, offset, limit) {
   let markdown = `# DataStore Query Results
 
@@ -2949,18 +2963,19 @@ function formatDatastoreSearchMarkdown(result, serverUrl, resourceId, offset, li
   markdown += `**Returned**: ${result.records ? result.records.length : 0} records
 
 `;
-  if (result.fields && result.fields.length > 0) {
+  const declaredFields = (result.fields || []).filter((f) => f.id !== "_full_text");
+  if (declaredFields.length > 0) {
     markdown += `## Fields
 
 `;
-    markdown += result.fields.map((f) => `- **${f.id}** (${f.type})`).join("\n") + "\n\n";
+    markdown += declaredFields.map((f) => `- **${f.id}** (${f.type})`).join("\n") + "\n\n";
   }
   if (result.records && result.records.length > 0) {
     markdown += `## Records
 
 `;
-    const fields = result.fields ? result.fields.map((f) => f.id).filter((id) => id !== "_id") : [];
-    const displayFields = fields.slice(0, 8);
+    const fields = result.fields ? result.fields.map((f) => f.id).filter((id) => !isInternalField(id)) : [];
+    const displayFields = fields.slice(0, MAX_TABLE_COLUMNS);
     markdown += `| ${displayFields.join(" | ")} |
 `;
     markdown += `| ${displayFields.map(() => "---").join(" | ")} |
@@ -2975,6 +2990,7 @@ function formatDatastoreSearchMarkdown(result, serverUrl, resourceId, offset, li
       markdown += `| ${values.join(" | ")} |
 `;
     }
+    markdown += formatOmittedColumnsNote(fields, 'use the `fields` parameter to select them, or `response_format: "json"` to get every column.');
     if (result.records.length > 50) {
       markdown += `
 ... and ${result.records.length - 50} more records
@@ -2994,7 +3010,7 @@ function formatDatastoreSearchMarkdown(result, serverUrl, resourceId, offset, li
 }
 function formatDatastoreSqlMarkdown(result, serverUrl, sql) {
   const records = result.records || [];
-  const fieldIds = (result.fields?.map((field) => field.id) || Object.keys(records[0] || {})).filter((id) => id !== "_id");
+  const fieldIds = (result.fields?.map((field) => field.id) || Object.keys(records[0] || {})).filter((id) => !isInternalField(id));
   let markdown = `# DataStore SQL Results
 
 `;
@@ -3005,17 +3021,18 @@ function formatDatastoreSqlMarkdown(result, serverUrl, sql) {
   markdown += `**Returned**: ${records.length} records
 
 `;
-  if (result.fields && result.fields.length > 0) {
+  const declaredFields = (result.fields || []).filter((field) => field.id !== "_full_text");
+  if (declaredFields.length > 0) {
     markdown += `## Fields
 
 `;
-    markdown += result.fields.map((field) => `- **${field.id}** (${field.type})`).join("\n") + "\n\n";
+    markdown += declaredFields.map((field) => `- **${field.id}** (${field.type})`).join("\n") + "\n\n";
   }
   if (records.length > 0 && fieldIds.length > 0) {
     markdown += `## Records
 
 `;
-    const displayFields = fieldIds.slice(0, 8);
+    const displayFields = fieldIds.slice(0, MAX_TABLE_COLUMNS);
     markdown += `| ${displayFields.join(" | ")} |
 `;
     markdown += `| ${displayFields.map(() => "---").join(" | ")} |
@@ -3030,6 +3047,7 @@ function formatDatastoreSqlMarkdown(result, serverUrl, sql) {
       markdown += `| ${values.join(" | ")} |
 `;
     }
+    markdown += formatOmittedColumnsNote(fieldIds, 'name them explicitly in the SELECT clause, or use `response_format: "json"` to get every column.');
     if (records.length > 50) {
       markdown += `
 ... and ${records.length - 50} more records
@@ -3043,11 +3061,10 @@ function formatDatastoreSqlMarkdown(result, serverUrl, sql) {
   return markdown;
 }
 function compactDatastoreResult(result) {
-  const fields = (result.fields || []).filter((f) => f.id !== "_id");
-  const records = (result.records || []).map((record) => {
-    const { _id, ...rest } = record;
-    return rest;
-  });
+  const fields = (result.fields || []).filter((f) => !isInternalField(f.id));
+  const records = (result.records || []).map(
+    (record) => Object.fromEntries(Object.entries(record).filter(([key]) => !isInternalField(key)))
+  );
   return {
     resource_id: result.resource_id || null,
     fields,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@aborruso/ckan-mcp-server",
-  "version": "0.4.115",
+  "version": "0.4.116",
   "mcpName": "io.github.aborruso/ckan-mcp-server",
   "description": "MCP server for interacting with CKAN open data portals",
   "repository": {