@hedeqiang/dsh-nl2sql 0.12.1 → 0.13.0

package/CHANGELOG.md

@@ -5,6 +5,42 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Unreleased]
 
+## [0.13.0] - 2026-11-12
+
+### Added
+- **Data health:** per-table freshness (latest timestamp, lag, thresholds), the `nl2sql_freshness` tool and stale-data warnings under results and in the schema view, notifications when a table stops updating and when it recovers; a daily data-quality patrol (null rates, duplicate keys, outliers, future timestamps, negative amounts) with a health score and a push when it drops.
+- **Reconciliation:** the `nl2sql_reconcile` tool and saved daily jobs that compare two queries on two data sources by key (tolerances, normalisation), with match rate, rows missing on either side, value differences, run history and pushes with the full differences as Excel.
+- **Chat subscriptions:** `nl2sql_subscribe`, `nl2sql_alert_create` and `nl2sql_subscriptions` create, list, pause and cancel scheduled results and alerts delivered to the chat they were created in, running with the creator's permissions at delivery time.
+- **Data finds people:** daily digests of notable metric changes with their main contributors, automatic attribution in alert messages, metric goals with progress and projection on dashboards, and time-series forecasts (`nl2sql_forecast`, result menu).
+- **Presentation:** PowerPoint and long-image export for dashboards and results, sending the long image to Lark / Telegram, presentation mode, and scheduled sync of results to Lark Bitable.
+- **MCP server:** a local Model Context Protocol endpoint with per-client tokens mapped to owner, member or role scopes, plus the `nl2sql-mcp` stdio bridge for Claude Desktop ([docs/mcp.md](docs/mcp.md)).
+- The **Manage** navigation groups sections into core, *Insight & automation* and *Integrations*.
+
+### Changed
+- Internal: feature modules plug in through `service/features.js` (RPCs, tools, scheduler jobs) and client extension points (`extend.*`); `runQuery` accepts a per-call `maxRows` up to 200,000.
+
+These features are tested with local databases and fake IM / Lark servers; real Lark, Telegram, WeCom, DingTalk and MCP clients have not been exercised yet.
+
+## [0.12.2] - 2026-11-12
+
+### Security
+- Row-level rules could be bypassed with MySQL `--` arithmetic (e.g. `WHERE 1--1) OR (1`): the filter injector now uses the dialect-aware lexer and refuses any rewrite where the injected rule is not a top-level conjunct.
+- The SQL guard followed JavaScript's notion of whitespace after `--`; it now follows MySQL exactly, and SQL containing control characters or non-ASCII whitespace outside literals is refused (closed table-policy and system-catalog bypasses).
+- `nl2sql_overview` ran its queries without the caller's row-level rules; it now runs as the caller.
+- A result filtered for one member could be reused (re-charted, drilled, sent) by a member with different row-level rules; results now carry a signature of the rules applied.
+- The bot's data source catalog listed tables hidden by the access policy.
+- Data values could inject mentions (`<at id=all>`) or links into Lark cards and other IM messages; data-derived text is now escaped per platform.
+
+### Fixed
+- Previewing an alert while another alert was saved could delete saved alerts; an alert edited during a check lost its state and re-fired every minute.
+- Bot commands matched any question starting with words such as 自动, 数据源, 切换 or help; commands must now be the whole message. English aliases: `help`, `sources`, `switch <name>`, `auto`, `new`.
+- Bots no longer change the page's current data source when they switch sources.
+- Deleting a channel left alerts, the schema-change notice, scheduled reports and bot conversations pointing at it.
+- Sending failed for titles longer than 100 characters; NULL numbers showed as 0; line breaks inside cells broke tables; changing a channel's type asked for unchanged secrets again; Excel sheet names with emoji could exceed Excel's limit; daily alerts handled daylight-saving transitions wrongly.
+- New file data sources had the description "上传的文件：undefined".
+- Page: the knowledge editor could save into another data source after switching; the Ask page briefly offered the previous data source's questions; dialogs and toasts did not appear inside conversation tool views; free-text dashboard filters could not take a comma; double clicks created duplicate widgets, alerts and knowledge items; unsaved settings were lost without warning; several actions failed silently; number formatting showed 0 for empty values and `10000万` / `1000K`; history labels for Telegram / WeCom / DingTalk bots; dates follow the UI language.
+- Accessibility: focus stays inside dialogs; policy rows work with the keyboard; Chinese punctuation no longer appears in the English UI.
+
 ## [0.12.1] - 2026-11-12
 
 ### Fixed
@@ -86,7 +122,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 - Redesigned UI; Lark push (Markdown and native charts); access control, feedback loop, dashboard global filters; PostgreSQL / Elasticsearch, SSH tunnels, privacy settings, automatic overviews, scheduled reports; semantic layer, ER graph, scan guard, history and audit, dashboards; first version.
 
-[Unreleased]: https://github.com/hedeqiang/dsh-nl2sql/compare/v0.12.1...HEAD
+[Unreleased]: https://github.com/hedeqiang/dsh-nl2sql/compare/v0.13.0...HEAD
+[0.13.0]: https://github.com/hedeqiang/dsh-nl2sql/compare/v0.12.2...v0.13.0
+[0.12.2]: https://github.com/hedeqiang/dsh-nl2sql/compare/v0.12.1...v0.12.2
 [0.12.1]: https://github.com/hedeqiang/dsh-nl2sql/compare/v0.12.0...v0.12.1
 [0.12.0]: https://github.com/hedeqiang/dsh-nl2sql/compare/v0.11.2...v0.12.0
 [0.11.2]: https://github.com/hedeqiang/dsh-nl2sql/compare/v0.11.1...v0.11.2

package/README.md

@@ -24,6 +24,7 @@ English · [简体中文](README.zh-CN.md)
 - [Installation](#installation)
 - [Quick start](#quick-start)
 - [IM channels and bots](#im-channels-and-bots)
+- [MCP server](#mcp-server)
 - [Security model](#security-model)
 - [Data and privacy](#data-and-privacy)
 - [Limitations](#limitations)
@@ -47,6 +48,17 @@ English · [简体中文](README.zh-CN.md)
 - Push to Lark / Feishu, Telegram, WeCom and DingTalk; charts are sent as native card charts, images or tables depending on the platform, and large results come with the full data as an Excel attachment where the platform allows files.
 - Chat bots that answer @-mentions in group chats without a public URL (Lark long connection, Telegram long polling, WeCom AI bot long connection, DingTalk Stream mode).
 
+**Data finds people**
+- Data freshness: the latest timestamp of every table, stale-data warnings in answers and alerts when a table stops updating; a daily data-quality patrol with a health score.
+- Cross-source reconciliation: compare two queries on two data sources by key, list rows missing on either side and value differences; save as a daily job that pushes differences with an Excel attachment.
+- Chat subscriptions: "@bot send this to the group every day at 9" or "tell me when withdrawals exceed 500k" creates a subscription or alert delivered to that chat.
+- Daily digest with the notable changes and their main contributors, automatic attribution in alert messages, metric goals with progress and projection, and time-series forecasts.
+
+**Present and connect**
+- Export a dashboard or results to PowerPoint or one long image (also sendable to Lark / Telegram); presentation mode for wall screens with auto refresh and rotation.
+- Sync query results to Lark Bitable on a schedule.
+- MCP server: Claude Desktop, Cursor, VS Code and other MCP clients can use the same governed access (see [docs/mcp.md](docs/mcp.md)).
+
 **Govern**
 - Read-only by construction, table / column access policies with masking, and a full-scan guard.
 - Members with roles (admin / analyst / viewer), data source scopes and row-level rules for people and group chats on every IM platform.
@@ -126,6 +138,16 @@ In a group chat, members @-mention the bot and ask. The bot knows which data sou
 
 > The Telegram, WeCom and DingTalk adapters are tested against local fakes of their APIs and have not yet been verified with real accounts; Lark has been used with real workspaces. Reports from real deployments are welcome.
 
+## MCP server
+
+Other AI tools — Claude Desktop, Cursor, Cline, VS Code — can query your data sources through the [Model Context Protocol](https://modelcontextprotocol.io/), with the same read-only admission, access policies, member scopes, row-level rules, privacy setting and audit. Enable it under **Admin → MCP access** (off by default; listens on `127.0.0.1:47860` only), create a scoped token (shown once), and paste the generated snippet into your client. HTTP clients use `http://127.0.0.1:47860/mcp` with `Authorization: Bearer <token>`; stdio clients such as Claude Desktop use the bundled bridge:
+
+```bash
+NL2SQL_MCP_TOKEN=<token> npx -y -p @hedeqiang/dsh-nl2sql nl2sql-mcp
+```
+
+Tools: `list_datasources`, `get_schema`, `query`, `query_metric`, `search_knowledge`, `drill`, `explain_change` (plus `reconcile` / `forecast` when available); resources: the schema of every visible data source. See [docs/mcp.md](docs/mcp.md).
+
 ## Security model
 
 - **Read-only queries.** Every statement passes a per-dialect lexer (MySQL, PostgreSQL, SQLite, Elasticsearch SQL) that admits only `SELECT` / `SHOW` / `DESCRIBE` / `EXPLAIN`; queries run in read-only transactions with server-side timeouts, PostgreSQL uses the extended protocol (no multi-statement strings), and large results are streamed and truncated.

package/README.zh-CN.md

@@ -24,6 +24,7 @@ Agent 编写只读 SQL、执行并在对话中画图。<br>
 - [安装](#安装)
 - [快速开始](#快速开始)
 - [IM 渠道与机器人](#im-渠道与机器人)
+- [MCP 服务](#mcp-服务)
 - [安全模型](#安全模型)
 - [数据与隐私](#数据与隐私)
 - [已知限制](#已知限制)
@@ -47,6 +48,17 @@ Agent 编写只读 SQL、执行并在对话中画图。<br>
 - 推送到飞书、Telegram、企业微信、钉钉：图表按平台能力用原生卡片图表、图片或表格发送；结果过大时，在能发文件的平台附带完整数据 Excel。
 - 群聊机器人回答 @提问，不需要公网地址（飞书长连接、Telegram 长轮询、企业微信智能机器人长连接、钉钉 Stream 模式）。
 
+**让数据找人**
+- 数据新鲜度：记录每张表的最新数据时间，回答近期数据时提示是否滞后，数据断更时告警；每日数据质量巡检并给出健康分。
+- 跨数据源对账：按主键对比两个数据源的两条查询，列出单边缺失和数值差异；可保存为每日任务，有差异时推送并附 Excel。
+- 群里订阅：「@机器人 每天 9 点把这个发到群里」「提现超过 50 万提醒我」直接创建订阅或告警，推送到当前群。
+- 每日数据简报（异常变化及主要来源）、告警消息自动归因、指标目标进度与达成预测、时间序列预测。
+
+**展示与集成**
+- 看板或结果导出 PPT、长图（可发送到飞书 / Telegram）；大屏模式（全屏、自动刷新、轮播）。
+- 定时把查询结果同步到飞书多维表格。
+- MCP 服务：Claude Desktop、Cursor、VS Code 等 MCP 客户端可以使用同样受控的数据访问（见 [docs/mcp.md](docs/mcp.md)）。
+
 **治理**
 - 只读执行；表 / 字段级访问控制与脱敏；大表全表扫描保护。
 - 成员：各 IM 平台的个人和群都可以设置角色（管理员 / 分析师 / 只读）、数据源范围和行级权限。
@@ -126,6 +138,16 @@ dsh plugin --profile desktop remove @hedeqiang/dsh-nl2sql
 
 > Telegram、企业微信、钉钉适配器目前只在本地模拟的平台接口上测试过，尚未在真实账号上验证；飞书已在真实环境中使用。欢迎反馈实际部署情况。
 
+## MCP 服务
+
+Claude Desktop、Cursor、Cline、VS Code 等 AI 工具可以通过 [Model Context Protocol](https://modelcontextprotocol.io/) 查询这里的数据源，只读准入、访问控制、成员范围、行级权限、隐私设置与审计同样生效。在 **管理 → MCP 接入** 中启用（默认关闭，只监听 `127.0.0.1:47860`），创建带权限范围的令牌（只显示一次），把页面生成的配置粘贴到客户端即可。HTTP 客户端使用 `http://127.0.0.1:47860/mcp` 并带上 `Authorization: Bearer <令牌>`；Claude Desktop 等 stdio 客户端使用内置的桥接命令：
+
+```bash
+NL2SQL_MCP_TOKEN=<令牌> npx -y -p @hedeqiang/dsh-nl2sql nl2sql-mcp
+```
+
+工具：`list_datasources`、`get_schema`、`query`、`query_metric`、`search_knowledge`、`drill`、`explain_change`（安装了对账 / 预测功能时还有 `reconcile` / `forecast`）；资源：每个可见数据源的表结构。详见 [docs/mcp.md](docs/mcp.md)（英文）。
+
 ## 安全模型
 
 - **只读查询。** 每条语句都经过按方言的词法检查（MySQL、PostgreSQL、SQLite、Elasticsearch SQL），只允许 `SELECT` / `SHOW` / `DESCRIBE` / `EXPLAIN`；查询在只读事务中执行并有服务端超时；PostgreSQL 使用扩展协议（不接受多语句）；大结果流式读取并截断。

package/analysis.js

@@ -9,6 +9,7 @@
  */
 
 import { timeBounds, TIME_RANGES } from './filters.js'
+import { dashDashCommentFollower } from './sql-guard.js'
 
 // ------------------------------------------------------------ quoting
 
@@ -83,7 +84,9 @@ function tokenize(sql, { dialect } = {}) {
     const c = sql[i]
     const n = sql[i + 1]
     if (/\s/.test(c)) { i++; continue }
-    if ((c === '-' && n === '-') || (c === '#' && dialect !== 'postgres')) { while (i < sql.length && sql[i] !== '\n') i++; continue }
+    // MySQL: `--` starts a comment only when followed by a control/space character (or end); other dialects: always.
+    const dashComment = c === '-' && n === '-' && (dialect === 'postgres' || dialect === 'sqlite' || dialect === 'elasticsearch' || dashDashCommentFollower(sql[i + 2]))
+    if (dashComment || (c === '#' && dialect !== 'postgres' && dialect !== 'sqlite' && dialect !== 'elasticsearch')) { while (i < sql.length && sql[i] !== '\n') i++; continue }
     if (c === '/' && n === '*') { const e = sql.indexOf('*/', i + 2); i = e < 0 ? sql.length : e + 2; continue }
     if (c === '\'' || c === '"' || c === '`') {
       const start = i

package/bin/nl2sql-mcp.mjs

@@ -0,0 +1,103 @@
+#!/usr/bin/env node
+/**
+ * nl2sql-mcp — stdio bridge to the DeepSeek Harness Data Q&A MCP server.
+ *
+ * Speaks MCP over stdio (newline-delimited JSON-RPC 2.0) and forwards every
+ * message to the plugin's Streamable HTTP endpoint, so stdio-only clients
+ * (e.g. Claude Desktop) can use it. Plain ESM, Node ≥ 18, no dependencies.
+ *
+ *   NL2SQL_MCP_URL    default http://127.0.0.1:47860/mcp
+ *   NL2SQL_MCP_TOKEN  bearer token created in Data Q&A → MCP access (required)
+ *
+ *   npx -y -p @hedeqiang/dsh-nl2sql nl2sql-mcp
+ */
+const URL_ = process.env.NL2SQL_MCP_URL || 'http://127.0.0.1:47860/mcp'
+const TOKEN = process.env.NL2SQL_MCP_TOKEN || ''
+const TIMEOUT_MS = Number(process.env.NL2SQL_MCP_TIMEOUT_MS) || 11 * 60 * 1000
+
+const log = (...a) => process.stderr.write(`[nl2sql-mcp] ${a.join(' ')}\n`)
+const write = (obj) => process.stdout.write(`${JSON.stringify(obj)}\n`)
+const errorReply = (id, code, message, data) => write({ jsonrpc: '2.0', id: id ?? null, error: { code, message, ...(data !== undefined ? { data } : {}) } })
+
+if (typeof fetch !== 'function') { log('Node.js 18 or newer is required (global fetch).'); process.exit(1) }
+if (!TOKEN) log('NL2SQL_MCP_TOKEN is not set; requests will be rejected (401). Create a token in DeepSeek Harness → Data Q&A → MCP access.')
+
+let sessionId
+let protocolVersion
+let pending = 0
+let ended = false
+
+/** Header-safe value (Base64 sentinel for non-ASCII), per the Streamable HTTP binding. */
+function headerValue(v) {
+  const s = String(v)
+  return /^[\x20-\x7e]*$/.test(s) && s.trim() === s && !/^=\?base64\?.*\?=$/.test(s) ? s : `=?base64?${Buffer.from(s, 'utf8').toString('base64')}?=`
+}
+
+async function forward(msg) {
+  const isRequest = msg && typeof msg === 'object' && 'id' in msg && typeof msg.method === 'string'
+  const headers = { 'content-type': 'application/json', accept: 'application/json, text/event-stream' }
+  if (TOKEN) headers.authorization = `Bearer ${TOKEN}`
+  if (sessionId) headers['mcp-session-id'] = sessionId
+  const metaVersion = msg?.params?._meta?.['io.modelcontextprotocol/protocolVersion']
+  const version = metaVersion || protocolVersion
+  if (version && msg?.method !== 'initialize') headers['mcp-protocol-version'] = version
+  if (typeof msg?.method === 'string') headers['mcp-method'] = msg.method
+  const name = msg?.params?.name ?? msg?.params?.uri
+  if (name !== undefined && metaVersion) headers['mcp-name'] = headerValue(name)
+  let res
+  try {
+    res = await fetch(URL_, { method: 'POST', headers, body: JSON.stringify(msg), signal: AbortSignal.timeout(TIMEOUT_MS) })
+  } catch (e) {
+    if (isRequest) errorReply(msg.id, -32000, `Cannot reach the nl2sql MCP server at ${URL_} (${e.cause?.code ?? e.message}). Enable it in DeepSeek Harness → Data Q&A → MCP access and keep DeepSeek Harness running.`)
+    return
+  }
+  const sid = res.headers.get('mcp-session-id')
+  if (sid) sessionId = sid
+  const type = res.headers.get('content-type') ?? ''
+  const text = await res.text()
+  if (res.status === 202 || !text) {
+    if (isRequest) errorReply(msg.id, -32000, `HTTP ${res.status} without a response body`)
+    return
+  }
+  const emit = (body) => {
+    if (body?.result?.protocolVersion && msg?.method === 'initialize') protocolVersion = body.result.protocolVersion
+    write(body)
+  }
+  if (type.includes('text/event-stream')) {
+    // Request-scoped SSE stream: forward every `data:` event.
+    for (const block of text.split(/\r?\n\r?\n/)) {
+      const data = block.split(/\r?\n/).filter(l => l.startsWith('data:')).map(l => l.slice(5).replace(/^ /, '')).join('\n')
+      if (data) { try { emit(JSON.parse(data)) } catch {} }
+    }
+    return
+  }
+  let body
+  try { body = JSON.parse(text) } catch {
+    if (isRequest) errorReply(msg.id, -32000, `HTTP ${res.status}: ${text.slice(0, 200)}`)
+    return
+  }
+  if (Array.isArray(body)) { for (const b of body) emit(b); return }
+  if (body && typeof body === 'object' && body.jsonrpc === '2.0') {
+    // Errors without an id (401 / 403) belong to this request.
+    if (isRequest && (body.id === null || body.id === undefined) && body.error) body.id = msg.id
+    if (res.status === 404 && sessionId && msg?.method !== 'initialize') sessionId = undefined
+    emit(body)
+  } else if (isRequest) errorReply(msg.id, -32000, `HTTP ${res.status}`)
+}
+
+let buffer = ''
+process.stdin.setEncoding('utf8')
+process.stdin.on('data', (chunk) => {
+  buffer += chunk
+  let i
+  while ((i = buffer.indexOf('\n')) >= 0) {
+    const line = buffer.slice(0, i).trim()
+    buffer = buffer.slice(i + 1)
+    if (!line) continue
+    let msg
+    try { msg = JSON.parse(line) } catch { errorReply(null, -32700, 'Parse error'); continue }
+    pending++
+    forward(msg).catch(e => log('forward failed:', e.message)).finally(() => { pending--; if (ended && !pending) process.exit(0) })
+  }
+})
+process.stdin.on('end', () => { ended = true; if (!pending) process.exit(0) })

package/channels/dingtalk.js

@@ -48,7 +48,7 @@
  */
 import { createHmac } from 'node:crypto'
 import { ChannelError, postJson, request } from './http.js'
-import { resultMeta, isKpi, kpiText, fmtNum, cellText, toPlainMarkdown, mentionsAll, splitText, MD_TABLE, parseMdTable } from './message.js'
+import { resultMeta, isKpi, kpiText, fmtNum, cellText, defangMd, toPlainMarkdown, mentionsAll, splitText, MD_TABLE, parseMdTable } from './message.js'
 
 export const OAPI = 'https://oapi.dingtalk.com'
 export const API = 'https://api.dingtalk.com'
@@ -153,11 +153,12 @@ export function dingApiError(json, status) {
 // ------------------------------------------------------------- rendering
 
 /** A result as text lines (DingTalk markdown has no tables). */
-export function resultAsDingText(r, maxRows = 10, maxCols = 6) {
+export function resultAsDingText(r, maxRows = 10, maxCols = 6, { escape = defangMd } = {}) {
   const cols = r.columns.slice(0, maxCols)
   const rows = r.rows.slice(0, maxRows)
-  const cell = (c, v) => (c.kind === 'number' ? fmtNum(v) : cellText(v, 30)).replace(/\|/g, '｜')
-  const out = [`**${cols.map(c => c.name).join(' ｜ ')}**`, '', ...rows.map((row, i) => `${i + 1}. ${cols.map((c, j) => cell(c, row[j])).join(' ｜ ')}`)]
+  // NULL → '—' (not 0); line breaks would split a numbered row; data must not form links / tags.
+  const cell = (c, v) => escape((c.kind === 'number' ? fmtNum(v) : cellText(v, 30)) || '—').replace(/\|/g, '｜')
+  const out = [`**${cols.map(c => escape(c.name)).join(' ｜ ')}**`, '', ...rows.map((row, i) => `${i + 1}. ${cols.map((c, j) => cell(c, row[j])).join(' ｜ ')}`)]
   const total = r.rowCount ?? r.rows.length
   if (total > rows.length) out.push('', `> 共 ${total} 行，仅显示前 ${rows.length} 行；完整数据可在数据问答页面查看或导出 Excel`)
   return out.join('\n')
@@ -167,7 +168,8 @@ export function resultAsDingText(r, maxRows = 10, maxCols = 6) {
 export function stripTables(md) {
   return String(md ?? '').replace(MD_TABLE, (m, lead, block) => {
     const t = parseMdTable(block)
-    return `${lead}${resultAsDingText({ ...t, rowCount: t.rows.length }, 50, 8)}\n`
+    // Tables authored in the message text keep their Markdown (links etc.).
+    return `${lead}${resultAsDingText({ ...t, rowCount: t.rows.length }, 50, 8, { escape: (x) => String(x ?? '') })}\n`
   })
 }
 
@@ -179,18 +181,18 @@ export function toDingMarkdown(md) {
 /** OutboundMessage → { title, text } for msgtype markdown / sampleMarkdown. */
 export function renderMarkdown(message, { maxRows = 10 } = {}) {
   const parts = []
-  if (message.title) parts.push(`### ${message.title}`)
+  if (message.title) parts.push(`### ${defangMd(message.title)}`)
   if (message.text) parts.push(toDingMarkdown(message.text))
   for (const r of (message.results ?? []).slice(0, 10)) {
-    const lines = [`#### ${r.title || '查询结果'}`, `> ${resultMeta(r)}`, '']
-    if (r.error) lines.push(r.error)
+    const lines = [`#### ${defangMd(r.title || '查询结果')}`, `> ${resultMeta(r)}`, '']
+    if (r.error) lines.push(defangMd(r.error))
     else if (!r.rows?.length) lines.push('无数据')
     else if (isKpi(r)) lines.push(kpiText(r, { separator: '\n\n' }))
     else lines.push(resultAsDingText(r, maxRows))
     if (r.sql && message.includeSql) lines.push('', '```sql\n' + r.sql.slice(0, 1500) + '\n```')
     parts.push(lines.join('\n'))
   }
-  if (message.footer) parts.push(toDingMarkdown(message.footer))
+  if (message.footer) parts.push(defangMd(toDingMarkdown(message.footer)))
   const all = message.mentionAll || mentionsAll(message.text) || mentionsAll(message.footer)
   if (all && !parts.some(p => p.includes('@所有人'))) parts.push('@所有人')
   const title = String(message.title || '数据问答').slice(0, 60)

package/channels/lark.js

@@ -16,7 +16,7 @@
  */
 import { createHash, createHmac } from 'node:crypto'
 import { ChannelError, request } from './http.js'
-import { MD_TABLE, cellText, chartOption, chartable, dualAxis, kpiText, isKpi, needsAttachment, num, omittedNote, parseMdTable, renderChartPng, resultMeta } from './message.js'
+import { MD_TABLE, cellText, chartOption, chartable, dualAxis, escapeLarkMd, kpiText, isKpi, needsAttachment, num, oneLine, omittedNote, parseMdTable, renderChartPng, resultMeta } from './message.js'
 
 export { ChannelError, ChannelError as ImError, chartOption, chartable, dualAxis, renderChartPng }
 
@@ -704,7 +704,13 @@ function tableElement(result, maxRows) {
     row_height: 'low',
     header_style: { background_style: 'grey', bold: true, text_size: 'normal' },
     columns: cols.map((c, i) => ({ name: `c${i}`, display_name: c.name, data_type: c.kind === 'number' ? 'number' : 'text', ...(c.kind === 'number' ? { format: { separator: true } } : {}), width: 'auto' })),
-    rows: result.rows.slice(0, maxRows).map(r => Object.fromEntries(cols.map((c, i) => [`c${i}`, c.kind === 'number' && Number.isFinite(Number(r[i])) ? Number(r[i]) : cellText(r[i])]))),
+    // NULL / '' numbers stay empty (Number(null) would show 0); line breaks would break rows.
+    rows: result.rows.slice(0, maxRows).map(r => Object.fromEntries(cols.flatMap((c, i) => {
+      const v = r[i]
+      if (c.kind !== 'number') return [[`c${i}`, oneLine(cellText(v))]]
+      if (v === null || v === undefined || v === '') return [] // empty cell, not 0
+      return [[`c${i}`, Number.isFinite(Number(v)) ? Number(v) : oneLine(cellText(v))]]
+    }))),
   }
 }
 
@@ -840,15 +846,17 @@ export function buildCard(message, { maxRows = 50, charts = true, actions = true
   for (const r of message.results ?? []) {
     if (elements.length) elements.push({ tag: 'hr' })
     if (r.error) {
-      elements.push({ tag: 'markdown', content: `**${r.title || '查询'}**\n<font color='red'>${r.error}</font>` })
+      elements.push({ tag: 'markdown', content: `**${escapeLarkMd(r.title || '查询')}**\n<font color='red'>${escapeLarkMd(r.error)}</font>` })
       continue
     }
-    elements.push({ tag: 'markdown', content: [`**${r.title || '查询结果'}**`, `<font color='grey'>${resultMeta(r)}</font>`].join('  ') })
+    // Everything data-derived (titles, values, errors, footer) is escaped: a cell like
+    // `<at id=all></at>` or `[x](https://…)` must show as text, not mention or link.
+    elements.push({ tag: 'markdown', content: [`**${escapeLarkMd(r.title || '查询结果')}**`, `<font color='grey'>${resultMeta(r, { escape: escapeLarkMd })}</font>`].join('  ') })
     const nativeChart = charts && chartCount < 5 && chartable(r) ? chartElement(r) : undefined
     if (r.rows.length === 0) {
       elements.push({ tag: 'markdown', content: '<font color=\'grey\'>无数据</font>' })
     } else if (isKpi(r)) {
-      elements.push({ tag: 'markdown', content: kpiText(r) })
+      elements.push({ tag: 'markdown', content: kpiText(r, { escape: escapeLarkMd }) })
     } else if (nativeChart) {
       elements.push(nativeChart)
       chartCount++
@@ -860,7 +868,7 @@ export function buildCard(message, { maxRows = 50, charts = true, actions = true
       const note = omittedNote(r, Math.min(maxRows, r.rows.length), { attached: Boolean(r.attached) })
       if (note) elements.push({ tag: 'markdown', content: `<font color='grey'>${note}</font>`, text_size: 'notation' })
     } else {
-      elements.push({ tag: 'markdown', content: r.rows.slice(0, 5).map(row => row.slice(0, 6).map(v => cellText(v)).join(' | ')).join('\n') })
+      elements.push({ tag: 'markdown', content: r.rows.slice(0, 5).map(row => row.slice(0, 6).map(v => escapeLarkMd(cellText(v))).join(' | ')).join('\n') })
       const note = omittedNote(r, Math.min(5, r.rows.length), { attached: Boolean(r.attached) })
       if (note) elements.push({ tag: 'markdown', content: `<font color='grey'>${note}</font>`, text_size: 'notation' })
     }
@@ -871,7 +879,7 @@ export function buildCard(message, { maxRows = 50, charts = true, actions = true
     }
     if (r.sql && message.includeSql) elements.push({ tag: 'markdown', content: `\`\`\`sql\n${r.sql.slice(0, 1500)}\n\`\`\`` })
   }
-  if (message.footer) elements.push({ tag: 'markdown', content: `<font color='grey'>${message.footer}</font>`, text_size: 'notation' })
+  if (message.footer) elements.push({ tag: 'markdown', content: `<font color='grey'>${escapeLarkMd(message.footer)}</font>`, text_size: 'notation' })
   const card = {
     schema: '2.0',
     config: { wide_screen_mode: true, summary: { content: String(message.title).slice(0, 60) } },
@@ -894,10 +902,10 @@ export function legacyCard(card) {
   const md = card.body.elements.map(e => {
     if (e.tag === 'markdown') return e.content
     if (e.tag === 'hr') return '---'
-    if (e.tag === 'table') return [e.columns.map(c => c.display_name).join(' | '), ...e.rows.slice(0, 10).map(r => e.columns.map(c => cellText(r[c.name])).join(' | '))].join('\n')
+    if (e.tag === 'table') return [e.columns.map(c => escapeLarkMd(c.display_name)).join(' | '), ...e.rows.slice(0, 10).map(r => e.columns.map(c => escapeLarkMd(cellText(r[c.name]))).join(' | '))].join('\n')
     if (e.tag === 'chart') {
       const v = e.chart_spec.data?.values ?? e.chart_spec.data?.[0]?.values ?? []
-      return v.slice(0, 10).map(x => Object.values(x).map(y => cellText(y)).join(' | ')).join('\n')
+      return v.slice(0, 10).map(x => Object.values(x).map(y => escapeLarkMd(cellText(y))).join(' | ')).join('\n')
     }
     return ''
   }).filter(Boolean).join('\n\n')

package/channels/message.js

@@ -14,8 +14,9 @@ export function num(v) {
   return v === null || v === undefined || v === '' ? null : Number(v)
 }
 
-/** Compact number: 1.23亿 / 4.56万 / integers as-is / two decimals. */
+/** Compact number: 1.23亿 / 4.56万 / integers as-is / two decimals. NULL / '' → ''. */
 export function fmtNum(v) {
+  if (v === null || v === undefined || v === '') return ''
   const n = Number(v)
   if (!Number.isFinite(n)) return String(v ?? '')
   if (Math.abs(n) >= 1e8) return `${(n / 1e8).toFixed(2)}亿`
@@ -30,10 +31,44 @@ export function cellText(v, max = 80) {
   return s.length > max ? `${s.slice(0, max - 1)}…` : s
 }
 
-/** A KPI-style result (one row) as `**col**：value ｜ …` Markdown. */
-export function kpiText(result, { separator = '　｜　', bold = true } = {}) {
+/** Line breaks (CR, LF, CRLF) → one space: keeps table rows and inline Markdown on one line. */
+export function oneLine(s) {
+  return String(s ?? '').replace(/\r\n?|\n/g, ' ')
+}
+
+/**
+ * Lark card Markdown: data-derived text (cell values, column names, result titles,
+ * errors, footers) as literal text. Lark documents HTML numeric entities for
+ * characters that would trigger Markdown / tags (`<at>`, `<font>`, links, emphasis).
+ * https://open.larksuite.com/document/uAjLw4CM/ukzMukzMukzM/feishu-cards/card-components/content-components/rich-text
+ */
+export function escapeLarkMd(s) {
+  return oneLine(s)
+    .replace(/&(?=#?\w+;)/g, '&#38;')
+    .replace(/[<>[\]*_~`\\]/g, ch => `&#${ch.charCodeAt(0)};`)
+}
+
+/**
+ * Neutralize link / tag syntax in data-derived text for platforms whose Markdown
+ * has no reliable escape (WeCom, DingTalk): `<` `>` `[` `]` become full-width look-alikes,
+ * so values cannot create links, images, `<@user>` mentions or `<font>` tags.
+ */
+export function defangMd(s) {
+  return oneLine(s).replace(/[<>[\]]/g, ch => ({ '<': '＜', '>': '＞', '[': '［', ']': '］' })[ch])
+}
+
+/**
+ * A KPI-style result (one row) as `**col**：value ｜ …` Markdown. NULL shows as '—'.
+ * `escape` is applied to data-derived text (column names and values).
+ */
+export function kpiText(result, { separator = '　｜　', bold = true, escape = defangMd } = {}) {
   const row = result.rows?.[0] ?? []
-  return result.columns.map((c, i) => `${bold ? `**${c.name}**` : c.name}：${c.kind === 'number' ? fmtNum(row[i]) : cellText(row[i])}`).join(separator)
+  return result.columns.map((c, i) => {
+    const raw = c.kind === 'number' ? fmtNum(row[i]) : cellText(row[i])
+    const value = raw === '' ? '—' : escape(oneLine(raw))
+    const name = escape(oneLine(c.name))
+    return `${bold ? `**${name}**` : name}：${value}`
+  }).join(separator)
 }
 
 /** True when the result should be shown as KPI text rather than a table/chart. */
@@ -59,8 +94,8 @@ export function needsAttachment(result, shown) {
   return (result.rowCount ?? result.rows.length) > shown
 }
 
-export function resultMeta(result) {
-  return `${result.rowCount ?? result.rows?.length ?? 0} 行${result.truncated ? '（已截断）' : ''}${result.filters?.length ? ` · 已筛选：${result.filters.map(f => f.description).join('；')}` : ''}`
+export function resultMeta(result, { escape = defangMd } = {}) {
+  return `${result.rowCount ?? result.rows?.length ?? 0} 行${result.truncated ? '（已截断）' : ''}${result.filters?.length ? ` · 已筛选：${result.filters.map(f => escape(f.description)).join('；')}` : ''}`
 }
 
 // ------------------------------------------------------- chart rendering
@@ -207,7 +242,7 @@ export function toPlainText(md) {
 }
 
 function mdCell(v, max = 40) {
-  return cellText(v, max).replace(/\|/g, '\\|').replace(/\n/g, ' ')
+  return defangMd(cellText(v, max)).replace(/\|/g, '\\|')
 }
 
 /** A result as a Markdown table (≤ maxRows rows, ≤ maxCols columns), with a "… N more" note when cut. */
@@ -229,7 +264,7 @@ export function resultAsMarkdownTable(result, maxRows = 10, { maxCols = 8 } = {}
 export function resultAsText(result, maxRows = 10, { maxCols = 6 } = {}) {
   const cols = (result.columns ?? []).slice(0, maxCols)
   const rows = (result.rows ?? []).slice(0, maxRows)
-  const lines = [cols.map(c => c.name).join(' | '), ...rows.map(r => cols.map((c, i) => (c.kind === 'number' ? fmtNum(r[i]) : cellText(r[i], 30))).join(' | '))]
+  const lines = [cols.map(c => c.name).join(' | '), ...rows.map(r => cols.map((c, i) => oneLine(c.kind === 'number' ? fmtNum(r[i]) : cellText(r[i], 30))).join(' | '))]
   const total = result.rowCount ?? result.rows?.length ?? 0
   if (total > rows.length) lines.push(`…共 ${total} 行`)
   return lines.join('\n')
@@ -243,11 +278,12 @@ export function resultAsText(result, maxRows = 10, { maxCols = 6 } = {}) {
 export function messageAsMarkdown(message, { markdown = 'basic', maxRows = 10, sqlMax = 1500 } = {}) {
   const clean = markdown === 'full' ? normalizeMarkdown : markdown === 'none' ? toPlainText : toPlainMarkdown
   const parts = []
-  if (message.title) parts.push(markdown === 'none' ? String(message.title) : `**${message.title}**`)
+  if (message.title) parts.push(markdown === 'none' ? oneLine(message.title) : `**${defangMd(message.title)}**`)
   if (message.text) parts.push(clean(message.text))
   for (const r of message.results ?? []) {
-    const head = markdown === 'none' ? `${r.title || '查询结果'}` : `**${r.title || '查询结果'}**`
-    if (r.error) { parts.push(`${head}\n${r.error}`); continue }
+    // Result titles, errors and cells are data-derived: no Markdown / tags from them.
+    const head = markdown === 'none' ? oneLine(r.title || '查询结果') : `**${defangMd(r.title || '查询结果')}**`
+    if (r.error) { parts.push(`${head}\n${markdown === 'none' ? r.error : defangMd(r.error)}`); continue }
     const lines = [`${head}  ${resultMeta(r)}`]
     if (!r.rows?.length) lines.push('无数据')
     else if (isKpi(r)) lines.push(kpiText(r, { bold: markdown !== 'none' }))
@@ -255,7 +291,7 @@ export function messageAsMarkdown(message, { markdown = 'basic', maxRows = 10, s
     if (r.sql && message.includeSql) lines.push(markdown === 'none' ? r.sql.slice(0, sqlMax) : `\`\`\`sql\n${r.sql.slice(0, sqlMax)}\n\`\`\``)
     parts.push(lines.join('\n'))
   }
-  if (message.footer) parts.push(clean(message.footer))
+  if (message.footer) parts.push(markdown === 'none' ? toPlainText(message.footer) : defangMd(toPlainMarkdown(message.footer)))
   return parts.join('\n\n')
 }
 

package/channels/telegram.js

@@ -18,7 +18,7 @@
  */
 import { randomBytes } from 'node:crypto'
 import { ChannelError, request } from './http.js'
-import { renderChartPng as defaultRenderChartPng, chartable, resultMeta, isKpi, kpiText, resultAsMarkdownTable, toPlainMarkdown } from './message.js'
+import { renderChartPng as defaultRenderChartPng, chartable, oneLine, resultMeta, isKpi, kpiText, resultAsMarkdownTable, toPlainMarkdown } from './message.js'
 
 export const DEFAULT_API_BASE = 'https://api.telegram.org'
 export const TEXT_LIMIT = 4096
@@ -280,12 +280,19 @@ export function inlineKeyboard(spec) {
 
 // ------------------------------------------------------------ rendering
 
+/**
+ * Data-derived text in Telegram Markdown: HTML is escaped by the converter anyway, so only
+ * link syntax and line breaks are neutralized (`[x](url)` must not become a link).
+ */
+const defangMd = (s) => oneLine(s).replace(/[[\]]/g, ch => (ch === '[' ? '［' : '］'))
+
 function resultMarkdown(r, { includeSql, maxRows = 20 } = {}) {
-  const head = `**${r.title || '查询结果'}**  ${resultMeta(r)}`
-  if (r.error) return `${head}\n${r.error}`
+  // Data-derived text (title, error, values) is defanged: no links / tags / mentions from data.
+  const head = `**${defangMd(r.title || '查询结果')}**  ${resultMeta(r)}`
+  if (r.error) return `${head}\n${defangMd(r.error)}`
   const lines = [head]
   if (!r.rows?.length) lines.push('无数据')
-  else if (isKpi(r)) lines.push(kpiText(r, { separator: '\n' }))
+  else if (isKpi(r)) lines.push(kpiText(r, { separator: '\n', escape: defangMd }))
   else lines.push(resultAsMarkdownTable(r, maxRows, { maxCols: 6 }))
   if (r.sql && includeSql) lines.push('```sql\n' + r.sql.slice(0, 1500) + '\n```')
   return lines.join('\n')
@@ -297,7 +304,7 @@ function resultMarkdown(r, { includeSql, maxRows = 20 } = {}) {
  */
 export function planMessage(message, { buttons = true } = {}) {
   const segs = []
-  const head = [message.title ? `**${message.title}**` : '', message.text ?? ''].filter(Boolean).join('\n\n')
+  const head = [message.title ? `**${defangMd(message.title)}**` : '', message.text ?? ''].filter(Boolean).join('\n\n')
   let text = head
   const flush = (keyboard) => { if (text.trim() || keyboard) segs.push({ kind: 'text', md: text, ...(keyboard ? { keyboard } : {}) }); text = '' }
   for (const r of (message.results ?? []).slice(0, 10)) {
@@ -310,7 +317,7 @@ export function planMessage(message, { buttons = true } = {}) {
     text = [text, resultMarkdown(r, { includeSql: message.includeSql })].filter(s => s.trim()).join('\n\n')
     if (keyboard) flush(keyboard)
   }
-  if (message.footer) text = [text, message.footer].filter(s => s.trim()).join('\n\n')
+  if (message.footer) text = [text, defangMd(toPlainMarkdown(message.footer, { mentionAll: '' }))].filter(s => s.trim()).join('\n\n')
   flush()
   return segs
 }