duckfn-docs-kit 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +47 -6
- package/bin/sql-verify.mjs +12 -0
- package/dist/sql/collect.d.ts +29 -0
- package/dist/sql/collect.js +65 -0
- package/dist/sql/nodeRunner.d.ts +51 -0
- package/dist/sql/nodeRunner.js +115 -0
- package/dist/sql/remark.d.ts +20 -0
- package/dist/sql/remark.js +1 -1
- package/dist/sql/verify.d.ts +61 -0
- package/dist/sql/verify.js +148 -0
- package/package.json +5 -1
- package/src/sql/collect.ts +136 -0
- package/src/sql/nodeRunner.ts +298 -0
- package/src/sql/remark.ts +21 -3
- package/src/sql/verify.ts +357 -0
- package/src/toc-toggle/TocToggle.css +28 -0
package/AGENTS.md
CHANGED
|
@@ -42,6 +42,8 @@ src/
|
|
|
42
42
|
│ # + PreviewTabs.ts(预览页签,末尾恒定 Table)+ remark.ts
|
|
43
43
|
│ # + extensions.ts(Node:扩展预加载插件)+ runtimeConfig.ts
|
|
44
44
|
│ # (两侧共享的注入配置契约,见「扩展预加载」)
|
|
45
|
+
│ # + collect.ts / nodeRunner.ts / verify.ts(Node:文档站自己的
|
|
46
|
+
│ # SQL 测试与 `duckfn-sql-verify` 命令,见「文档站 SQL 测试」)
|
|
45
47
|
│ # + client.ts(dfk-* 元素注册,插件注入到每个页面)
|
|
46
48
|
├── theme/ # tokens.css —— 全局设计基础设施,无业务归属,单独放
|
|
47
49
|
├── kit.css # 全局 CSS 聚合入口(@import theme + toc-toggle + sql)
|
|
@@ -76,6 +78,8 @@ src/
|
|
|
76
78
|
- `duckfn-docs-kit/remark`(**Node 构建期**:版本占位符 remark 插件)
|
|
77
79
|
- `duckfn-docs-kit/sql/remark`(**Node 构建期**:可运行 SQL remark 插件)
|
|
78
80
|
- `duckfn-docs-kit/sql/extensions`(**Node 构建期**:扩展预加载 Docusaurus 插件)
|
|
81
|
+
- `duckfn-docs-kit/sql/verify`(**Node 运行期**:文档站 SQL 测试的入口与 CLI;
|
|
82
|
+
`sql/collect`、`sql/nodeRunner` 是它的两半,也可单独用)
|
|
79
83
|
- `duckfn-docs-kit/toc-toggle/plugin`(**Node 构建期**:TOC 胶水插件)
|
|
80
84
|
- `duckfn-docs-kit/sql/client`、`duckfn-docs-kit/toc-toggle/client`(浏览器引导,
|
|
81
85
|
由上面两个插件注入,站点不要手写引用)
|
|
@@ -85,6 +89,20 @@ src/
|
|
|
85
89
|
与源路径脱钩的别名。`home.css` **不作为**全局 CSS 导出 —— 它由
|
|
86
90
|
`home/styles.ts` 内联进 JS bundle,注入各组件的 shadow root。
|
|
87
91
|
|
|
92
|
+
### TOC 折叠控件(`toc-toggle/`)
|
|
93
|
+
|
|
94
|
+
- Docusaurus 的桌面目录元素是 `.theme-doc-toc-desktop`,它自带
|
|
95
|
+
`position: sticky; top: calc(var(--ifm-navbar-height) + 1rem)`(见主题里的
|
|
96
|
+
`theme/TOC/styles.module.css`);而按钮由本模块插在它**前面**、同一个
|
|
97
|
+
`.toc-column` 里。
|
|
98
|
+
- 所以按钮必须自己 `position: sticky`:否则文章一长,TOC 留在视口里悬浮,按钮却随页面滚上去,
|
|
99
|
+
读者把目录折叠后再往下滚就找不回来。TOC 的 `top` / `max-height` 要相应下移一个按钮高度
|
|
100
|
+
(按钮 2rem + 上下留白),两者才不会重叠 —— 三条规则都在 `TocToggle.css` 的
|
|
101
|
+
`@layer docusaurus.theme-classic` 段里,且带 `.toc-column` 前缀:TOC 那条来自 CSS module 的
|
|
102
|
+
哈希类名,同层同特异性时胜负取决于样式表顺序,前缀是唯一稳妥的写法。
|
|
103
|
+
- 折叠态隐藏的是整个 `.theme-doc-toc-desktop`(按钮在它外面,因此仍在),列收成 2.5rem 宽,
|
|
104
|
+
按钮此时居中而不是靠右。
|
|
105
|
+
|
|
88
106
|
### 可运行 SQL:渲染契约与 DuckDB-Wasm 事实
|
|
89
107
|
|
|
90
108
|
`sql/` 是一条单向链:`remark.ts`(构建期)→ `DfkSql.ts`(元素)→ `runtime.ts`
|
|
@@ -285,6 +303,28 @@ src/
|
|
|
285
303
|
戳严格校验(1.5.4 的原生 duckdb 会拒绝 v1.5.5 构建的扩展)。升级 duckdb-wasm 或
|
|
286
304
|
改 CI 的 `duckdb_version` 时必须成对验证(跑一遍可运行 SQL 页的两个示例块即可)。
|
|
287
305
|
|
|
306
|
+
**文档站 SQL 测试(`sql/verify` + `sql/collect` / `sql/nodeRunner`)**
|
|
307
|
+
|
|
308
|
+
- 入口是 `bin/sql-verify.mjs`(bin 名 `duckfn-sql-verify`),它 import `dist/sql/verify.js`。
|
|
309
|
+
bin **手写**、不进构建:Vite lib 产物是 ESM、不保留 shebang,而 npm 只需要一个带 shebang
|
|
310
|
+
且有执行位的文件。站点侧接成 `npm test` 即可(本仓库见 `docs/package.json`)。
|
|
311
|
+
- 收集与渲染共用一份 meta 解析(`sql/remark.ts` 导出的 `parseRunnableSqlMeta`):站点上不是
|
|
312
|
+
可运行块的,测试也不会跑。
|
|
313
|
+
- 围栏按 CommonMark 收口:闭合围栏同字符、不短于开启围栏、且无 info string —— 这样
|
|
314
|
+
````md 包着的 ```sql 示例(`runnable-sql.md` 就这么展示 meta)不会被当成块。
|
|
315
|
+
- 执行用 **Node worker target**(`duckdb-node.cjs`),不用 blocking target:注册期会自行打开
|
|
316
|
+
连接的扩展(能用文件系统的那些)在 blocking 目标上**死锁**,表现为 `LOAD` 永久卡住(心跳静默,
|
|
317
|
+
进程内超时也打不断)。浏览器没这问题,因为那边的扩展在 worker 线程里加载。
|
|
318
|
+
- 扩展只能经 **http URL** 加载(裸文件名 / 本地 VFS 路径在 wasm 上直接挂死,`registerFileBuffer`
|
|
319
|
+
也救不了),所以运行器起一个 loopback 服务。三个随之而来的约束:DuckDB 把拉下来的扩展暂存到
|
|
320
|
+
`~/.duckdb/extensions/<host>/<URL 一级路径段>/`,故 (1) URL 必须带一层路径段,(2) 该目录要
|
|
321
|
+
**预先建好**(加载器自己的 `mkdir` 非递归),(3) **Windows 上必须用 80 端口**让 URL 不含端口
|
|
322
|
+
—— 冒号在 Windows 路径里非法;其它平台用任意空闲端口。缓存目录每次运行前清空,免得测的是上一
|
|
323
|
+
次的旧扩展。
|
|
324
|
+
- 形态与页面一致:**每页新实例 + 页内共用连接**(页内可以依赖前一个块建的宏/表,页与页隔离)。
|
|
325
|
+
- 故意失败的块在 meta 里声明 `"expect": "error"`(`expectsError(config)` 读它,默认 `ok`):
|
|
326
|
+
期望是数据,不能靠对注释做字符串匹配;校验**双向** —— 声明会失败却跑成功同样要报出来。
|
|
327
|
+
|
|
288
328
|
## 代码风格(硬性要求)
|
|
289
329
|
|
|
290
330
|
这些规则是本包存在的意义所在,评审时逐条对照。
|
|
@@ -555,8 +595,9 @@ Docusaurus 预渲染在 Node 里 import 本包。
|
|
|
555
595
|
才调用):模块级 `new CSSStyleSheet()` 会在 Node 预渲染 import 时直接崩。
|
|
556
596
|
`?inline` import 进来的只是字符串,模块级安全。
|
|
557
597
|
- `iconify-icon` 在 Node 里 import 是安全的(官方包已处理)。
|
|
558
|
-
- Node
|
|
559
|
-
|
|
598
|
+
- Node 侧模块只有 `src/remark.ts`、`src/sql/remark.ts`、`src/sql/extensions.ts`(构建期)
|
|
599
|
+
与 `src/sql/collect.ts`、`src/sql/nodeRunner.ts`、`src/sql/verify.ts`(测试期),
|
|
600
|
+
连同无依赖的共享契约 `src/sql/runtimeConfig.ts`;它们都不得 import 任何浏览器模块。
|
|
560
601
|
|
|
561
602
|
### 11. React 19 自定义元素
|
|
562
603
|
|
|
@@ -610,7 +651,7 @@ npm whoami # 没登录先 npm login
|
|
|
610
651
|
```
|
|
611
652
|
|
|
612
653
|
`npm pack --dry-run` 打印 tarball 的文件清单,是发布前最值得看的一项:预期是 `dist/**` +
|
|
613
|
-
`src/**` + `AGENTS.md` + `README.md` + `LICENSE` + `package.json`(当前
|
|
654
|
+
`src/**` + `bin/**` + `AGENTS.md` + `README.md` + `LICENSE` + `package.json`(当前 75 个文件)。
|
|
614
655
|
多出别的东西时先查 `files` 白名单,不要靠 `.npmignore` 追着排除。
|
|
615
656
|
|
|
616
657
|
### 1. 提升版本号
|
|
@@ -669,9 +710,9 @@ just release_kit_dev 0.1.2-dev.0
|
|
|
669
710
|
|
|
670
711
|
包内容 = `files` 白名单 + npm 的固定规则,所以**不需要**在包的结构之外维护清单:
|
|
671
712
|
|
|
672
|
-
- `files: ["dist", "src", "AGENTS.md"]`:`dist` 是构建产物;`src` 必须带上(CSS 子路径
|
|
673
|
-
`duckfn-docs-kit/src/kit.css` 直接指向源文件);`
|
|
674
|
-
|
|
713
|
+
- `files: ["dist", "src", "bin", "AGENTS.md"]`:`dist` 是构建产物;`src` 必须带上(CSS 子路径
|
|
714
|
+
`duckfn-docs-kit/src/kit.css` 直接指向源文件);`bin` 是 `duckfn-sql-verify` 的入口(它 import
|
|
715
|
+
`dist/`,所以两者都要在包里);`AGENTS.md` 随包发布,下游读者因此能看到本包的全部约定。
|
|
675
716
|
- `README.md`、`LICENSE`、`package.json` 由 npm 自动收录:本包目录下有自己的 `LICENSE`
|
|
676
717
|
(根目录那份不会被带进来),README 是 npm 包页的正文。
|
|
677
718
|
- `dist/` 在 `.gitignore` 里,但照样进包 —— `files` 白名单优先于 gitignore。`prepack` 脚本
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* The `duckfn-sql-verify` executable.
|
|
4
|
+
*
|
|
5
|
+
* A hand-written wrapper rather than a built entry: Vite's library build emits
|
|
6
|
+
* ESM and does not preserve a shebang, and npm only needs one file with one and
|
|
7
|
+
* an executable bit. `dist/` is built by `prepack`, so the import below always
|
|
8
|
+
* resolves in a published tarball.
|
|
9
|
+
*/
|
|
10
|
+
import {cliMain} from '../dist/sql/verify.js';
|
|
11
|
+
|
|
12
|
+
await cliMain(process.argv.slice(2));
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
import { type RunnableSqlConfig } from './remark';
|
|
2
|
+
/** One runnable block, positioned so a failure can name it. */
|
|
3
|
+
export interface RunnableSqlBlock {
|
|
4
|
+
/** Path relative to the site root, always with forward slashes. */
|
|
5
|
+
file: string;
|
|
6
|
+
/** 1-based line of the opening fence. */
|
|
7
|
+
line: number;
|
|
8
|
+
config: RunnableSqlConfig;
|
|
9
|
+
sql: string;
|
|
10
|
+
}
|
|
11
|
+
export interface CollectRunnableSqlOptions {
|
|
12
|
+
/** Site root the reported paths are relative to, and the base of a relative dir. */
|
|
13
|
+
siteDir: string;
|
|
14
|
+
/**
|
|
15
|
+
* Directories to scan (absolute, or relative to `siteDir`). Missing ones are
|
|
16
|
+
* skipped rather than reported: a site may have no translations.
|
|
17
|
+
*/
|
|
18
|
+
contentDirs: readonly string[];
|
|
19
|
+
/** File extensions to scan; both `.md` and `.mdx` are markdown to us. */
|
|
20
|
+
extensions?: readonly string[];
|
|
21
|
+
}
|
|
22
|
+
/** Every runnable block under `contentDirs`, in file order, then line order. */
|
|
23
|
+
export declare function collectRunnableSql(options: CollectRunnableSqlOptions): RunnableSqlBlock[];
|
|
24
|
+
/**
|
|
25
|
+
* Whether a block is expected to fail — the block's own `"expect": "error"`
|
|
26
|
+
* metadata, never its prose or its comments: an expectation that the SQL test
|
|
27
|
+
* suite acts on has to be data, not a string match on a comment.
|
|
28
|
+
*/
|
|
29
|
+
export declare function expectsError(config: RunnableSqlConfig): boolean;
|
|
@@ -0,0 +1,65 @@
|
|
|
1
|
+
import { parseRunnableSqlMeta as e } from "./remark.js";
|
|
2
|
+
import { join as t, relative as n, sep as r } from "node:path";
|
|
3
|
+
import { readFileSync as i, readdirSync as a, statSync as o } from "node:fs";
|
|
4
|
+
//#region src/sql/collect.ts
|
|
5
|
+
var s = [".md", ".mdx"];
|
|
6
|
+
function c(e) {
|
|
7
|
+
let { siteDir: a, contentDirs: o, extensions: c = s } = e, d = [];
|
|
8
|
+
for (let e of o) l(t(a, e), c, d);
|
|
9
|
+
let f = [];
|
|
10
|
+
for (let e of d.sort()) for (let t of u(i(e, "utf8"))) f.push({
|
|
11
|
+
...t,
|
|
12
|
+
file: n(a, e).split(r).join("/")
|
|
13
|
+
});
|
|
14
|
+
return f;
|
|
15
|
+
}
|
|
16
|
+
function l(e, n, r) {
|
|
17
|
+
let i;
|
|
18
|
+
try {
|
|
19
|
+
i = a(e);
|
|
20
|
+
} catch {
|
|
21
|
+
return;
|
|
22
|
+
}
|
|
23
|
+
for (let a of i) {
|
|
24
|
+
let i = t(e, a);
|
|
25
|
+
o(i).isDirectory() ? l(i, n, r) : n.some((e) => a.endsWith(e)) && r.push(i);
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
function u(t) {
|
|
29
|
+
let n = t.split("\n"), r = [], i = null, a = [];
|
|
30
|
+
for (let t = 0; t < n.length; t++) {
|
|
31
|
+
let o = n[t] ?? "", s = /^(`{3,}|~{3,})(.*)$/.exec(o);
|
|
32
|
+
if (!s) {
|
|
33
|
+
i && a.push(o);
|
|
34
|
+
continue;
|
|
35
|
+
}
|
|
36
|
+
let [c, l] = [s[1] ?? "", (s[2] ?? "").trim()];
|
|
37
|
+
if (!i) {
|
|
38
|
+
i = {
|
|
39
|
+
marker: c,
|
|
40
|
+
info: l,
|
|
41
|
+
line: t + 1
|
|
42
|
+
}, a = [];
|
|
43
|
+
continue;
|
|
44
|
+
}
|
|
45
|
+
if (!(c.charAt(0) === i.marker.charAt(0) && c.length >= i.marker.length && l === "")) {
|
|
46
|
+
a.push(o);
|
|
47
|
+
continue;
|
|
48
|
+
}
|
|
49
|
+
if (i.info.startsWith("sql")) {
|
|
50
|
+
let t = e(i.info.slice(3).trim());
|
|
51
|
+
t && r.push({
|
|
52
|
+
line: i.line,
|
|
53
|
+
config: t,
|
|
54
|
+
sql: a.join("\n")
|
|
55
|
+
});
|
|
56
|
+
}
|
|
57
|
+
i = null, a = [];
|
|
58
|
+
}
|
|
59
|
+
return r;
|
|
60
|
+
}
|
|
61
|
+
function d(e) {
|
|
62
|
+
return e.expect === "error";
|
|
63
|
+
}
|
|
64
|
+
//#endregion
|
|
65
|
+
export { c as collectRunnableSql, d as expectsError };
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
export interface DuckdbQueryResult {
|
|
2
|
+
numRows: number;
|
|
3
|
+
schema: {
|
|
4
|
+
fields: {
|
|
5
|
+
name: string;
|
|
6
|
+
}[];
|
|
7
|
+
};
|
|
8
|
+
}
|
|
9
|
+
export interface DuckdbConnection {
|
|
10
|
+
query(sql: string): Promise<DuckdbQueryResult>;
|
|
11
|
+
}
|
|
12
|
+
export type WasmPlatform = 'eh' | 'mvp';
|
|
13
|
+
export interface RunnerOptions {
|
|
14
|
+
/**
|
|
15
|
+
* The extension to `LOAD`: a local `.duckdb_extension.wasm` path (served to
|
|
16
|
+
* the worker over a loopback http server) or an absolute `http(s)` URL.
|
|
17
|
+
*/
|
|
18
|
+
extension: string;
|
|
19
|
+
/**
|
|
20
|
+
* DuckDB-Wasm platform. Must match how the extension was built — a site
|
|
21
|
+
* serving `duckfn-wasm_eh.duckdb_extension.wasm` runs the `eh` bundle.
|
|
22
|
+
*/
|
|
23
|
+
platform?: WasmPlatform;
|
|
24
|
+
/** The engine wasm; defaults to the one shipped beside the worker bundle. */
|
|
25
|
+
engine?: string;
|
|
26
|
+
}
|
|
27
|
+
export interface RunResult {
|
|
28
|
+
rows: number;
|
|
29
|
+
columns: number;
|
|
30
|
+
}
|
|
31
|
+
/**
|
|
32
|
+
* One DuckDB-Wasm instance at a time, driven from Node.
|
|
33
|
+
*
|
|
34
|
+
* `newPage()` is what a docs site does per page load: a fresh instance, a fresh
|
|
35
|
+
* connection, the extension loaded again. Blocks of one page then share state
|
|
36
|
+
* (a table created in one block is visible to the next), while pages stay
|
|
37
|
+
* isolated — which is why the runner is used one page at a time rather than
|
|
38
|
+
* over a single long-lived connection.
|
|
39
|
+
*/
|
|
40
|
+
export declare class WasmSqlRunner {
|
|
41
|
+
#private;
|
|
42
|
+
private constructor();
|
|
43
|
+
static create(options: RunnerOptions): Promise<WasmSqlRunner>;
|
|
44
|
+
/** Drops the current instance and starts a fresh page: new instance, new connection. */
|
|
45
|
+
newPage(): Promise<void>;
|
|
46
|
+
/** Runs one block; the caller decides whether a failure is expected. */
|
|
47
|
+
run(sql: string): Promise<RunResult>;
|
|
48
|
+
/** Where the fetched extension was staged, for diagnostics. */
|
|
49
|
+
get stagingDir(): string | null;
|
|
50
|
+
close(): Promise<void>;
|
|
51
|
+
}
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
import { createRequire as e } from "node:module";
|
|
2
|
+
import { basename as t, dirname as n, join as r } from "node:path";
|
|
3
|
+
import { mkdirSync as i, readFileSync as a, rmSync as o } from "node:fs";
|
|
4
|
+
import { createServer as s } from "node:http";
|
|
5
|
+
import { Worker as c } from "node:worker_threads";
|
|
6
|
+
import { homedir as l } from "node:os";
|
|
7
|
+
//#region src/sql/nodeRunner.ts
|
|
8
|
+
var u = e(import.meta.url), d = "@duckdb/duckdb-wasm/dist/duckdb-node.cjs", f = class {
|
|
9
|
+
#e;
|
|
10
|
+
#t = /* @__PURE__ */ new Map();
|
|
11
|
+
onmessage = null;
|
|
12
|
+
onerror = null;
|
|
13
|
+
onclose = null;
|
|
14
|
+
constructor(e) {
|
|
15
|
+
this.#e = new c(u.resolve(d), { workerData: {
|
|
16
|
+
mod: e,
|
|
17
|
+
name: "",
|
|
18
|
+
type: ""
|
|
19
|
+
} }), this.#e.on("message", (e) => this.#n("message", e)), this.#e.on("error", (e) => this.#n("error", e)), this.#e.on("exit", () => this.#n("close"));
|
|
20
|
+
}
|
|
21
|
+
#n(e, t) {
|
|
22
|
+
let n = {
|
|
23
|
+
type: e,
|
|
24
|
+
data: t,
|
|
25
|
+
target: this,
|
|
26
|
+
currentTarget: this
|
|
27
|
+
}, r = this[`on${e}`];
|
|
28
|
+
typeof r == "function" && r(n);
|
|
29
|
+
for (let t of this.#t.get(e) ?? []) t(n);
|
|
30
|
+
}
|
|
31
|
+
addEventListener(e, t) {
|
|
32
|
+
this.#t.set(e, [...this.#t.get(e) ?? [], t]);
|
|
33
|
+
}
|
|
34
|
+
removeEventListener(e, t) {
|
|
35
|
+
this.#t.set(e, (this.#t.get(e) ?? []).filter((e) => e !== t));
|
|
36
|
+
}
|
|
37
|
+
postMessage(e, t) {
|
|
38
|
+
this.#e.postMessage(e, t ?? []);
|
|
39
|
+
}
|
|
40
|
+
terminate() {
|
|
41
|
+
return this.#e.terminate();
|
|
42
|
+
}
|
|
43
|
+
}, p = class e {
|
|
44
|
+
#e = null;
|
|
45
|
+
#t = null;
|
|
46
|
+
#n = null;
|
|
47
|
+
#r = null;
|
|
48
|
+
#i;
|
|
49
|
+
#a;
|
|
50
|
+
#o;
|
|
51
|
+
constructor(e, t, n) {
|
|
52
|
+
this.#i = e, this.#a = t, this.#o = n;
|
|
53
|
+
}
|
|
54
|
+
static async create(t) {
|
|
55
|
+
let i = t.platform ?? "eh", a = u.resolve(`@duckdb/duckdb-wasm/dist/duckdb-node-${i}.worker.cjs`), o = t.engine ?? r(n(a), `duckdb-${i}.wasm`), s = /^https?:\/\//i.test(t.extension) ? null : await m(t.extension), c = s ? s.url : t.extension, l = new e(c, o, a);
|
|
56
|
+
return l.#n = s ? s.server : null, l.#r = s ? h(s.stagingHost, s.stagingSegment) : null, l;
|
|
57
|
+
}
|
|
58
|
+
async newPage() {
|
|
59
|
+
this.#e && (await this.#e.terminate(), this.#e = null, this.#t = null);
|
|
60
|
+
let e = await import(
|
|
61
|
+
/* @vite-ignore */
|
|
62
|
+
d
|
|
63
|
+
), t = new e.AsyncDuckDB(new e.ConsoleLogger(), new f(this.#o));
|
|
64
|
+
await t.instantiate(this.#a, null), await t.open({ allowUnsignedExtensions: !0 });
|
|
65
|
+
let n = await t.connect();
|
|
66
|
+
await n.query(`LOAD '${this.#i}'`), this.#e = t, this.#t = n;
|
|
67
|
+
}
|
|
68
|
+
async run(e) {
|
|
69
|
+
let t = this.#t;
|
|
70
|
+
if (!t) throw Error("sql/verify: no page is open — call newPage() first");
|
|
71
|
+
let n = await t.query(e);
|
|
72
|
+
return {
|
|
73
|
+
rows: n.numRows,
|
|
74
|
+
columns: n.schema.fields.length
|
|
75
|
+
};
|
|
76
|
+
}
|
|
77
|
+
get stagingDir() {
|
|
78
|
+
return this.#r;
|
|
79
|
+
}
|
|
80
|
+
async close() {
|
|
81
|
+
this.#e && (await this.#e.terminate(), this.#e = null, this.#t = null), this.#n &&= (await new Promise((e) => this.#n?.close(() => e())), null);
|
|
82
|
+
}
|
|
83
|
+
};
|
|
84
|
+
async function m(e) {
|
|
85
|
+
let n = a(e), r = t(e), i = g(e), o = s((e, t) => {
|
|
86
|
+
t.writeHead(200, {
|
|
87
|
+
"content-type": "application/octet-stream",
|
|
88
|
+
"content-length": n.length
|
|
89
|
+
}), t.end(n);
|
|
90
|
+
}), c = process.platform === "win32" ? 80 : 0;
|
|
91
|
+
await new Promise((e, t) => {
|
|
92
|
+
o.once("error", (e) => {
|
|
93
|
+
t(/* @__PURE__ */ Error(`sql/verify: cannot serve the extension on port ${c} (${e.code}): a port in the URL would put a colon into DuckDB's staging path, which is not a legal Windows path — free port 80, or load the extension from an http(s) URL with --extension`));
|
|
94
|
+
}), o.listen(c, "localhost", e);
|
|
95
|
+
});
|
|
96
|
+
let l = o.address(), u = typeof l == "object" && l ? l.port : c, d = u === 80 ? "localhost" : `localhost:${u}`;
|
|
97
|
+
return {
|
|
98
|
+
server: o,
|
|
99
|
+
url: `http://${d}/${i}/${r}`,
|
|
100
|
+
stagingHost: d,
|
|
101
|
+
stagingSegment: i
|
|
102
|
+
};
|
|
103
|
+
}
|
|
104
|
+
function h(e, t) {
|
|
105
|
+
let n = r(l(), ".duckdb", "extensions", e, t);
|
|
106
|
+
return o(n, {
|
|
107
|
+
recursive: !0,
|
|
108
|
+
force: !0
|
|
109
|
+
}), i(n, { recursive: !0 }), n;
|
|
110
|
+
}
|
|
111
|
+
function g(e) {
|
|
112
|
+
return t(e).split(".")[0] ?? "";
|
|
113
|
+
}
|
|
114
|
+
//#endregion
|
|
115
|
+
export { p as WasmSqlRunner };
|
package/dist/sql/remark.d.ts
CHANGED
|
@@ -36,6 +36,17 @@ export interface RunnableSqlConfig {
|
|
|
36
36
|
* iframe, so scripts run with an opaque origin.
|
|
37
37
|
*/
|
|
38
38
|
show?: 'table' | 'html' | 'iframe' | 'svg' | 'text';
|
|
39
|
+
/**
|
|
40
|
+
* What this block is expected to do when the docs' own SQL test suite runs it
|
|
41
|
+
* (`duckfn-docs-kit/sql/verify`). Defaults to `'ok'`; `'error'` marks a block
|
|
42
|
+
* that demonstrates a failure — the suite then *requires* it to fail, and
|
|
43
|
+
* reports it when it unexpectedly succeeds instead.
|
|
44
|
+
*
|
|
45
|
+
* This is the only source of truth for the expectation: the prose around a
|
|
46
|
+
* block, and a `-- error: …` comment inside it, are there for readers, and
|
|
47
|
+
* neither is machine-checked.
|
|
48
|
+
*/
|
|
49
|
+
expect?: 'ok' | 'error';
|
|
39
50
|
/**
|
|
40
51
|
* The column holding the markup, for the preview renderers. A single-column
|
|
41
52
|
* result is unambiguous and is used as-is.
|
|
@@ -85,4 +96,13 @@ export interface RunnableSqlOptions {
|
|
|
85
96
|
}
|
|
86
97
|
/** The custom element the plugin emits; must match `register.ts`. */
|
|
87
98
|
export declare const DFK_SQL_TAG = "dfk-sql";
|
|
99
|
+
/**
|
|
100
|
+
* Parse the metastring; `null` means "not a runnable block, leave it alone".
|
|
101
|
+
*
|
|
102
|
+
* Exported because the block contract has two consumers: this plugin, which
|
|
103
|
+
* turns a block into `<dfk-sql>` at build time, and `sql/verify` (via
|
|
104
|
+
* `sql/collect`), which runs those same blocks in CI. Both have to agree on
|
|
105
|
+
* what counts as runnable, so there is one parser.
|
|
106
|
+
*/
|
|
107
|
+
export declare function parseRunnableSqlMeta(meta: string | null | undefined): RunnableSqlConfig | null;
|
|
88
108
|
export declare const remarkRunnableSql: Plugin<[RunnableSqlOptions?]>;
|
package/dist/sql/remark.js
CHANGED
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
import { type WasmPlatform } from './nodeRunner';
|
|
2
|
+
export interface VerifyOptions {
|
|
3
|
+
/** Docs site root; defaults to the working directory. */
|
|
4
|
+
siteDir?: string;
|
|
5
|
+
/** Content directories relative to the site root; defaults to the site layout. */
|
|
6
|
+
contentDirs?: readonly string[];
|
|
7
|
+
/**
|
|
8
|
+
* The extension to `LOAD`: a path, or an absolute `http(s)` URL. Defaults to
|
|
9
|
+
* the single file under `<siteDir>/static/duckdb-extensions/`.
|
|
10
|
+
*/
|
|
11
|
+
extension?: string;
|
|
12
|
+
/** DuckDB-Wasm platform, which must match the extension build. */
|
|
13
|
+
platform?: WasmPlatform;
|
|
14
|
+
/** Engine wasm override, for pinning a specific DuckDB-Wasm build. */
|
|
15
|
+
engine?: string;
|
|
16
|
+
/** Per-block timeout in milliseconds; a hang is reported instead of blocking CI. */
|
|
17
|
+
timeoutMs?: number;
|
|
18
|
+
/**
|
|
19
|
+
* Directory the blocks run in. Defaults to a fresh temporary directory that
|
|
20
|
+
* is removed afterwards: a block may `COPY … TO 'a.csv'`, and on Node
|
|
21
|
+
* DuckDB's file system is the real one, relative to the working directory.
|
|
22
|
+
*/
|
|
23
|
+
workingDir?: string;
|
|
24
|
+
/** Write the full result list here as JSON. */
|
|
25
|
+
reportFile?: string;
|
|
26
|
+
}
|
|
27
|
+
/** What happened to a block, judged against what it declared. */
|
|
28
|
+
export type BlockOutcome =
|
|
29
|
+
/** Ran, and was expected to run. */
|
|
30
|
+
'ok'
|
|
31
|
+
/** Failed, and declared `"expect": "error"`. */
|
|
32
|
+
| 'error-as-expected'
|
|
33
|
+
/** Failed, but was expected to run. */
|
|
34
|
+
| 'unexpected-error'
|
|
35
|
+
/** Ran, but declared `"expect": "error"`. */
|
|
36
|
+
| 'unexpected-success';
|
|
37
|
+
export interface BlockResult {
|
|
38
|
+
file: string;
|
|
39
|
+
line: number;
|
|
40
|
+
outcome: BlockOutcome;
|
|
41
|
+
detail: string;
|
|
42
|
+
}
|
|
43
|
+
export interface VerifyReport {
|
|
44
|
+
blocks: BlockResult[];
|
|
45
|
+
/** Blocks that behaved as declared: they ran, or they failed as declared. */
|
|
46
|
+
asDeclared: BlockResult[];
|
|
47
|
+
/** Blocks that did not behave as declared — the ones that should fail CI. */
|
|
48
|
+
unexpected: BlockResult[];
|
|
49
|
+
}
|
|
50
|
+
/** Runs every runnable block of the site and returns the outcome of each. */
|
|
51
|
+
export declare function verifySqlDocs(options?: VerifyOptions): Promise<VerifyReport>;
|
|
52
|
+
export interface CliOptions extends VerifyOptions {
|
|
53
|
+
quiet?: boolean;
|
|
54
|
+
}
|
|
55
|
+
/**
|
|
56
|
+
* `duckfn-sql-verify` — the command line around {@link verifySqlDocs}.
|
|
57
|
+
*
|
|
58
|
+
* Exit code 1 when a block did not behave as it declared, so a docs site can
|
|
59
|
+
* wire it straight into `npm test`.
|
|
60
|
+
*/
|
|
61
|
+
export declare function cliMain(argv: readonly string[]): Promise<void>;
|
|
@@ -0,0 +1,148 @@
|
|
|
1
|
+
import { collectRunnableSql as e, expectsError as t } from "./collect.js";
|
|
2
|
+
import { WasmSqlRunner as n } from "./nodeRunner.js";
|
|
3
|
+
import { join as r, resolve as i } from "node:path";
|
|
4
|
+
import { mkdirSync as a, mkdtempSync as o, readdirSync as s, rmSync as c, statSync as l, writeFileSync as u } from "node:fs";
|
|
5
|
+
import { tmpdir as d } from "node:os";
|
|
6
|
+
//#region src/sql/verify.ts
|
|
7
|
+
var f = ["docs", "i18n"], p = 3e4;
|
|
8
|
+
async function m(t = {}) {
|
|
9
|
+
let n = i(t.siteDir ?? process.cwd()), s = e({
|
|
10
|
+
siteDir: n,
|
|
11
|
+
contentDirs: t.contentDirs ?? v(n)
|
|
12
|
+
}), l = t.extension, f = l ? /^https?:\/\//i.test(l) ? l : i(l) : y(n), m = t.timeoutMs ?? p, g = t.workingDir ? i(t.workingDir) : o(r(d(), "duckfn-sql-verify-"));
|
|
13
|
+
a(g, { recursive: !0 });
|
|
14
|
+
let _ = process.cwd();
|
|
15
|
+
process.chdir(g);
|
|
16
|
+
let b;
|
|
17
|
+
try {
|
|
18
|
+
b = await h(s, f, t, m);
|
|
19
|
+
} finally {
|
|
20
|
+
process.chdir(_), t.workingDir || c(g, {
|
|
21
|
+
recursive: !0,
|
|
22
|
+
force: !0
|
|
23
|
+
});
|
|
24
|
+
}
|
|
25
|
+
let x = {
|
|
26
|
+
blocks: b,
|
|
27
|
+
asDeclared: b.filter((e) => e.outcome === "ok" || e.outcome === "error-as-expected"),
|
|
28
|
+
unexpected: b.filter((e) => e.outcome === "unexpected-error" || e.outcome === "unexpected-success")
|
|
29
|
+
};
|
|
30
|
+
return t.reportFile && u(t.reportFile, `${JSON.stringify(x, null, 2)}\n`), x;
|
|
31
|
+
}
|
|
32
|
+
async function h(e, t, r, i) {
|
|
33
|
+
let a = await n.create({
|
|
34
|
+
extension: t,
|
|
35
|
+
platform: r.platform,
|
|
36
|
+
engine: r.engine
|
|
37
|
+
}), o = [];
|
|
38
|
+
try {
|
|
39
|
+
let t = null;
|
|
40
|
+
for (let n of e) n.file !== t && (t = n.file, await a.newPage()), o.push(await g(a, n, i));
|
|
41
|
+
} finally {
|
|
42
|
+
await a.close();
|
|
43
|
+
}
|
|
44
|
+
return o;
|
|
45
|
+
}
|
|
46
|
+
async function g(e, n, r) {
|
|
47
|
+
let i = t(n.config);
|
|
48
|
+
try {
|
|
49
|
+
let t = await _(e.run(n.sql), r);
|
|
50
|
+
return {
|
|
51
|
+
file: n.file,
|
|
52
|
+
line: n.line,
|
|
53
|
+
outcome: i ? "unexpected-success" : "ok",
|
|
54
|
+
detail: i ? `declared "expect": "error" but succeeded (${t.rows}×${t.columns})` : `ok (${t.rows}×${t.columns})`
|
|
55
|
+
};
|
|
56
|
+
} catch (e) {
|
|
57
|
+
let t = String(e?.message ?? e).split("\n")[0] ?? "";
|
|
58
|
+
return {
|
|
59
|
+
file: n.file,
|
|
60
|
+
line: n.line,
|
|
61
|
+
outcome: i ? "error-as-expected" : "unexpected-error",
|
|
62
|
+
detail: t
|
|
63
|
+
};
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
function _(e, t) {
|
|
67
|
+
let n, r = new Promise((e, r) => {
|
|
68
|
+
n = setTimeout(() => r(/* @__PURE__ */ Error(`timed out after ${t}ms`)), t);
|
|
69
|
+
});
|
|
70
|
+
return Promise.race([e, r]).finally(() => clearTimeout(n));
|
|
71
|
+
}
|
|
72
|
+
function v(e) {
|
|
73
|
+
let t = [];
|
|
74
|
+
b(r(e, f[0])) && t.push(f[0]);
|
|
75
|
+
let n = r(e, f[1]);
|
|
76
|
+
if (b(n)) for (let e of s(n)) b(r(n, e, "docusaurus-plugin-content-docs", "current")) && t.push(r("i18n", e, "docusaurus-plugin-content-docs", "current"));
|
|
77
|
+
return t.length > 0 ? t : ["."];
|
|
78
|
+
}
|
|
79
|
+
function y(e) {
|
|
80
|
+
let t = r(e, "static", "duckdb-extensions"), n = b(t) ? s(t).filter((e) => e.endsWith(".duckdb_extension.wasm")) : [];
|
|
81
|
+
if (n.length === 0) throw Error(`sql/verify: no extension found in ${t} — pass --extension <file|url>, or let the site's extension preload plugin fetch it first`);
|
|
82
|
+
if (n.length > 1) throw Error(`sql/verify: several extensions found in ${t} (${n.join(", ")}) — pass --extension to pick one`);
|
|
83
|
+
return r(t, n[0]);
|
|
84
|
+
}
|
|
85
|
+
function b(e) {
|
|
86
|
+
return l(e, { throwIfNoEntry: !1 })?.isDirectory() ?? !1;
|
|
87
|
+
}
|
|
88
|
+
async function x(e) {
|
|
89
|
+
let t = C(e);
|
|
90
|
+
if (t.help) {
|
|
91
|
+
process.stdout.write(S);
|
|
92
|
+
return;
|
|
93
|
+
}
|
|
94
|
+
let n = Date.now(), r = await m(t), i = ((Date.now() - n) / 1e3).toFixed(1), a = r.blocks.filter((e) => e.outcome === "error-as-expected").length;
|
|
95
|
+
if (t.quiet || (process.stdout.write(`\n${r.blocks.length} block(s) in ${i}s\n`), process.stdout.write(` ${r.asDeclared.length} as declared (${a} erroring on purpose), ${r.unexpected.length} unexpected\n`)), r.unexpected.length > 0) {
|
|
96
|
+
process.stdout.write("unexpected behaviour:\n");
|
|
97
|
+
for (let e of r.unexpected) process.stdout.write(`- ${e.file}:${e.line} [${e.outcome}] :: ${e.detail}\n`);
|
|
98
|
+
process.exitCode = 1;
|
|
99
|
+
}
|
|
100
|
+
}
|
|
101
|
+
var S = "Usage: duckfn-sql-verify [options]\n\nRuns every runnable SQL block of a duckfn docs site in DuckDB-Wasm, and checks\nthat each one behaves as its own metadata declares (\"expect\": \"error\" for a\nblock that demonstrates a failure).\n\n --site <dir> Docs site root (default: the working directory)\n --content <dir> Content directory, relative to the site root (repeatable;\n default: docs/ plus every i18n/<locale>/… translation)\n --extension <path> The extension to preload: a .duckdb_extension.wasm path,\n or an absolute http(s) URL (default: the single file under\n static/duckdb-extensions/)\n --platform <eh|mvp> DuckDB-Wasm bundle, which must match the extension build\n (default: eh)\n --engine <path> Engine wasm override\n --timeout <ms> Per-block timeout (default: 30000)\n --working-dir <dir> Directory the blocks run in (default: a temporary one)\n --report <file> Write the full result list as JSON\n --quiet Only report unexpected behaviour\n --help Show this help\n";
|
|
102
|
+
function C(e) {
|
|
103
|
+
let t = {}, n = [];
|
|
104
|
+
for (let r = 0; r < e.length; r++) {
|
|
105
|
+
let i = e[r], a = () => {
|
|
106
|
+
let t = e[++r];
|
|
107
|
+
if (t === void 0) throw Error(`sql/verify: ${i} needs a value`);
|
|
108
|
+
return t;
|
|
109
|
+
};
|
|
110
|
+
switch (i) {
|
|
111
|
+
case "--site":
|
|
112
|
+
t.siteDir = a();
|
|
113
|
+
break;
|
|
114
|
+
case "--content":
|
|
115
|
+
n.push(a());
|
|
116
|
+
break;
|
|
117
|
+
case "--extension":
|
|
118
|
+
t.extension = a();
|
|
119
|
+
break;
|
|
120
|
+
case "--platform":
|
|
121
|
+
t.platform = a();
|
|
122
|
+
break;
|
|
123
|
+
case "--engine":
|
|
124
|
+
t.engine = a();
|
|
125
|
+
break;
|
|
126
|
+
case "--timeout":
|
|
127
|
+
t.timeoutMs = Number(a());
|
|
128
|
+
break;
|
|
129
|
+
case "--working-dir":
|
|
130
|
+
t.workingDir = a();
|
|
131
|
+
break;
|
|
132
|
+
case "--report":
|
|
133
|
+
t.reportFile = a();
|
|
134
|
+
break;
|
|
135
|
+
case "--quiet":
|
|
136
|
+
t.quiet = !0;
|
|
137
|
+
break;
|
|
138
|
+
case "--help":
|
|
139
|
+
case "-h":
|
|
140
|
+
t.help = !0;
|
|
141
|
+
break;
|
|
142
|
+
default: throw Error(`sql/verify: unknown option ${i}`);
|
|
143
|
+
}
|
|
144
|
+
}
|
|
145
|
+
return n.length > 0 && (t.contentDirs = n), t;
|
|
146
|
+
}
|
|
147
|
+
//#endregion
|
|
148
|
+
export { x as cliMain, m as verifySqlDocs };
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "duckfn-docs-kit",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.2.0",
|
|
4
4
|
"description": "Shared Docusaurus building blocks (TOC toggle, home-page web components, brand tokens, remark version placeholder) for duckfn-family extension docs sites.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"keywords": [
|
|
@@ -24,6 +24,9 @@
|
|
|
24
24
|
"type": "module",
|
|
25
25
|
"main": "./dist/index.js",
|
|
26
26
|
"types": "./dist/index.d.ts",
|
|
27
|
+
"bin": {
|
|
28
|
+
"duckfn-sql-verify": "./bin/sql-verify.mjs"
|
|
29
|
+
},
|
|
27
30
|
"exports": {
|
|
28
31
|
".": {
|
|
29
32
|
"types": "./dist/index.d.ts",
|
|
@@ -38,6 +41,7 @@
|
|
|
38
41
|
"files": [
|
|
39
42
|
"dist",
|
|
40
43
|
"src",
|
|
44
|
+
"bin",
|
|
41
45
|
"AGENTS.md"
|
|
42
46
|
],
|
|
43
47
|
"scripts": {
|